Assessment and Plagiarism – Research Opportunities

As one of my previous posts has indicated, it is rare for assessment and plagiarism to considered as equal topics within educational research.

The book chapter, Assessment and Plagiarism, by Thomas Lancaster (me), Anthony Robins and Sally Fincher addresses that issue for the computing discipline. It is part of The Cambridge Handbook for Computing Education Research, a book that “describes the extent and shape of computing education research today“.

As well as discussing the importance of assessment and taking steps to minimise plagiarism, the chapter focuses specifically on techniques that are most suitable for computing. The chapter also provides recommendations for future research in the field.

In this post, I’ve picked out five ideas for research opportunities from the chapter that have implications for multiple disciplines (beyond Computing). Of course, you should still read the full chapter for more ideas and a lot of background that will help any future research plans (and make the literature review sections of papers much easier to complete).

Collated and Reusable Assessments

In previous years, there have been pushes across the sector to build up collections of reusable learning components, including assessment banks intended for wider use. How well are those projects working? What measures are taken to keep the assessment banks up to date? Do students and educators see value in these activities continuing? And how can plagiarism and contract cheating be avoided with these standard assessments?

Essay Spinning

This isn’t a new topic for the blog (see these posts), but is still one that hasn’t been widely investigated. When a student automatically converts one version of an essay to another, perhaps through back translation, how can this plagiarism be detected? Are there indicators that academics should be looking out for when they are marking? Or are indicators that a machine could identify? Failing that, could multiple versions of an assignment be generated in multiple languages to use with text matching software?

Academic Integrity Processes

It is thought that these still vary greatly across the sector. Is that the case? More specifically, what about at discipline level? Are processes applied consistently and are penalties (when necessary) given out in a fair manner? What recommendations exist for best practice at a discipline level?

Gamification of Assessment

Gamification techniques are now widely used across many walks of life, everything from encouraging continued play of computer games to getting people to continue to shop in certain ways. How far will these techniques work with assessment? Are there methods that will make assessment more engaging and encourage students to develop their understanding to a more in-depth level than they otherwise would have done?

Automated Assessment

Many methods have been developed to reduce the burden of assessment on educators, including using automated techniques that have different levels of success. At one end of the scale, there are systems that will automatically mark essays, although this is usually through metric based assessment writing style and keyword analysis of content. There are also many systems for marking simple exam questions, such as multiple choice and short answer questions. Can these systems be developed further? Can better feedback be developed? There are also many ethical questions worthy of investigation such as, is it fair on students to have their work marked in this way?

 

Feel free to share your own ideas for good topics for future assessment and plagiarism research in the comments section.

Contract Cheating in Health Courses (Unpublished Paper)

Here is an unpublished full paper that was otherwise gathering virtual dust on my hard drive.

Examining Contract Cheating, Essay Mill Use and Academic Misconduct by Students on Health Courses – from 2015 – Thomas Lancaster and Robert Clarke
(Full Text including PDF Download on ResearchGate)

I originally wrote this in 2015 based on a talk that Robert Clarke and I delivered. The problem of contract cheating in health and nursing education was prominent then and I have addressed this in subsequent talks and on this blog.

The paper was never submitted for publication, as the conference I was originally aiming this at didn’t run and I haven’t subsequently seen the right outlet. Looking back at the paper with 2018 eyes, it would need a substantial rewrite to fit suitably into the current academic integrity climate. This would include updating the sources and examples, so that it was substantially a new paper.

Due to this, I am providing the original unpublished paper here as drafted in the hope that it may be useful to researchers instead.

Abstract

When students obtain academic awards in the health industry that they do not deserve, they may emerge unfit for professional practice. This paper explores the challenges posed by academic misconduct in public-facing health fields, such as nursing and medicine. Specifically, the paper explores contract cheating, where students employ a third party or ghostwriter to complete assessed work. The problem appears more crucial in health than some other academic disciplines, since here fitness for practice is important and human lives may be at stake.

The paper argues about the importance of academic integrity in health through multiple examples. This includes showcasing media cases where medical professionals have been put in positions which their skills did not warrant and giving three specific examples of attempts by students to cheat that have been detected online. The examples demonstrate that such contract cheating starts before students arrive at university. This misconduct continues throughout their academic career up to postgraduate level. The overall findings in this field support the view that contract cheating is habitual and repeated regularly by some students.

Several sources are used to show that contract cheating in health is amongst the most popular subjects that students cheat on. Other examples show that original essays and assessments can be purchased by students for affordable prices. These essays will not be detected as unoriginal by Turnitin. The paper concludes by arguing that increased academic pressure is needed to change the wider health culture that is affording contract cheating.

There is still a need for research in this field. In particular, this includes gathering more data and implementing subject specific solutions. I would like to look back at this area again as time and opportunities permit.

I’m always open to speak on contract cheating and essay mill use in health education (you can contact me here).

 

 

 

Introverts In Higher Education Teaching

Is it a good thing to be an introvert when working as a teacher or professor in higher education?

It’s quite a common question, particularly when people think of school teachers needing to demonstrate their personality, engage their pupils and keep control in the classroom.

This question was originally asked on Quora, where I like to provide answers to interesting questions when I have time. I answered the question on introverts in teaching on Quora and this post is an edited and extended version of the answer I gave there.

If you prefer video, I’ve also provided a (different) answer on my YouTube account, which you can see embedded below.

Do Introverts Make Good University Lecturers?

(or good University Professors if you prefer the American terminology)

The short answer to this question is yes.

I’ve worked in a variety of higher education/university roles and they include teaching and lecturing. I enjoy speaking and helping students. And I, like many of my colleagues are very much an introvert.

If anything, I think that introverts have an easier time working in university lecturing roles than extroverts do. This can vary very slightly by subject, but generally introverts have the most natural set of skills for success.

Most lecturers are naturally introverts, myself included. It’s part and parcel of the type of person who enters the profession and is willing to make the sacrifices for study to reach that point. It also reflects the wider responsibilities of being a lecturer.

Lecturers typically get qualified as being able to teach by being educated up to PhD level. That’s gaining a doctorate, so the same number of years of study as a medical doctor (and often one year more, as many lecturers in the UK will have taken an additional years at Masters level).

There are short teaching courses that Lecturers often take too, but these are mostly done after having started teaching. It’s rather a “learn on the job” profession.

Gaining a PhD requires great immersion in a subject discipline. It means that you have to demonstrate that you can think and provide original knowledge in detail. As well as conducting research, as this is primarily intended as research training for a career, you have to document this in a formal thesis and pass a challenging viva examination.

You may be able to picture PhD students spending three or four years sharing a large office and not distracting the others. All deep in concentration and getting on with their unique studies. It’s an environment where extroverts really have to reign in their behaviour.

Career Management

Throughout their career, those people who both get through their PhD and get one of the limited jobs as lecturers have had to work hard. Often, there’s a gap between completing a PhD and gaining a lectureship, where the individual continues to work on short-term research projects to build up their reputation. It’s not the most secure work. Some people don’t get their first lectureship until their 40s or 50s. Those people who make it have shown dedication.

The appointed lecturers then have to balance multiple responsibilities. Typically, these include teaching, research, administrative functions and external engagement, although the balance between those will vary between individuals. Teaching is, of course, very important. The ability to inspire students and communicate knowledge is essential, but you don’t need to be an extrovert to do this.

For most professors, large group teaching will only be part of the role and not where the majority of time is spent. After all, those years building up research ability and credibility have to count for something.

The requirements to conduct research continue. By this stage of their career professors are typically busy writing research grants and books. Many administrative aspects also require deep and private concentration.

There are some externally facing responsibilities, for instance in my case I work a lot with external companies, present research, deliver training and work in student recruitment, but these are all manageable and enjoyable. Being an introvert does not mean that you dislike these activities, just that you can’t do them continually without quieter reflective time. It is also very different being in a controlling position able to shape a class of people who are looking at you for guidance than being hostage to being a member of a large and noisy class.

And, even within teaching, there are opportunities for introverts to work with small groups. These include activities like supervising students on individual projects, which is very enjoyable work.

The Changing World Of The University Lecturer

I want to end this blog post with a world of caution.

The academic world has changed. Many people joined academia for the autonomy, but the profession is now much more target and metric driven, similar to a commercial organisation.

Like many jobs, academia doesn’t automatically offer lecturers tenure and job security any more. Many lecturers are employed on short-term contracts, which are only renewed if they meet their targets. Lecturers also have to move positions if they’re looking for promotion. So, lecturers changing universities every few years is common now.

That can be quite stressful and not everyone enjoys that, regardless of whether they’re an introvert or extrovert.

The moves can also be exciting, with the opportunity to meet new people, teach new subjects and get immersed in different university cultures. But this can also be distracting when you’re trying to complete research projects.

There are also some aspects of academia where it is beneficial to be an extrovert. For instance, these may include later career managerial roles or working in services like marketing. There are disciplines where lecturers are likely to join as a second career from an industry background too and these attract a higher percentage of extroverts. Nursing lecturers and journalism lecturers are examples.

Overall though, providing you have a plan of sustainable academic research, are good at multi-tasking, cope well with a changing profession and, most importantly, want to help students, then yes, aiming to be a university lecturer offers a good career choice for an introvert.

 

Emerging Issues In Plagiarism Prevention And Detection – My View From 2004

Writing About Plagiarism In 2004

My hard disk contains quite an archive of material I’ve prepared, but which has never seen the light of the day. Some of it is good, some of it deserves to be formally completed, some of it I could never quite work into a shape that I was happy with at the time.

I want to share with you some extracts from a partial paper I wrote in early 2004. All the quoted text is presented, unedited, just I left it in the draft over 13 years ago. Had this paper been completed in a form I was happy with, the choice of words would likely have gone through further fine tuning.

To put these extracts into their historical context as part of my research journey, I completed my PhD on plagiarism detection in 2003. Later that year began working as a Lecturer in Computing at the University of Central England (now known as Birmingham City University).

My working title for the paper was “Fresh Issues in Plagiarism Prevention and Detection” and the paper was constructed to:

discuss the issues that will be relevant to plagiarism prevention and detection in the near future

as well as to:

inform the directions in which it is necessary for future research to proceed

The planned paper ended up taking a back seat with the pressures of adapting to the pressures of a new teaching intensive environment. My subsequent research efforts ended up going in a different direction.

In hindsight, perhaps this paper did deserve to have been completed. My experience is that this type of paper tends to be well-received.

An earlier paper of mine, Plagiarism Issues In Higher Education, which I wrote alongside my PhD, is one of my most cited papers. That is despite this being one of the first papers I wrote – and also one of the easiest to write. I presume that, being introductory in nature, meant that this paper was accessible by a wide audience.

Now, I tend to publish material of this type as blog posts. Perhaps not the best strategy if the results also suitable for citation…

Eight Plagiarism Issues

The draft I wrote in 2004 included updated ideas from my PhD thesis together with observations I’d made during the intervening year.

Here’s what I said in the draft paper…

Eight main issues have been identified that are worthy of further investigation. These include both issues of academic and practical interest.

The issues are:
 
outsourced submissions – has work submitted by a student actually been produced by that student?
 
ownership of work – is it both legal and ethical to submit work from students to detection services.
 
tool usability – there are many technical solutions available to find out if work is similar to another source, but are those tools produced to ease tutor workload?
 
extent of cheating – conflicting evidence exists stating how common cheating is, can parity exist between different subjects and different researchers?
 
policy – how far is reducing the level of plagiarism and the methods to deal with plagiarisers related to appropriate from the upper echelon of an academic institute?
 
earlier exposure – are students plagiarising due to practices accepted in further education being condoned in higher education and, if so, can what are the solutions?>
 
transparency – how far can students see that a due-process is being followed for plagiarism prevention and detection?
 
open source detection – does an institution committing themselves to commercial detection technology hinder them in long term planning?

 
Although my writing style has developed since and I would likely use more supportive language, many of these issues are still equally relevant today.

The issue referred to as outsourced assessment, of course, has been much developed in the form of research into contract cheating. The particular example given, that is authenticating if the author of an assessment solution and the student submitting this for academic credit are the same person, has still not been solved for anything other than very specific cases.

User Experience

I’d like to pick up on one of the 2004 issues as worthy of more immediate attention.

Whether or not software tools for plagiarism detection are optimised for user experience continues to be questionable. The fact that similarity reports are often misinterpreted – and that users cannot always differentiate between similarity and plagiarism, suggests otherwise.

Much valuable progress has been made since 2004 on working with students as academic integrity partners. That includes supporting students in developing their academic writing by providing them with controlled access to appropriate software tools, such as those that show similarity. I have seen far too many tweets where students are boasting about getting their similarity (plagiarism) score down to an unrealistic level.

Improving the usability of support tools, for instance by making the results more readable and the practical steps to take more intuitive, is now important for students too.

The user interfaces for originality checking software tools do not seem to have evolved, in any real sense, since the first commercial providers came onto the market. There is an opportunity for thought leadership here.

One of the major challenges for academics investigating possible non-originality is taking the output from a tool and converting this into a format considered acceptable for a university academic misconduct panel. Often, panels still require information in a printed format and I know of academics who have had to spend many hours laboriously marking sources up by hand. This is an area which is ripe for improving the user experience.

There is certainly the opportunity for a PhD to look at redefining these user interfaces. If you would be interested in working on that area, under my supervision, please contact me and let me know.

I also believe that the potential exists for artificial intelligence techniques to be used to provide personalised help for a user accessing a similarity report. Such AI could be used to consider whether or not similarity is likely to represent plagiarism and where in the document a user should focus their priority (whether this is a student learning academic writing who has forgotten to cite their source, or a tutor investigating possible plagiarism).

Plagiarism Prevention And Detection Issues Of 2017 And 2018

What are the main issues that exist for individuals researching plagiarism prevention and plagiarism detection today? Is it appropriate to consider issues previously identified, such as my ideas from 2004, during the production of a more up to date list?

Which of the many issues then most deserve to be quickly addressed?

Do feel free to share your thoughts using the Comment box at the end of the post.

(and now that I have extracted some value from them, the rest of my outdated materials from “Fresh Issues in Plagiarism Prevention and Detection” can safely be moved to the Recycle Bin)

Cutting The Costs Of Open Access Research

Is it feasible to run a high quality open access journal with operating costs of just $6 USD (£4.50 GBP) per paper?

Other open access journals often charge upwards of $500 USD to get a paper reviewed and published, but $6 USD per paper is the model that has been proposed by Kyle Niemeyer.

$6 USD Per Paper?

I came across an interesting presentation from Kyle given at SciPy 2017 and also documented in a more traditional paper where he discussed the design and development of the Journal of Open Source Software (JOSS). This particular journal is used to archive software packages and largely exists within the $6 USD per paper cost range, although there’s no reason that a similar technique wouldn’t work for more traditional papers.

Kyle calculates the $6 USD per paper figure, which depends on the journal publishing 100 articles per year, as follows:

  • Crossref membership (needed for DOIs and journal indexing) = $275 USD per year + $1 USD per paper = $375 per year
  • Web hosting using Heroku = $19 USD per month = $228 per year

Total = $603 USD per year

(or $6.03 USD per paper)

The system looks to largely be dependent on GitHub.

As expected, many attendees at the open source conference where JOSS was discussed expressed positive views of the idea:

Subsequent discussion has however noted that there are some sacrifices needed to get the $6 USD per paper cost.

For instance, this requires heavily on volunteer labour, including from those people developing the software to “run” the journal in the first place. A lot of free work is put in by reviewers and editors, although that’s true of many open source journals. There may also be issues with creating redundancy in the system, which is something that’s important for the long-term archiving of academic papers.

At present, charges aren’t made directly to authors. The journal is relying on funding that has been put into it to cover the running costs. For this to be more sustainable in the longer-term, consideration to funding would need to be made, including all of the legal entity issues that come with handling money and the needs to guarantee service.

Alternative Approaches

There may also be ways to cut the costs still further. Martin Paul Eve suggests that CrossRef membership with 50 DOIs included could be possible for €75 per year (£66 GBP, $90 USD). He also recommends the use of the CLOCKSS archival service at $200 USD per year, which may solve the issue of needing reliable long-term service and archiving. He also suggests the use of Open Journal Systems, which could remove some of the technical complexity.

One idea that I’d like to see explored further would be more use of peer-to-peer hosting to archive academic papers. (Legal) torrent style services could be used which would also introduce some further redundancy into the system.

There could well be an interesting student project looking at putting these different approaches together in a way that is both cost-effective and allows for a new open access journal to be set up with the minimum possible technical complexity.

Taking all of these issues into account, it would be challenging for a journal to maintain a $6 USD publication point. But it should be possible to substantially cut the costs of open access publishing from the figures that researchers are charged by many journals today.

Page 1 of 6123456»