Attributing Contract Cheating Through The Turnitin Text Matching Service

Birmingham City University holds a research conference, Rescon, in December of each year, providing short accessible shorts from staff and PhD students about their research.

My session built on work presented to the Higher Education academy looking at how text matching algorithms supplied by Turnitin could be used as part of the attribution of contract cheating. This is the challenging process where a request to have assessed work completed can be found online, but the academic institution to which the work is associated is hard to identify.

The slides, can be accessed on my SlideShare account, or viewed below.

The talk led to a wide-ranging discussion about wider aspects of contract cheating, plagiarism and academic misconduct. There was clearly a lot of local interest in the use of Turnitin and the associated training as well as the reasons that students cheat and how that could be avoided.

The group also had a discussion about translation plagiarism. Although I did undertake some initial research on this a few year’s ago, this is still a problem and needs to be the topic of continued research. Several academics believed that they had seen attempts to cheat by taking work and automatically translating it into different languages. The actual behaviour that I have previously observed and researched is more subtle and this may also be something that I discuss in more detail on this site in the future.

Winning The Contract Cheating Battle

These slides come from a talk delivered for Engineering at the University of Sheffield. They contain many more examples specific to that academic discipline than my usual talks.

The slides, can be accessed on my SlideShare account, or viewed below.

The slides also contain details of the cheating scandal in Australia, including the link between YingCredible Tutoring and MyMaster – which is one of the largest known examples of contract cheating.

After the talk, we had a long discussion about how an assignment and test can be linked together, with lots of good ideas shared for Engineering. Such an approach means that a cheating student would still not be able to prosper, as even though they would complete the coursework element, they would struggle with the linked assessment.

Developing New Approaches For The Automated Detection Of Contract Cheating

Some of the recent published work by Ann Rogerson is of interest to those of us involved with contract cheating research. Ann has been looking at the indicators within texts that might show that work has not been written by the student who handed it in.

The work was presented at the 6th International Integrity and Plagiarism Conference. Unfortunately, I wasn’t able to attend and the full paper is not yet available online, but the abstract can be accessed here. There is also an easily accessible version of the findings courtesy of this Times Higher Education article on tips for detecting and beating cheating.

Indicators From The Cheating And Plagiarism Processes

During the contract cheating and plagiarism processes, there are often indicators that find their way into student work. For plagiarism from the web, this is the simplest, since the indicators relate to the site from which the work has been sourced. This is the same, whether the document is compiled in patchwork manner from multiple sites, or lifted directly from a site containing ready written essays.

For contract cheating, the process of finding indicators can be more difficult, but clues do left behind. Searching for these clues may well be a method which can be automated, or at least computer supported. However, there is still an underlying computer algorithm problem about how to identify these clues and how to minimise the number of false positive results that are obtained.

Some Indicators For Automated Investigation

One suggestion from the Times Higher Article is to look for inconsistencies within the text. For instance, this could be when the same document contains both perfect English and examples heavy with automated errors. Some of the ongoing work into using stylometrics to detect contract cheating could be useful here, which look for changes in writing style within a single document. Monitoring the quality of English would be a metric which could be tracked within different sections of a longer document. This would allow outliers to be shown.

Another possible indicator listed is that of “blandness”, tracking students who write in very generic terms and do not show any real insight. To me, that is mainly an issue of assessment design, as many of the best assignment briefs and marking schemes do insist upon that additional level of insight which makes taking work from a third party source difficult. How easy this would be to detect using an automated system is questionable. It may, for instance, be possible to develop a system to look for indicators that a student has local knowledge. The corresponding absence of such indicators could then provide a measure of blandness.

The final suggestion from Ann Rogerson is to check the correctness of references. Examples of made up references are given, although any intelligent computerised system needs to be wary of students who have created fictitious references and those who just have poor referencing skills. An additional complication here exists when students include legitimate references, but these do not actually support the information provided within the submitted text. Measures of the quality of referencing, and of the types of sources referenced, are already available. These techniques could be developed to use with automated systems. Alternatively, search mechanisms could be used to find common types of references that are not appropriate.

Further Considerations For Automated Contract Cheating Detection

There is an additional process consideration that is needed for the information in the article to be easily applied. First, there has to be a mechanism through which students can be called to verbally account for their work. This has to be supported by university cheating and plagiarism regulations. Many current regulations require the indication of a source document before cheating cases are considered. Just “a feeling” that the work was not written by the student would not be sufficient to allow a case or an interview to proceed.

Overall, automated detection of contract cheating is one area of research which is struggling, in terms of getting appropriate interest levels from researchers and in terms of finding techniques that work. Adding in new approaches like the ones suggested in this post, where several different indicators need to be found, may be the breakthrough that this field of research needs.

Contract Cheating In 2014 – What Online Cheating Is Happening Today

Here is a teaching focused talk on contract cheating I delivered for the Computer Science department at the University of Sheffield.

The slides, can be accessed on my SlideShare account, or viewed below.

There are several new and recent examples in this presentation, with less direct focus on research results. One example includes a sting operation from Local 6 News in Florida, who commissioned original work and caught the writer being paid on camera.

Some suggestions about how the Computing community can help to stop the impact of contract cheating are also provided. I believe that there is certainly scope for technical approaches for detection to be developed, but that sensible methods of setting assignments should also be considered.

An Initial Analysis Of The Contextual Information Available Within Auction Posts On Contract Cheating Agency Websites Video Presentation

Here is a presentation of a recent paper I coauthored with Robert Clarke presented in video form. The presentation muses on the detection opportunities afforded to contract cheating through a consideration of the wider context on which requests on agency sites are made.

The video is also available on the YouTube account for Thomas Lancaster

Automated methods of detecting contract cheating are one of the areas in which there is currently a real need. Whilst people like Robert Clarke provide a human detection process, this can never be 100% successful. For instance, there’s no way that a human can be continually monitoring every post on an agency site such as Freelancer.

Whilst an intelligent system would not be a sole solution to this problem, since it would require human checking, it would also increase the consistency through which posts are monitored, and would allow more information from such sites to be captured.

Page 10 of 13« First...«8910111213»