Attributing Contract Cheating Through The Turnitin Text Matching Service

Birmingham City University holds a research conference, Rescon, in December of each year, providing short accessible shorts from staff and PhD students about their research.

My session built on work presented to the Higher Education academy looking at how text matching algorithms supplied by Turnitin could be used as part of the attribution of contract cheating. This is the challenging process where a request to have assessed work completed can be found online, but the academic institution to which the work is associated is hard to identify.

The slides, can be accessed on my SlideShare account, or viewed below.

The talk led to a wide-ranging discussion about wider aspects of contract cheating, plagiarism and academic misconduct. There was clearly a lot of local interest in the use of Turnitin and the associated training as well as the reasons that students cheat and how that could be avoided.

The group also had a discussion about translation plagiarism. Although I did undertake some initial research on this a few year’s ago, this is still a problem and needs to be the topic of continued research. Several academics believed that they had seen attempts to cheat by taking work and automatically translating it into different languages. The actual behaviour that I have previously observed and researched is more subtle and this may also be something that I discuss in more detail on this site in the future.

Winning The Contract Cheating Battle

These slides come from a talk delivered for Engineering at the University of Sheffield. They contain many more examples specific to that academic discipline than my usual talks.

The slides, can be accessed on my SlideShare account, or viewed below.

The slides also contain details of the cheating scandal in Australia, including the link between YingCredible Tutoring and MyMaster – which is one of the largest known examples of contract cheating.

After the talk, we had a long discussion about how an assignment and test can be linked together, with lots of good ideas shared for Engineering. Such an approach means that a cheating student would still not be able to prosper, as even though they would complete the coursework element, they would struggle with the linked assessment.

Developing New Approaches For The Automated Detection Of Contract Cheating

Some of the recent published work by Ann Rogerson is of interest to those of us involved with contract cheating research. Ann has been looking at the indicators within texts that might show that work has not been written by the student who handed it in.

The work was presented at the 6th International Integrity and Plagiarism Conference. Unfortunately, I wasn’t able to attend and the full paper is not yet available online, but the abstract can be accessed here. There is also an easily accessible version of the findings courtesy of this Times Higher Education article on tips for detecting and beating cheating.

Indicators From The Cheating And Plagiarism Processes

During the contract cheating and plagiarism processes, there are often indicators that find their way into student work. For plagiarism from the web, this is the simplest, since the indicators relate to the site from which the work has been sourced. This is the same, whether the document is compiled in patchwork manner from multiple sites, or lifted directly from a site containing ready written essays.

For contract cheating, the process of finding indicators can be more difficult, but clues do left behind. Searching for these clues may well be a method which can be automated, or at least computer supported. However, there is still an underlying computer algorithm problem about how to identify these clues and how to minimise the number of false positive results that are obtained.

Some Indicators For Automated Investigation

One suggestion from the Times Higher Article is to look for inconsistencies within the text. For instance, this could be when the same document contains both perfect English and examples heavy with automated errors. Some of the ongoing work into using stylometrics to detect contract cheating could be useful here, which look for changes in writing style within a single document. Monitoring the quality of English would be a metric which could be tracked within different sections of a longer document. This would allow outliers to be shown.

Another possible indicator listed is that of “blandness”, tracking students who write in very generic terms and do not show any real insight. To me, that is mainly an issue of assessment design, as many of the best assignment briefs and marking schemes do insist upon that additional level of insight which makes taking work from a third party source difficult. How easy this would be to detect using an automated system is questionable. It may, for instance, be possible to develop a system to look for indicators that a student has local knowledge. The corresponding absence of such indicators could then provide a measure of blandness.

The final suggestion from Ann Rogerson is to check the correctness of references. Examples of made up references are given, although any intelligent computerised system needs to be wary of students who have created fictitious references and those who just have poor referencing skills. An additional complication here exists when students include legitimate references, but these do not actually support the information provided within the submitted text. Measures of the quality of referencing, and of the types of sources referenced, are already available. These techniques could be developed to use with automated systems. Alternatively, search mechanisms could be used to find common types of references that are not appropriate.

Further Considerations For Automated Contract Cheating Detection

There is an additional process consideration that is needed for the information in the article to be easily applied. First, there has to be a mechanism through which students can be called to verbally account for their work. This has to be supported by university cheating and plagiarism regulations. Many current regulations require the indication of a source document before cheating cases are considered. Just “a feeling” that the work was not written by the student would not be sufficient to allow a case or an interview to proceed.

Overall, automated detection of contract cheating is one area of research which is struggling, in terms of getting appropriate interest levels from researchers and in terms of finding techniques that work. Adding in new approaches like the ones suggested in this post, where several different indicators need to be found, may be the breakthrough that this field of research needs.

Six Lessons From The Development Of My First Set Of Podcasts

As part of a recent project with the Higher Education Academy, I developed a set of audio podcasts looking at how students can create their own Professional Online Presence to aid in their employability.

The podcasts created are available here, and a blog detailing their production is also available.

During the time podcasting, I’ve come up with a set of six lessons which I think will help anyone looking to follow in my footsteps.

Overview of the lessons from developing podcasts for the Higher Education Academy

 

Lesson One – Podcasts should be developed as a series of short audios (10 to 20) minutes

I shied away from the idea of long audios or an open-ended series, since planning out a short sequence of podcasts is much easier. Each podcast can also serve a very clearly defined purpose, so this makes them simpler to record. Any extra ideas are better covered in an additional podcast, rather than making each one longer. Adapting a short section of a longer lecture can work well.

There are educational advantages to this too, since modern learners engage better with smaller chunks (and there is nothing to stop someone from choosing to listen to two or more podcasts together if they feel so inclined).

 

Lesson Two – Don’t just audio record lectures and assume that they will be “good enough”. Repurposing and rerecording parts of existing lecture notes works for some subjects

Much of the research I read suggested that just recording a lecture was a bad idea. My own experience of listening to recordings of live events concurs with this.

I think a podcast is much better thought of as a simplified version of a radio recording. Within education, this will often be without all the “bells and whistles” since this is often being developed for a small audience on one particular course (although trying to make things more widely useful is never a bad idea).

 

Lesson Three – Time needs to be allocated to produce podcasts to a high standard

This is one on the sticklers when using podcasts in an educational setting. Creating good online resources of any kind takes time and this is no different where podcasts are concerned. If these were just requested to be done on top of all other types of teaching, it’s very unlikely that much of use would ever be developed.

Some institutions get around this by just recording audio versions of lectures, which, as I’ve already discussed, I feel is undesirable. A separate time allowance to engage with podcasting and to learn the associated technical skills is needed.

 

Lesson Four – Focus on creating podcasts using information that works in a non-visual format

This was one of the biggest challenges with the set of materials I set to adapt, which were PowerPoint slides. By their very nature, Professional Online Presences are visual and tutorials on areas such as LinkedIn would have benefited from being able to show what the screens looked like. I got around this by keeping the material at a high level, which is probably sensible, since versions of social media sites such as LinkedIn do change rapidly and I didn’t want the podcasts to date.

Otherwise, careful consideration has to be made as to what materials are turned into podcasts. Some material does not require a display at all and so would be idea for podcasting. Highly technical presentations, by contrast, seem unlikely to be suitable source material for good podcasts (but alternatives, such as video presentations, could be considered).

 

Lesson Five – Podcasts do not have to be perfect. So long as the sound quality is good, slight wording errors and corrections are fine

I don’t think anyone expects an unscripted audio to be perfect. This is no more the case in a radio interview or anything that could be considered as an equivalent to podcasts to the masses. It is okay to make small mistakes and then just keep going, or add a small correction where needed.

This is an area that holds people back, but just like people barely notice a mistake in a conversation, it’s the same with podcasts. Even where are multiple mistakes, or you end up at a loss for words, the format of short podcasts as mentioned in Lesson One is also useful, as this means that there is only a relatively short amount of material that has to be recorded again.

 

Lesson Six – Audio recording is simple, but the technical challenge comes from making the podcasts available and getting them listed on iTunes

Although I’m a technical person, this is where I found the biggest challenge with the podcasts. Creating the audios was easy enough and once I had a proper plan to work from (essentially, an edited version of the PowerPoint slides) I was able to talk around them and record the audio. This is where it does help creating podcasts covering material that I’ve taught to multiple groups of students and to other academics.

The biggest technical challenge was getting the podcasts online, getting them working in an iTunes formatted RSS feed, and getting the feed submitted to iTunes. Even using a WordPress site and plugin, there were still parts that just wouldn’t work correctly and took a lot of hours searching for solutions. My considered opinion of this one is that every web host is set up slightly differently for podcasting. There are some paid solutions that will host podcasts and these might be better considered than trying the DIY approach like I did.

 

I hope that this set of lessons does provide some talking points and things to consider if you are thinking about podcasting. I think it’s very worthwhile, but just start off with a short series of podcasts like I did – and don’t worry too much if things go wrong. The second series of podcasts will be much better!

Contract Cheating In 2014 – What Online Cheating Is Happening Today

Here is a teaching focused talk on contract cheating I delivered for the Computer Science department at the University of Sheffield.

The slides, can be accessed on my SlideShare account, or viewed below.

There are several new and recent examples in this presentation, with less direct focus on research results. One example includes a sting operation from Local 6 News in Florida, who commissioned original work and caught the writer being paid on camera.

Some suggestions about how the Computing community can help to stop the impact of contract cheating are also provided. I believe that there is certainly scope for technical approaches for detection to be developed, but that sensible methods of setting assignments should also be considered.

Page 20 of 31« First...10«181920212223»30...Last »