Assessing With Integrity – The Role of Technology – Webinar Slides

As part of my academic work with the UK’s Quality Assurance Agency I delivered a short webinar presentation looking at technology and assessment in light of Covid-19 and the move to online teaching.

The slides considered why academic integrity is still important, the threats posed to academic integrity in the current situation and the associated technological tools available to support education and quality assurance.

You can see the slides I used below (and also on my SlideShare account).


The webinar was well-received and led into an interesting panel discussion. One of the areas that attracted interest was my point that technology is a tool, not a solution. Software designed to identify similarity and detect plagiarism can be useful for helping students to improve their writing and ensuring academic integrity, but the use of such a tool does not mean that documents are all free of plagiarism. There are similar analogies to consider for software designed to ensure the integrity of online exams.

I do think the sector has done well to adapt to supporting students using teaching and assessment modalities that are new to everyone in a very short time period, but this does not mean that we should be complacent. We can use this as an opportunity to innovate and improve the standard of education for all students, whilst still preserving academic integrity.

Plagiarism Across Europe and Beyond 2020 Conference

The annual Plagiarism Across Europe and Beyond conference took place in Dubai in April 2020, although the delegates weren’t there in person. They were distributed around the world.

Conference chair Zeenath Khan and her team did an excellent job in moving the conference online due to Covid-19 in a very short timeframe. Of course, discussion of Covid-19 became something of a running theme at the conference as well, as this has the potential to become one of the biggest challenges to academic integrity we are likely to see in our lifetimes. It was an area I also picked up on in my opening keynote (post here and video here).

As always, I could only attend a small number of the parallel presentation sessions, so missed out on many interesting talks. But, also as always, there were many topics of interest discussed in the sessions I could attempt. Here is a summary of five sets of developments that caught my attention.

#1 – Emerging Technology Offers A Fresh Challenge, But Also Provides Us With New Opportunities To Develop Solutions

Clare Johnson presenting on forensic investigation techniques at Plagiarism Across Europe and Beyond 2020

More work using forensic investigation techniques on student work had been completed by Clare Johnson, to build on the research she presented at the conference in 2019. Clare identified several suspicious tags that could be seen when extracting a Word document that showed possible evidence of plagiarism and contract cheating. Clare plans next to test the techniques against student work that has gone through academic misconduct panels to see if the same techniques hold. Assuming they do, it would be good to see if a tool could be produced to automate much of this investigation and to supply a summary of the evidence to markers.

Elsewhere at the conference, PlagScan discussed a new software development. They are adding metadata analysis to their academic integrity software, an area which would provide useful parallel data to Clare’s plans. There’s lots of potential for future work looking inside files to identify academic misconduct.

In my keynote presentation, I discussed many of the challenges to academic integrity that we will have to consider in the future. These include artificial intelligence systems that can write text that look like they have been written by a human, as well as automated marking systems that contract cheating firms can tap into. How we address these challenges is an ongoing area for discussion.

Another delegate shared their own example of an emerging trend, that of essay spinning or machine translation. Their version had a twist to it. Students studying at an institution not using English would buy an English language essay, then use automated translation software such as Google Translate to convert this to their local language. I presume they would then also edit the result to make sure that the text made sense. Detecting translated documents has always been a challenge, but it would certainly be nearly impossible to detect bespoke written work which has then gone through a translation process.

#2 – We Need To Develop Our Support Mechanisms For All Stakeholders In Academic Integrity

Often the support we provide, the written guidance we give and the training offered to students is not all aligned with modern developments in the contract cheating industry. Mary Davis talked about why students cheat. Her main finding, from talking to students, was cheating was due to a lack of time. Mary recommended helping students to structure their time and ensuring they put timelines in place for larger assignments such as dissertations.

Teddi Fishman reminded us that the language students use to think about academic integrity and cheating is often different to the language that academics use. That makes it difficult to conduct research in this are and to rely on student answers. We probably don’t know how much cheating is really going on. The question was asked, how can we address this?

We need to be more sophisticated in the type of training we offer students and allow for more complex and nuanced discussion. Mary Davis recommends getting students to go on popular contract cheating sites and evaluate what they offer (or what they say they offer). Students know about contract cheating and they probably have classmates who have used contract cheating services, so we shouldn’t be hiding our awareness from them.

One delegate shared an example related to this, which was funny but also revealing. One of their students had accidentally shared their screen and it showed that the student had been making assignment orders from an essay mill, in this case with the number running to double digits. It’s more common that most of us think.

Conferences like this one always attract people with a wide variety of experience in academic integrity. It’s always worth us including some introductory level training for the people attending a conference like this for the first time. But one theme that kept coming up is that we should be looking at having a more advanced level of training and then seeing how this can be fed back into individual institutions and discipline areas.

Robin Crockett discussing how to analyse student writing style to identify contract cheating

Robin Crockett has used R statistical analysis tools to analyse student work and to identify who wrote it. These tools are useful but it can be hard to get started with them. Perhaps we could roll out stylometric training more widely?

Mark Ricksen from Turnitin also advocated for the use of software. I’ve shared his research with Deakin University before, showing how if markers are trained and have access to authorship attribution software, they will detect more contract cheating. But that training has to be put widely into place.

How do we support researchers who are themselves victims of plagiarism or academic misconduct? They are often an overlooked group of people. Tomáš Foltýnek revealed that ENAI is setting up a new support service for such victims. A very timely and important development.

#3 – We Need Legislation To Reduce The Risks To Students

In my keynote presentation, I briefly shared an example of attempted blackmail that a student had shared with me. This was a very blackmail elaborate scheme, with the firm creating fake letters from the UK Government and stealing mailing list details from existing companies to make it look like they were acting legitimately. I can understand how students could fall for this type of scam, particularly as they know they are at risk from their information being shared.

Mark Ricksen from Turnitin also discussed his experience buying essays for research purposes, noting how contract cheating companies ask for information from students about their universities as that’s one way they can extort you.

Lesley Sefcik has been investigating the very real risk of students being blackmailed by contract cheating providers

The most comprehensive study in this area has been conducted by Lesley Sefcik and Jon Yorke from Curtin University and they presented their findings at the conference. Lesley and Jon found that students would rather pay up to people extorting them than risk their contract cheating being reported to their universities. They also shared their own example of how firms blackmail and extort students. In this case, a firm waited until a student had graduated, then threatened to report them to their university to have their degree rescinded unless they made continual micro-payments. We do need to consider how we will offer support to students who could be at risk of blackmail.

Michael Draper shared his latest findings on how legislation can be used to prevent contract cheating. He had two parallel examples. One was the UK, where there has been a lot of discussion on legislation and firms have been encouraged to voluntarily change their practice. Michael said that essay mills have not made any voluntary changes despite these requests. By contrast, Ireland had brought in legislation and that had an immediate impact in marketing and advertising. This does suggest we should continue to pursue the legislative route as one of a number of necessary parallel solutions to help us to preserve academic integrity and reduce the risk to students posed by predatory firms.

#4 – We Should Continue To Explore Data Analytics Research Opportunities

One question that emerged from my keynote presentation related to how publishing of research papers in the academic integrity field had changed. I looked at two sample terms, “contract cheating” and “academic integrity” and showed that the number of publications with those terms in the title has been increasing. But does this mean that there is more interest in the area, or have we retired the use of other terms such as “academic misconduct“? There is a potentially interesting study to be completed looking at the changing terminology of academic integrity.

Tim Daly from Zahed University had investigated how contract cheating providers use content marketing to get customers. The whole funnel of how providers develop relationships with customers is fascinating and one I’ve looked at in my own research. But Tim has really drilled down into the data and the terms students looking for help will type into Google and, rather than finding helpful pages from universities, will find information from contract cheating providers instead.

Looking for just searches from the US for the term “business essay” alone, Tim found 52 sites getting over 9,000 organic visitors each per month. That translated to over 2 million visits per month across all of those sites.

Tim Daly identified search terms contract cheating providers were using to market their services to students

Out of the top 10 contract cheating providers for the US market, Tim found terms such as “essay writing”, “argumentative essay”, “essay outline” and “how to write an essay” were dominated by contract cheating providers. Tim also found legitimate universities linking to help pages owned by the contract cheating industry. Tim says that we need to make sure that we create good content on all of those topics on university websites so that those results appear in search engine resultss and students get directed to good information, rather than them ending up in contract cheating provider marketing funnels.

There are many opportunities to expand internationally using Tim Daly’s research methodology as a base, which only focused on the US market.

#5 Academic Integrity Is For Everyone

Are we leading by example, asked Tracey Bretag in her keynote presentation? Tracey noted that many academics take shortcuts in their own research, often bypassing ethical approval processes or submitting papers with poor analysis or referencing. If academics can’t get integrity right, can we expect this from students, asked Tracey? Tracey encouraged us to be generous with our time to grow the next generation of research and to take professional development courses where necessary.

We need the whole community to buy into the idea of academic integrity too. Teddi Fishman reminded us how important buy-in is when she presented. She advocated that we should be looking at Covid-19 as providing us with an opportunity, not just a challenge. I discussed how staff and students need to work in partnership, but really this has to extend become even the academic area. Student ethical views are heavily shaped by their home environment, where they grow up and normative behaviour within their culture.

Zeenath Khan has conducted initial research into how far parents are willing to help their primary school age children with their homework. Zeenath found evidence of parents and siblings doing homework for their children, particularly for “show and tell” presentations, along with evidence that teachers are being condescending to children whose family haven’t helped them in that way. That lack of academic integrity can then follow children throughout their time at school and influence how they choose to act when subsequently reach university level study.

Conference Chair Zeenath Khan leading the way with academic integrity in the UAE

Veena Mulani shared an example of how a child of 10 year old was cheating on a class being taught remotely due to Covid-19. The child had replaced their camera feed with a picture of themselves working, knowing that the teacher was looking after a large number of pupils and wouldn’t be able to tell.

Teddi Fishman also reminded us that we need to be transparent with students. She said that many of the challenges with Covid-19 could have been avoided had the world subscribed to this view of transparency.

After Teddi’s presentation, I shared the analogy of herd immunity for Covid-19 and asked delegates to consider how this could be applied to academic integrity. Can we get everyone in agreement that acting with integrity is important? If so, we can go a long way towards ensuring it simply becomes unacceptable to cheat. That’s what we need to show that conferences like Plagiarism Across Europe and Beyond are having an impact and to truly make academic integrity an important issue for everyone.

Academic Integrity 2020 – Keynote Presentation Slides

I delivered the opening keynote for Plagiarism Across Europe and Beyond 2020. The conference was moved online creating an interesting dynamic, but worked very well in the circumstances.

In my presentation, I looked back at the 20 years I’d been working in academic integrity, starting off trying to use technology to detect plagiarism, moving through the current changing situation with Covid-19, finishing by considering the challenges likely to face the sector in the future.

You can see the slides I used below (and also on my SlideShare account).


The talk was wide ranging, paying credit to many of the researchers who have laid the foundation to contract cheating research and showing how many of the issues we’re looking at now are the same ones we were talking about over a decade ago.

At this time when we’ve moved a lot of assessment online, how will we ensure continued academic integrity and how will we address the contract cheating industry aggressively marketing to vulnerable students? How will we react when students struggle with extortion and blackmail demands? The issues are very real and the threats posed by Artificial Intelligence will only see continued challenges for academic integrity in the future.

International Center For Academic Integrity Conference 2020

My trip to Portland Oregon in March 2020 to the International Center for Academic Integrity Annual Conference may turn out to be my only physical visit to a conference in 2020. As always, there were a range of interesting talks presented, including (I hope) my own session on Discipline Differences in Contract Cheating.

It has taken me longer than usual to reflect on the conference as the whole positive experience was slightly shattered by hearing about the sudden passing of my long-time friend and research collaborator Robert Clarke. You can read more about the man we all knew as “Uncle Bob” in this tribute published in Times Higher Education.

As always, the conference saw a range of parallel sessions, so I was only able to attend some of those, with much of my interest focusing on the sessions on contract cheating. It’s true that many of the same ideas appear repeatedly at academic integrity. But here are five themes that I think are a good reminder of where we’ve been, what is still happening now and what we need to consider for the future.

#1 – We Should Be Detecting Much More Contract Cheating

During her presentation, Cath Ellis reminded us that the contract cheating industry markets itself to students based on the idea that they won’t be detected.

I hope things are changing. Many software vendors at the conference were presenting their contract cheating detection solutions, but they are not yet in widespread use.

Kane Murdoch is a contract cheating detective

Kane Murdoch proudly billed himself as a contract cheating detective, something I’m proud to see in light of Bob Clarke’s role in this area (particularly as we once had a paper heavily criticised by reviewers for suggesting such a role should exist). Kane talked about the workflow he uses, with contract cheating processes following four stages: (1) suspicion, (2) investigation, (3) determination, (4) appeal.

Cath reminded us of the need to closely check references for evidence of contract cheating and provided some pointers. As a Computer Scientist, I do feel there’s lots of scope for automating the exploration of references and identify where things look amiss.

Ann Rogerson provided a related powerful tip, simply saying that we should get students to show us the process they used to find their sources. If a student can’t recreate this process, there’s a good suggestion that they didn’t complete the original searches for themselves.

Based on the numbers she’s collected in this field, Cath left us with this stark quote about where institutions should be at with contract cheating detection. “If your institution is not detecting contract cheating in 6% of the student population, you’re not succeeding”.

#2 – Exam Cheating Is Still A Problem

One of the big changes we’ve seen in the light of Covid-19 is a move to remote exams, often unsupervised, as a means of assessment. But if exams themselves are vulnerable to breaches of academic integrity, then surely ill-considered remote exams have many of the same issues as face-to-face exams and more?

Leonore Neary encouraging academics to improve their exam practices to make cheating more difficult

During the conference, Leonore Neary reminded us of some of the ways that students cheat. A long list of methods were presented included technical solutions like cell phone, smart watch, ear buds and camera use, but also more traditional methods like signalling, notes on clothing and notes concealed in pencil cases.

Leonore mentioned how unsuitable many room types are for taking tests and exams. Spacing and desk layouts often make them hard to invigilate and sometimes make it difficult for students to avoid seeing other student answers, even if they don’t plan to cheat. She shared her biggest success getting 85% approval from students by using privacy folders for multiple choice exams, essentially folders with small windows through which students can only view and answer one question at a time.

Leonore also shared a big tip for detecting cheating during the marking process, which is to look for evidence of eraser marks, often suggesting that answers had been changed when students were able to share their thoughts with one another.

#3 – What Students Share About You And Your Course Matters

One powerful reminder shared at the conference is that students have a lot of power to express their opinion about university life, teaching and research using the Internet.

Leonore Neary reminded us to “Google” ourselves and our class codes to see what they say. Students who have been put through an academic misconduct process or have a grudge often use sites like Rate My Professor to get even. Poor online feedback can be disastrous to the future careers of academics.

Students have also been observed encouraging their friends to gang up on by giving poor feedback to academics and were said to have even taken physical action, such as puncturing car tyres. University level support for people upholding academic integrity needs to be in place.

Ann Rogerson reminded us of the dangers of file sharing sites, said to be a bigger problem than contract cheating. Here, students share questions, answers, notes and resources online, often behind a paywall making them hard for academics to monitor. In many ways, this is an extension of the twenty-plus year old idea of an essay bank.

The file sharing site Course Hero was given particular attention in several talks. Eric Gibbs from Urkund said that this was now one of the most common sites found as a source of plagiarism in submitted student work.

I was inspired by the ongoing work of Kelly George to develop Course Villain, a web crawler for Course Hero. The work is still only in its early stages, so was presented in the form of a poster, but the software should enable academics to track when their work ends up on Course Hero, or when students are using this to share illicit materials. Kelly’s early work has again demonstrated just how big a problem this area really is.

Poster about the Course Villain software being developed

#4 – The International Dimension Of Academic Integrity Is Important

In my own presentation, I stressed the need for us to develop localised case studies and to realise that information on contract cheating needs to be considered based on an awareness of wider subject and institutional needs.

Several presenters reminded us that cultural awareness is important, with David Ison suggesting plagiarism is not universally seen as bad.

Stephen Gow has extensively researched how Chinese students adapt to studying in the UK, noting a disconnect between their expectations before arrival and the reality once they arrived. Motivated by the opportunity for independence, they often found a reality of language problems and limited opportunities for interaction with other students, areas we should we aware of when designing curriculum and scaffolding support into our courses.

Sarah Eaton has been continuing excellent work on contract cheating from a Canadian context. One striking observation Sarah shared is that contract cheating providers in Canada are marketing to students before they hit high school, but most teachers are still completely unaware of this. We need to be finding ways to get teachers to talk to their students about academic integrity from a young age.

International contract cheating firms are also finding news ways to appeal to students to get them to sign up their friends as customers. Mark Ricksen shared a fun example of an affiliate programme where students grow their virtual cactus with every customer they refer.

Mark also reminded us of the very real risk to students of blackmail and extortion by contract cheating companies. Having purchased a large number of custom essays for research work with Deakin University in Australia, Mark found essay mills that were requiring photographs of his credit card, leading to a real security risk of identity fraud. Mark didn’t share this personal information and avoided those firms, but worries that students may have felt compelled to proceed with the transaction.

I feel like I have to share the concluding findings of David Ison’s talk on international differences in academic misconduct here, just because it provided the quote of the conference. “When we cheat in the US, we cheat real good!”

Conference presenters Mark Ricksen (Turnitin), Sarah Eaton and Stephen Gow

#5 – Graphic Plagiarism Is An Underexplored Issue

One of the most interesting talks at the conference came from a team at Ryerson University, exploring how students are cheating in graphic design and in how they create artwork. This matched nicely with my own talk, where I discussed the dangers of students in these areas being exploited by contract cheating providers.

As the Ryerson team mentioned, students have easy access to vast quantities of images and graphic design ideas through the power of Internet searches. There are currently major problems with students searching for designs, then plagiarising them or using them as inspiration with only minor changes to colours or layout.

Similar to how we might ask students to show drafts of written work, graphic design students can be asked to show their process. The Ryerson team also found evidence of students faking this process work, for example by getting friends to pose in ways similar to a final plagiarised illustration to look as if the friend was their inspiration rather than the image found online.

The team also provided a stark reminder of the dangers of sharing graphic work online, with one image they created ending up used without attribution on all manner of items from quilt covers to as a dating site logo. One Chinese company were said to have even claimed the copyright for that image.

In my own talk, I stressed that we need more research on academic integrity and contract cheating at discipline level. Art, for both its good practice and weaknesses, is definitely an area that should be at the forefront of that research movement.

Logo plagiarism example from Ryerson. Student found design online, then faked process drawing.

Next Year In Chicago

I fear we will learn a lot more about academic integrity in the coming months, including what happens when online assessment goes wrong.

Next year’s conference is planned for Chicago. Assuming the world is back to normal, I’m already looking forward to it.

Discipline Differences in Contract Cheating – Presentation Slides

Here are the slides I prepared for the International Center for Academic Integrity Conference 2020, which was held in Portland, Oregon.

The presentation focused on why we need to address contract cheating at discipline level and why we need to consider methods other than surveys for deciding what is happening regarding contract cheating.

Unfortunately, for me the presentation was rather a blur, as I heard that my long-term research partner and friend, Robert Clarke, had passed away. I got the news just before delivering the presentation. I’m very grateful to the audience regarding how they received the presentation in what was a very challenging time.

You can see the slides I used below (and also on my SlideShare account).


Robert Clarke and I have collected data on contract cheating dating back to 2004, so we’ve seen lots of interest in the area since that point, but little change in academic practice. But, at the same time, the companies marketing contract cheating services are very aware about the need to market to different subjects and students and how they need to play a long-term game to develop relationships. We need to adapt to this and continue academic integrity research at all levels.

Page 1 of 33123456»102030...Last »