Assessment and Plagiarism: Exploring Opportunities For Future CSEd Researchers

I was asked to talk about research opportunities within the assessment and plagiarism fields at the CSEdGrad Conference 2020 (with the focus aimed towards Masters and PhD students interested in Computer Science education).

You can see the slides I used below (and also on my SlideShare account).

This was only a brief presentation, as much of the session focus was on discussion and considering ways to help the students with their own research. But it was interesting to hear the students tell stories of their own experiences of teaching, including one to whom a former student confessed that one of their peers had done their assignments, but only after the course was completed. The term contract cheating wasn’t then known, but I hope it is now.

Putting the research angle aside, if nothing else this reiterates to me how important it is that we support everyone involved with teaching, including Teaching Assistants with understanding academic integrity and the role they have in preserving it.

Assessment and Plagiarism – Chapter Video Overview

Here is a video overview of a chapter I wrote, along with Anthony Robins and Sally Fincher, for The Cambridge Handbook of Computing Education Research.

The history behind this video is that I was asked to prepare a very short summary video about the chapter for a conference. I turned out to have far too much material as usual, so I’ve prepared a longer version. But I do still encourage you to read the whole chapter.

The video mentions a previous blog post on plagiarism and assessment, where I noted that it’s rare for both of these to be considered as part of the same academic research paper.

The chapter is based on Computing Education, so there’s a particular focus on assessment and plagiarism information as it relates to computer programming, although many of the ideas are much more widely applicable.

You can also access the Assessment and Plagiarism slides on Slideshare.

Plagiarism Across Europe and Beyond 2020 Conference

The annual Plagiarism Across Europe and Beyond conference took place in Dubai in April 2020, although the delegates weren’t there in person. They were distributed around the world.

Conference chair Zeenath Khan and her team did an excellent job in moving the conference online due to Covid-19 in a very short timeframe. Of course, discussion of Covid-19 became something of a running theme at the conference as well, as this has the potential to become one of the biggest challenges to academic integrity we are likely to see in our lifetimes. It was an area I also picked up on in my opening keynote (post here and video here).

As always, I could only attend a small number of the parallel presentation sessions, so missed out on many interesting talks. But, also as always, there were many topics of interest discussed in the sessions I could attempt. Here is a summary of five sets of developments that caught my attention.

#1 – Emerging Technology Offers A Fresh Challenge, But Also Provides Us With New Opportunities To Develop Solutions

Clare Johnson presenting on forensic investigation techniques at Plagiarism Across Europe and Beyond 2020

More work using forensic investigation techniques on student work had been completed by Clare Johnson, to build on the research she presented at the conference in 2019. Clare identified several suspicious tags that could be seen when extracting a Word document that showed possible evidence of plagiarism and contract cheating. Clare plans next to test the techniques against student work that has gone through academic misconduct panels to see if the same techniques hold. Assuming they do, it would be good to see if a tool could be produced to automate much of this investigation and to supply a summary of the evidence to markers.

Elsewhere at the conference, PlagScan discussed a new software development. They are adding metadata analysis to their academic integrity software, an area which would provide useful parallel data to Clare’s plans. There’s lots of potential for future work looking inside files to identify academic misconduct.

In my keynote presentation, I discussed many of the challenges to academic integrity that we will have to consider in the future. These include artificial intelligence systems that can write text that look like they have been written by a human, as well as automated marking systems that contract cheating firms can tap into. How we address these challenges is an ongoing area for discussion.

Another delegate shared their own example of an emerging trend, that of essay spinning or machine translation. Their version had a twist to it. Students studying at an institution not using English would buy an English language essay, then use automated translation software such as Google Translate to convert this to their local language. I presume they would then also edit the result to make sure that the text made sense. Detecting translated documents has always been a challenge, but it would certainly be nearly impossible to detect bespoke written work which has then gone through a translation process.

#2 – We Need To Develop Our Support Mechanisms For All Stakeholders In Academic Integrity

Often the support we provide, the written guidance we give and the training offered to students is not all aligned with modern developments in the contract cheating industry. Mary Davis talked about why students cheat. Her main finding, from talking to students, was cheating was due to a lack of time. Mary recommended helping students to structure their time and ensuring they put timelines in place for larger assignments such as dissertations.

Teddi Fishman reminded us that the language students use to think about academic integrity and cheating is often different to the language that academics use. That makes it difficult to conduct research in this are and to rely on student answers. We probably don’t know how much cheating is really going on. The question was asked, how can we address this?

We need to be more sophisticated in the type of training we offer students and allow for more complex and nuanced discussion. Mary Davis recommends getting students to go on popular contract cheating sites and evaluate what they offer (or what they say they offer). Students know about contract cheating and they probably have classmates who have used contract cheating services, so we shouldn’t be hiding our awareness from them.

One delegate shared an example related to this, which was funny but also revealing. One of their students had accidentally shared their screen and it showed that the student had been making assignment orders from an essay mill, in this case with the number running to double digits. It’s more common that most of us think.

Conferences like this one always attract people with a wide variety of experience in academic integrity. It’s always worth us including some introductory level training for the people attending a conference like this for the first time. But one theme that kept coming up is that we should be looking at having a more advanced level of training and then seeing how this can be fed back into individual institutions and discipline areas.

Robin Crockett discussing how to analyse student writing style to identify contract cheating

Robin Crockett has used R statistical analysis tools to analyse student work and to identify who wrote it. These tools are useful but it can be hard to get started with them. Perhaps we could roll out stylometric training more widely?

Mark Ricksen from Turnitin also advocated for the use of software. I’ve shared his research with Deakin University before, showing how if markers are trained and have access to authorship attribution software, they will detect more contract cheating. But that training has to be put widely into place.

How do we support researchers who are themselves victims of plagiarism or academic misconduct? They are often an overlooked group of people. Tomáš Foltýnek revealed that ENAI is setting up a new support service for such victims. A very timely and important development.

#3 – We Need Legislation To Reduce The Risks To Students

In my keynote presentation, I briefly shared an example of attempted blackmail that a student had shared with me. This was a very blackmail elaborate scheme, with the firm creating fake letters from the UK Government and stealing mailing list details from existing companies to make it look like they were acting legitimately. I can understand how students could fall for this type of scam, particularly as they know they are at risk from their information being shared.

Mark Ricksen from Turnitin also discussed his experience buying essays for research purposes, noting how contract cheating companies ask for information from students about their universities as that’s one way they can extort you.

Lesley Sefcik has been investigating the very real risk of students being blackmailed by contract cheating providers

The most comprehensive study in this area has been conducted by Lesley Sefcik and Jon Yorke from Curtin University and they presented their findings at the conference. Lesley and Jon found that students would rather pay up to people extorting them than risk their contract cheating being reported to their universities. They also shared their own example of how firms blackmail and extort students. In this case, a firm waited until a student had graduated, then threatened to report them to their university to have their degree rescinded unless they made continual micro-payments. We do need to consider how we will offer support to students who could be at risk of blackmail.

Michael Draper shared his latest findings on how legislation can be used to prevent contract cheating. He had two parallel examples. One was the UK, where there has been a lot of discussion on legislation and firms have been encouraged to voluntarily change their practice. Michael said that essay mills have not made any voluntary changes despite these requests. By contrast, Ireland had brought in legislation and that had an immediate impact in marketing and advertising. This does suggest we should continue to pursue the legislative route as one of a number of necessary parallel solutions to help us to preserve academic integrity and reduce the risk to students posed by predatory firms.

#4 – We Should Continue To Explore Data Analytics Research Opportunities

One question that emerged from my keynote presentation related to how publishing of research papers in the academic integrity field had changed. I looked at two sample terms, “contract cheating” and “academic integrity” and showed that the number of publications with those terms in the title has been increasing. But does this mean that there is more interest in the area, or have we retired the use of other terms such as “academic misconduct“? There is a potentially interesting study to be completed looking at the changing terminology of academic integrity.

Tim Daly from Zahed University had investigated how contract cheating providers use content marketing to get customers. The whole funnel of how providers develop relationships with customers is fascinating and one I’ve looked at in my own research. But Tim has really drilled down into the data and the terms students looking for help will type into Google and, rather than finding helpful pages from universities, will find information from contract cheating providers instead.

Looking for just searches from the US for the term “business essay” alone, Tim found 52 sites getting over 9,000 organic visitors each per month. That translated to over 2 million visits per month across all of those sites.

Tim Daly identified search terms contract cheating providers were using to market their services to students

Out of the top 10 contract cheating providers for the US market, Tim found terms such as “essay writing”, “argumentative essay”, “essay outline” and “how to write an essay” were dominated by contract cheating providers. Tim also found legitimate universities linking to help pages owned by the contract cheating industry. Tim says that we need to make sure that we create good content on all of those topics on university websites so that those results appear in search engine resultss and students get directed to good information, rather than them ending up in contract cheating provider marketing funnels.

There are many opportunities to expand internationally using Tim Daly’s research methodology as a base, which only focused on the US market.

#5 Academic Integrity Is For Everyone

Are we leading by example, asked Tracey Bretag in her keynote presentation? Tracey noted that many academics take shortcuts in their own research, often bypassing ethical approval processes or submitting papers with poor analysis or referencing. If academics can’t get integrity right, can we expect this from students, asked Tracey? Tracey encouraged us to be generous with our time to grow the next generation of research and to take professional development courses where necessary.

We need the whole community to buy into the idea of academic integrity too. Teddi Fishman reminded us how important buy-in is when she presented. She advocated that we should be looking at Covid-19 as providing us with an opportunity, not just a challenge. I discussed how staff and students need to work in partnership, but really this has to extend become even the academic area. Student ethical views are heavily shaped by their home environment, where they grow up and normative behaviour within their culture.

Zeenath Khan has conducted initial research into how far parents are willing to help their primary school age children with their homework. Zeenath found evidence of parents and siblings doing homework for their children, particularly for “show and tell” presentations, along with evidence that teachers are being condescending to children whose family haven’t helped them in that way. That lack of academic integrity can then follow children throughout their time at school and influence how they choose to act when subsequently reach university level study.

Conference Chair Zeenath Khan leading the way with academic integrity in the UAE

Veena Mulani shared an example of how a child of 10 year old was cheating on a class being taught remotely due to Covid-19. The child had replaced their camera feed with a picture of themselves working, knowing that the teacher was looking after a large number of pupils and wouldn’t be able to tell.

Teddi Fishman also reminded us that we need to be transparent with students. She said that many of the challenges with Covid-19 could have been avoided had the world subscribed to this view of transparency.

After Teddi’s presentation, I shared the analogy of herd immunity for Covid-19 and asked delegates to consider how this could be applied to academic integrity. Can we get everyone in agreement that acting with integrity is important? If so, we can go a long way towards ensuring it simply becomes unacceptable to cheat. That’s what we need to show that conferences like Plagiarism Across Europe and Beyond are having an impact and to truly make academic integrity an important issue for everyone.

Academic Integrity 2020 – Keynote Presentation Slides

I delivered the opening keynote for Plagiarism Across Europe and Beyond 2020. The conference was moved online creating an interesting dynamic, but worked very well in the circumstances.

In my presentation, I looked back at the 20 years I’d been working in academic integrity, starting off trying to use technology to detect plagiarism, moving through the current changing situation with Covid-19, finishing by considering the challenges likely to face the sector in the future.

You can see the slides I used below (and also on my SlideShare account).

The talk was wide ranging, paying credit to many of the researchers who have laid the foundation to contract cheating research and showing how many of the issues we’re looking at now are the same ones we were talking about over a decade ago.

At this time when we’ve moved a lot of assessment online, how will we ensure continued academic integrity and how will we address the contract cheating industry aggressively marketing to vulnerable students? How will we react when students struggle with extortion and blackmail demands? The issues are very real and the threats posed by Artificial Intelligence will only see continued challenges for academic integrity in the future.

Plagiarism Across Europe And Beyond 2019

Plagiarism Across Europe And Beyond is an established series of conferences, promoted on behalf of the European Network for Academic Integrity. The 2019 edition of the conference took place in Vilnius, Lithuania in June 2019.

I attended and delivered talks looking at contract cheating at subject level and exploring the academic writing market enabled through

The whole conference was interesting, with lots of practical tips and research shared. This conference attracted many of the top names in the academic integrity world to present. There were parallel sessions, so I could only attend some of them, mostly those on contract cheating. But here are some of my take-home ideas from the conference.

Findings From Plagiarism Across Europe And Beyond 2019

There are many methods that we can use to detect contract cheating

Clare Johnson has had success examining student work for contract cheating using forensic techniques

Detecting contract cheating has been a long-standing interest of mine. My PhD work focused on detecting plagiarism and, as a Computer Scientist, I believe that technology can form part of the process for addressing contract cheating.

Three sessions picked up on contract cheating detection. PlagScan focused on its new authorship tracking solution, which takes as its premise the idea that students who are taught together should develop a similar writing style from which a metric based solution can be used to look for outliers.

Clare Johnson showed how she’d been successfully using steganalysis methods, the same ones used for forensic computing investigations, to track unexpected edits within MS Word files. GSM London had developed software to automate looking at document properties across large numbers of documents. One of their finding was that many students used the same external writer and the same writer name could be tracked within multiple files.

We really don’t know much about student cheating at subject level

Rebecca Awdry warning about the risks posed by outsourcing in Agriculture

As my own talk showed, the traditional belief is that students are most likely to buy assignments in the Business discipline. But contract cheating services have now developed their advertising to focus on disciplines individually. I found three disciplines groups: (1) Computer Science, (2) Architecture, Building and Planning, and (3) Law, that were already highly exploited. I also found that the Creative Arts and Design group was at risk of further exploitation.

Rebecca Awdry presented her survey results of subject level cheating. Her finding was that students in Agriculture were most likely to outsource their work, with students in two areas: (1) Health, and (2) Languages being the least likely. These are different findings to other studies and frankly we just don’t know a lot about subject level contract cheating. As I said at the conclusion of my talk, we need to find more out about contract cheating in individual disciplines and to put subject level interventions in place before the essay industry does.

Felicity Prentice looked at essay spinning within medicine and found that the results of paraphrasing text through Google Translate were poor. For example, the common term “Emergency Department” was translated as “Erectile Dysfunction“, potentially making it not too hard for anyone trying to detect disguised plagiarism in that academic discipline.

I showed that, despite demand, contract cheating pricing has been driven down by competition.

The contract cheating industry offers cheap work and use sophisticated recruitment techniques

In my research, I found that the price being requested by contract cheating writers had been driven down by competition, often hitting just $5 USD for 1,000 words. This isn’t necessarily reflected when students order through the more established essay mills, as they make their own mark up on the amount paid to the writer.

And, that the end quality of purchased work may not be very good. Wendy Sutherland-Smith spoke about she had purchased essays and checked the quality. 52% of the purchased essays had gained a failing mark.

Wendy also noted that students buying essays were putting themselves at risk. She said that she’d been contacted many times by writers outraged that students hadn’t paid them or had reversed credit card transactions.

Irene Glendinning found that a contract cheating service had recruited two students to work with them from a university based near Coventry, UK. By employing students, this would enable the contract cheating services to get access to other students. Recruiting students as brand ambassadors is now thought to be an established method used by these services, with the individuals referred to as “classroom moles“.

Felicity Prentice talked about the tools contract cheating services claim to be providing to students. One example of a paraphrasing tool, allegedly there to rewrite text so that this did not appear as plagiarism, provided intentionally bad output, whilst at the same time providing adverts for a better solution, buying an essay from them.

Felicity Prentice demonstrating a paraphrasing tool with links to a contract cheating provider

There needs to be more of a focus on continuing academic integrity outside the classroom

Irene Glendinning has been research international corruption in higher education

Several examples were given of where students were being encouraged to take shortcuts before reaching university level. Zeenath Khan said that her five year old had been given an assignment to make a physical model, something that they just couldn’t be expected to do. Their school must have known that parents would have to do the work.

Another conference attendant went further. She said that her son had won a prize for homework she’d completed for him.

The involvement of friends and family in the contract cheating process cannot be understated. In her survey, Rebecca Awdry said she’d found that 11.2% of students had used friends and family members to complete their work for them.

Irene Glendinning also found corruption in international quality assurance bodies, meaning that they could not always be trusted to ensure academic integrity in the universities they represented. In one startling example, 60% of higher education providers in India were said to have no quality assurance process.

We need to work with and engage students in spreading the academic integrity message

Wendy Sutherland-Smith is part of an international team collaborating with Student Unions to investigate the student experience of contract cheating investigations

Many students are concerned about academic integrity. Wendy Sutherland-Smith spoke about successful interventions at Deakin University, most notably illustrated with a recent example where a student posted a message on Facebook asking for copies of other students’ assignments. Wendy said that four students reported this post within minutes. However, not all students wanted to see more done about contract cheating. For example, when the UK’s Bloomsbury Institute surveyed their students, most were not in favour of the institution implemented a whistleblowing policy.

Discussing contract cheating with students is essential. Rebecca Awdry surveyed a large number of students and found that students who thought other students cheated were most likely to cheat themselves. Many Student Unions are now supporting the need for this discussion. Early results from collaborative research between Coventry University and Deakin University has questioned if students understand institutional academic integrity processes and found that the way an investigative process is communicated with students may be problematic.

Students at one university were said to have complained when they were caught contract cheating. They said that they’d approached this assignment in the same way that they’d approached all their other assignments. This means that contract cheating needs to be addressed early and methods of detection also needs to be in place early.

The whole conversation surrounding integrity can be difficult. Anna Krajewska from Bloomsbury Institute, UK, found that her students said they cheated so as not to disappoint her. They did not always want to engage with the subject. One student summed up why they were really studying higher education by complaining “We pay this school £6000 per year… just to have stress”.

It is important to find methods of discussing academic integrity that engage students. Penny Beale from Suffolk County Community College shared an example that worked well for her, that of discussing tattoo plagiarism, presumably valuable in these days that so many students care passionately about the uniqueness of their body art.


Towards Plagiarism Across Europe And Beyond 2020

Overall, Plagiarism Across Europe And Beyond 2019 proved to be a useful and important conference. I look forward to attending Plagiarism Across Europe And Beyond 2020 in Dubai, where I am delivering one of the keynote addresses.

Page 1 of 7123456»...Last »