How easy would it be to generate publishable academic research papers in bulk, using ChatGPT? That was a challenge I set myself after observing authors who were starting to publish very formulaic looking comparison style papers, comparing ChatGPT information with that obtainable from other sources.
The full details are in the video, but I set out to compare how closely the top Quora questions and answers related to the gig economy compared with a ChatGPT simulation of Quora questions and answers.
I used a LaTeX template to draft the paper, which did have some challenges when inserted ChatGPT output, but now I have a better template were I to create more papers like this in the future.
I found relevant papers on Google Scholar, then used ChatGPT to turn the information from the abstracts into the background section.
For the main section of the paper, I collected 10 Quora questions on the gig economy, and got ChatGPT to summarise the top answer for each. I also simulated this process with ChatGPT. I then got ChatGPT to assign categories to the questions, allowing the Quora and ChatGPT versions to be compared.
ChatGPT also generated various tables, a chart comparing the Quora and ChatGPT versions, and discussion sections. I did have to solve technical problems, and I did edit much of the output, but I ended up with a reasonable looking academic research paper, better than many I’d seen on Google Scholar, although less polished than my publications would usually be. The use of ChatGPT was acknowledged.
The Gig’s Up: How ChatGPT Stacks Up Against Quora on Gig Economy Insights
Abstract
Generative AI is changing the way in which humans seek to find answers to questions in different fields including on the gig economy and labour markets, but there is limited information available about closely ChatGPT simulated output matches that obtainable from existing question and answer platforms. This paper uses ChatGPT as a research assistant to explore how far ChatGPT can replicate Quora question and answers, using data from the gig economy as an indicative case study. The results from content analysis suggest that Quora is likely to be asked questions from users looking to make money and answers are likely to include personal experiences and examples. ChatGPT simulated versions are less personal and more concept-based, including considerations on employment implications and labour rights. It appears therefore that generative AI simulates only part of what a human would want in their answers relating to the gig economy. The paper proposes that a similar comparative methodology would also be useful across other research fields to help in establishing the best real world uses of generative AI.
The paper title was one of a number of ChatGPT suggestions. The abstract text was largely mine, based on the final paper.
You can read the paper here.
The Risk
I have no plan to mass publish academic research papers of this style, but if someone was so inclined, they could set up a research production line.
This paper took me six and a half hours using ChatGPT as a research aid. With practice, everything set up, and avoiding ChatGPT timing out, I could complete a paper like this in three hours, including limited editing. With web scraping set up, and a script to automate this, I could prepare papers like this in even less time, particularly if I didn’t care about editing.
What does this mean for research integrity and the future of academic publishing?