Matomo

OpenAI: Launch of new tool to investigate for plagiarism

OpenAI: Launch of new tool to investigate for plagiarism

Sam Altman, CEO of OpenAI, which makes ChatGPT, has previously stated that it is a problem that in class, you cannot tell whether it is ChatGPT or the student who has written an assignment. Therefore, they are working on several technologies to help solve the problem, such as watermarking texts. 

On January 31, OpenAI launched a new service that works like the tools that have dominated the market for checking whether texts are written by artificial intelligence. They have trained an artificial intelligence to distinguish between texts written by artificial intelligence and by humans.

Try the new tool yourself here: https://platform.openai.com/ai-text-classifier

Still not trustworthy

Unfortunately, one should not set one's hopes too high concerning the fact that the new tool can catch all students who use ChatGPT because it still shows many false negative and positive results. OpenAi writes itself:

Our classifier is not fully reliable. In our evaluations on a "challenge set" of English texts, our classifier correctly identifies 26% of AI-written text (true positives) as "likely AI-written," while incorrectly labeling human-written text as AI-written 9% of the time (false positives). Our classifier's reliability typically improves as the length of the input text increases. 

Limitations

At the same time, there are several limitations to the new service.

  • You still cannot test content in Danish - only English! This means that you can NOT make a translation of a student text and then test it.
  • It only tells whether the text is "very unlikely", "unlikely", "unclear if it is", "possibly", or "likely AI-generated". Thus, assessing which parts of a text are written by artificial intelligence is also difficult.
  • The texts should be over 1000 characters, about 150 - 250 words.
  • Highly predictable texts, such as lists or lists, cannot be tested.
  • You can easily edit an AI-written text so that it is not judged as written by an AI.

So what else can we do with a program?

We now have another tool to test texts and even one from the company that helped create ChatGPT. It may not make it better, but they are at least trying to do something about the problem. And they are continuously working to improve it.

If you suspect a text is written by artificial intelligence, you can use different tools to examine it. Also, read our article on plagiarism and ChatGPT:

Plagiarism and ChatGPT – How to Detect the Use of AI in Written Assignments?
In the education system, we are challenged by the new opportunities of ChatGPT. Students use it extensively in their written work, both for idea generation and auxiliary tools. But some use it uncritically and submit texts written by artificial intelligence as their own work. It is very difficult...

The new option is still a very small patch for a large wound. Instead of relying on new tools to examine texts, we should rather examine how we can incorporate the new technologies into our teaching design:

Prevent ChatGPT Cheating: 10 Tips for Task Types
In this article, we will look at how we can set tasks so that it becomes harder to use ChatGPT and similar language models to cheat.
If you can't beat them, join them: Include ChatGPT in your teaching
In the article "Prevent ChatGPT cheating: 10 tips for assignment types" we describe how to set written assignments to make it harder for students to use ChatGPT to create answers. We have also investigated the possibilities of catching possible cheating using AI detectors and plagiarism...

In the near future, we will see many new versions of chatbots, where the individual service is trained for a specific task., e.g. trained on all books from a specific publisher.

Overall, one must conclude that it is challenging to examine the texts for using artificial intelligence - especially it is interesting that even OpenAI has difficulty with it. That fact must also dismiss the rumors that you can ask ChatGPT if it has written a text.

Kilder

OpenAI API
An API for accessing new AI models developed by OpenAI
CEO of ChatGPT maker responds to schools’ plagiarism concerns: ‘We adapted to calculators and changed what we tested in math class’
OpenAI CEO Sam Altman said in an interview that the company will work on tech like watermarking to prevent plagiarism but warns it won’t be perfect.