Sam Altman, CEO of OpenAI, which makes ChatGPT, has previously stated that it is a problem that in class, you cannot tell whether it is ChatGPT or the student who has written an assignment. Therefore, they are working on several technologies to help solve the problem, such as watermarking texts.
On January 31, OpenAI launched a new service that works like the tools that have dominated the market for checking whether texts are written by artificial intelligence. They have trained an artificial intelligence to distinguish between texts written by artificial intelligence and by humans.
Try the new tool yourself here: https://platform.openai.com/ai-text-classifier
Still not trustworthy
Unfortunately, one should not set one's hopes too high concerning the fact that the new tool can catch all students who use ChatGPT because it still shows many false negative and positive results. OpenAi writes itself:
Our classifier is not fully reliable. In our evaluations on a "challenge set" of English texts, our classifier correctly identifies 26% of AI-written text (true positives) as "likely AI-written," while incorrectly labeling human-written text as AI-written 9% of the time (false positives). Our classifier's reliability typically improves as the length of the input text increases.
Limitations
At the same time, there are several limitations to the new service.
- You still cannot test content in Danish - only English! This means that you can NOT make a translation of a student text and then test it.
- It only tells whether the text is "very unlikely", "unlikely", "unclear if it is", "possibly", or "likely AI-generated". Thus, assessing which parts of a text are written by artificial intelligence is also difficult.
- The texts should be over 1000 characters, about 150 - 250 words.
- Highly predictable texts, such as lists or lists, cannot be tested.
- You can easily edit an AI-written text so that it is not judged as written by an AI.
So what else can we do with a program?
We now have another tool to test texts and even one from the company that helped create ChatGPT. It may not make it better, but they are at least trying to do something about the problem. And they are continuously working to improve it.
If you suspect a text is written by artificial intelligence, you can use different tools to examine it. Also, read our article on plagiarism and ChatGPT:
The new option is still a very small patch for a large wound. Instead of relying on new tools to examine texts, we should rather examine how we can incorporate the new technologies into our teaching design:
In the near future, we will see many new versions of chatbots, where the individual service is trained for a specific task., e.g. trained on all books from a specific publisher.
Overall, one must conclude that it is challenging to examine the texts for using artificial intelligence - especially it is interesting that even OpenAI has difficulty with it. That fact must also dismiss the rumors that you can ask ChatGPT if it has written a text.