ChatGPT Creator Releases Tool to Detect AI-Generated Text, Calls It ‘Unreliable’

The San Francisco-based startup, which launched ChatGPT in November and recently announced a multiyear, multibillion-dollar partnership with

Microsoft Corp.

MSFT 2.10%

, released the detection tool on Tuesday. It said in a blog post that the tool was designed to help people distinguish between text written by a human versus a range of artificial intelligence programs—not just ChatGPT. OpenAI said that in evaluations its new tool correctly identified 26% of AI-written text as “likely AI-written.” It said the classifier also had false positives 9% of the time in which it incorrectly labeled human-written text as AI-written.

“Our classifier is not reliable,” the company said, referring to it as a “work-in-progress.”

The tool isn’t good enough on its own, though it can be used to complement methods that educators, employers and others rely on to determine the source of a piece of text, OpenAI said. 

“While it is impossible to reliably detect all AI-written text, we believe good classifiers can inform mitigations for false claims that AI-generated text was written by a human,” the company said.

ChatGPT became a viral sensation due to its ability to produce human-sounding essays, poetry, screenplays and sales pitches on virtually any subject in seconds. 

Microsoft invested in OpenAI in 2019 and 2021 before announcing the major expansion of their partnership last week, and has said it plans to integrate the company’s technology into many of its products.

Soon after ChatGPT was released, the potential for it to be misused to do things such as spread misinformation and write spam became apparent. Schools and educators also have warned of the potential for students to use it to write essays or other work they have been assigned. In December, the software passed all three parts of the U.S. Medical Licensing Examination as part of a research experiment. 

Some schools have moved to ban students from using ChatGPT, while others are attempting to embrace it. Edward Tian, a Princeton University senior, created software called GPTZero to try to recognize writing generated by the software. 

OpenAI said it had schools in mind when developing its latest classifier tool. “We recognize that identifying AI-written text has been an important point of discussion among educators, and equally important is recognizing the limits and impacts of AI-generated text classifiers in the classroom,” it said.

Journalists, researchers and others can also use the tool to detect AI-generated content, the company said. 

OpenAI said ChatGPT is still unreliable on short texts and longer texts are sometimes labeled incorrectly. It performs “significantly worse” in languages other than English and is “unreliable” in detecting AI use in computer code.

Another problem is that the tool can’t easily tell if a list of facts—U.S. state capitals for example—was written by a person or AI, because the correct answer would be the same, OpenAIsaid. AI-written text can also be edited to evade the classifier, the company said.

These kinds of caveats raise questions about just how beneficial the tool can be, the company said. 

“Classifiers like ours can be updated and retrained based on successful attacks,” OpenAI said. “But it is unclear whether detection has an advantage in the long-term.”

With feedback from users, OpenAI hopes to improve the tool. It said it has reached out to U.S. educators to discuss ChatGPT’s capabilities and limitations. 

“These are important conversations to have as part of our mission is to deploy large language models safely, in direct contact with affected communities,” the company said.

Write to Sarah E. Needleman at [email protected]