OpenAI Takes Cautious Steps in Releasing ChatGPT Writing Detection Tools

OpenAI is developing a text watermarking tool to identify AI-generated content, but concerns about its functionality and potential impact on non-English speakers have prevented its release.

OpenAI has made a tool that could catch students who cheat by asking ChatGPT to do their homework, but The WSJ says the company isn’t sure if it should be released yet. A spokesperson for OpenAI confirmed in a statement that the company is looking into the text watermarking method described in the Journal’s story.

However, the spokesperson said the company is taking a “deliberate approach” to the research because of “the complexities involved and its likely impact on the broader ecosystem beyond OpenAI.”

“The text watermarking method we’re working on looks good on paper, but it comes with some big risks that we’re thinking about while we look into other options,” the spokesperson said. “These risks include the fact that bad actors might be able to get around it, and it might affect non-English speakers more than other groups.”

This is a different approach from most of the other attempts to find text made by AI, which have mostly failed. They even shut down their old AI text detector last year because it had a “low accuracy rate.” With text watermarking, OpenAI would only look for writing in ChatGPT, not in models from other companies.

It would do this by making small changes to how ChatGPT chooses words. This would leave an invisible mark in the writing that another tool could find later.

AI-Generated Content Research

Following the Journal’s story, OpenAI revised a May blog post discussing its AI-generated content research. When it comes to localized tampering, such as paraphrasing, text watermarking has been “highly accurate and even effective.”

However, it has been “less robust against globalized tampering,” such as using translation systems, rewording with another generative model, or telling the model to put a special character in between every word and then deleting that character. As a result, OpenAI asserts that this method is “trivial to circumvention by bad actors.”

OpenAI’s update also echoes the spokesperson’s point about non-English speakers, stating that text watermarking may “Make it look unprofessional for people who don’t speak English as their first language to use AI as a writing tool.”