If you feed America's most important legal document—the US Constitution—into a tool designed to detect text written by AI models like ChatGPT, it will tell you that the document was almost certainly written by AI. But unless James Madison was a time traveler, that can't be the case. Why do AI writing detection tools give false positives? We spoke to several experts—and the creator of AI writing detector GPTZero—to find out.

Among news stories of overzealous professors flunking an entire class due to the suspicion of AI writing tool use and kids falsely accused of using ChatGPT, generative AI has education in a tizzy. Some think it represents an existential crisis. Teachers relying on educational methods developed over the past century have been scrambling for ways to keep the status quo—the tradition of relying on the essay as a tool to gauge student mastery of a topic. //

As tempting as it is to rely on AI tools to detect AI-generated writing, evidence so far has shown that they are not reliable. Due to false positives, AI writing detectors such as GPTZero, ZeroGPT, and OpenAI's Text Classifier cannot be trusted to detect text composed by large language models (LLMs) like ChatGPT. //

"I think they're mostly snake oil," said AI researcher Simon Willison of AI detector products. "Everyone desperately wants them to work—people in education especially—and it's easy to sell a product that everyone wants, especially when it's really hard to prove if it's effective or not."

Additionally, a recent study from Stanford University researchers showed that AI writing detection is biased against non-native English speakers, throwing out high false-positive rates for their human-written work and potentially penalizing them in the global discourse if AI detectors become widely used.

Ted Chiang wrote about: that ChatGPT is a “blurry JPEG of all the text on the Web.” But the paper includes the math that proves the claim.

What this means is that text from before last year—text that is known human-generated—will become increasingly valuable. //

Tatütata • July 5, 2023 8:47 AM

What this means is that text from before last year—text that is known human-generated—will become increasingly valuable.

A bit like steel smelted before 8 August 1945… //

Tatütata • July 5, 2023 8:54 AM

The tails of the original content distribution disappear. Within a few generations, text becomes garbage, as Gaussian distributions converge and may even become delta functions. We call this effect model collapse.

Academia just discovered GIGO and the telephone game. Alleluia!

Just as we’ve strewn the oceans with plastic trash and filled the atmosphere with carbon dioxide,

and low-orbit space with débris.

so we’re about to fill the Internet with blah.

Isn’t it already? I just made my daily contribution. //

Winter • July 5, 2023 9:04 AM

I see a very lucrative market appearing for (high school) students working part-time as “real” human text producers. //

NC • July 5, 2023 9:27 AM

Hah, normal people don’t get paid! If a big tech company decides they want highschooler’s essays, they’ll just have Pierson or a pierson-alike company make essay-writing a part of the homework program they distribute with their textbooks, and thousands of teachers will require hundreds of thousands of students to submit millions of hours of work for free. For which Pierson might make a few bucks. //

Winter • July 5, 2023 9:49 AM

@NC

Hah, normal people don’t get paid!

Damn, my scheme is already torpedoed by those pesky capitalists.

But the matter is not really solved yet:
Who Owns Student Work?
https://designobserver.com/feature/who-owns-student-work/12667/

I know local Universities claim copyright to student’s works by way of some overarching educational contract (this is EU). I am not sure whether that has ever been tested in court. But I have never heard of schools being allowed to sell student work without getting the student involved. //

The U.S.’ ongoing naval challenges give Red China an opportunity to accumulate more power throughout the Indo-Pacific.

If you’re using an oximeter at home and your oxygen saturation level is 92% or lower, call your healthcare provider. If it’s at 88% or lower, get to the nearest emergency room as soon as possible.