AI Slop Is Testing the Gatekeepers of Science and Wikipedia

A doctor of applied mathematics generated a plausible scientific paper by, in the words of one IEEE Spectrum editor, doing nothing at all to the machine's output — and then submitted it to one of the world's largest scholarly publishers to see what would happen.

That experiment sits at the center of a growing problem for the institutions that curate human knowledge. Freelance writer Benjamin Skuse, author of the IEEE Spectrum feature "I Used AI to Fake a Scientific Paper," set out to test whether AI-generated scholarship could slip past IEEE's defenses. According to Eliza Strickland, Spectrum's features and AI beat editor, the answer was sobering: producing a paper that read as more or less plausible science required almost no effort at all.

"He basically hit 'run experiment' and 'write paper,'" Strickland said. "And then he didn't do a damn thing to it. He just launched it into the maw of the publishing engine."

Why does one AI-generated paper matter?

The context is a six-year erosion of trust in machine-generated text. The term "hallucination" — the label for a chatbot's confident delivery of a plausible but false answer — entered the lexicon with GPT-3, when users could ask the model anything and receive instant, authoritative-sounding nonsense. Several versions of several different chatbots later, the problem has not been solved. Even users who find Claude or Gemini genuinely useful still operate on the maxim "trust but verify," because verification remains mandatory rather than optional.

Scale that single-user maxim up to the entire scientific record and the stakes become clear. Peer-reviewed publishing exists to guarantee that what enters the corpus of human knowledge has been checked by humans. If AI-generated "slop" — machine-produced text, often undetectable to casual readers — can enter that corpus, the guarantee breaks. Social media feeds already illustrate the failure mode at the consumer level: roughly half of one Spectrum editor's feed consisted of AI slop that sharp users could identify, sometimes shared unwittingly by embarrassed users and sometimes spread deliberately by posters who found it amusing or useful for sowing confusion.

What did Skuse's experiment actually show?

Skuse's finding was not a blanket collapse of peer review. According to Strickland, the combination of IEEE's human-review processes and its research-integrity checker programs is currently keeping pace — but only in what she described as a cat-and-mouse game against people using free tools.

The qualifier matters. Free AI tools make what Strickland called "dumb mistakes" that detection systems can catch. More advanced large language models (LLMs, the systems underlying chatbots like Claude and Gemini) no longer make those same errors. The result is an arms race: as generated text grows increasingly plausible, the systems designed to detect it must improve at the same rate simply to hold the line.

For now, the gatekeepers are winning against cheap attacks. The source does not say how long that will last, and neither, implicitly, do the gatekeepers.

Who else is fighting the same battle?

Scholarly publishers are not the only institutions holding the line. Wikipedia, described by Strickland as "that global depository of human-generated knowledge," faces the same flood of AI-generated material — with a fraction of the resources.

For Spectrum's monthly Five Questions interview, Strickland spoke with Ellia Lebleu, a founding member of WikiProject AI Cleanup, a volunteer effort to detect and delete AI-generated material from the platform. Lebleu's team confronts the identical problem confronting commercial publishers, but with fewer people and fewer tools.

"Wikipedia is known as a repository of human-generated information, and they want to hold on to that," Strickland said. "Who knows how long they'll be able to do that."

The structural asymmetry is the story. IEEE can deploy research-integrity software and paid editorial staff. Wikipedia relies on volunteers. Both are defending the same commodity — a record of verified human knowledge — against generation costs that have fallen to nearly zero.

What happens if the gatekeepers fall?

The consequences extend beyond individual retracted papers. If the public cannot collectively assume the scientific record is trustworthy, or that basic facts about the world are accurate, then shared understanding of what is real unspools into uncertainty — a condition the source likens to Schrödinger's cat, the thought experiment in which a state is simultaneously true and false until observed.

That environment has beneficiaries. The source names two: quantum scientists, presumably as a joke, and grifters, who thrive precisely where no shared reality exists to contradict them. The deeper casualty is institutional trust — confidence in the organizations that exist to curate the corpus of human knowledge and, in IEEE's own formulation of its mission, to promote the advancement of science and technology for the benefit of humanity.

There is also a stranger possibility. In a world where AI conducts the science and writes the papers, human trust in the outputs may simply stop being part of the process. Strickland framed the endpoint bluntly.

"Doesn't it just feel crazy that we'll have AI writing for other AIs at some point?" she said. "We'll just get left out of the loop."

What is the outlook?

The immediate trajectory is defined by the arms race Skuse documented: increasingly plausible generated text on one side, human reviewers plus detection software on the other, with the free-tool tier currently contained and the advanced-LLM tier not yet fully tested by his experiment. Wikipedia's volunteer cleanup effort, led by figures like Lebleu, will determine whether the open web's largest reference work remains a repository of human-generated information — or whether, as Strickland put it, its defenders lose the ability to hold that line. If both fronts fail, the question shifts from how humans verify AI output to whether humans remain in the verification loop at all.