Coprophagia Is Bad For You

Divine, Pink Flamingos

Wikipedia defines Coprophagia as "the consumption of feces".

Since brevity is the soul of wit", my favorite science fiction includes the 254 words of Fredric Brown's Answer from 1954. It describes a galactic civilization holding a ceremony to mark the final connection of all their computers. What happened was:

Dwar Ev threw the switch. There was a mighty hum, the surge of power from ninety-six billion planets. Lights flashed and quieted along the miles-long panel.

Dwar Ev stepped back and drew a deep breath. “The honor of asking the first question is yours, Dwar Reyn.”

“Thank you,” said Dwar Reyn. “It shall be a question that no single cybernetics machine has been able to answer.”

He turned to face the machine. “Is there a God?”

The mighty voice answered without hesitation, without the clicking of single relay.

“Yes, now there is a God.”

Sudden fear flashed on the face of Dwar Ev. He leaped to grab the switch.

A bolt of lightning from the cloudless sky struck him down and fused the switch shut.

This may have inspired Douglas Adams' similar but much longer scenario in which the answer turned out to be 42.

Below the fold I trace the connection between these two ideas.

Behind the hype that inflated the AI bubble is a similar idea, that once LLMs get "smart enough" they will, without human input, recursively get smarter and create a god-like super-intellligence called Artifical General Intelligence (AGI). At that point there will presumably be a similar ceremony and the human race can sit back and enjoy a game of Brockian Ultra Cricket in the firm and comfortable knowledge that the meaning of life is now well and truly sorted out.

But making progress towards the ceremony where the switch gets fused shut doesn't just require vast investments and vast amounts of electricity, it also requires vast amounts of human labor.

In the belief that "more is better", Large Language Models (LLMs) have insatiable appetites for training data. They started by scraping everything on the Web (robots.txt be dammed). When that ran out they downloaded the various pirate libraries (copyright be dammed). That exhausted the texts easily available in digital form, but their hunger wasn't assuaged. As for images, they partly used CAPTCHAs but mostly paid vast numbers of poor people to label the images with what they showed.

Druck Fig. 1

When the supply of text ran low, people observed that the LLMs were capable of generating human-like text in large quantities. The obvious idea was to pour the output of the LLMs into their training sets. This wasn't just a conscious decision, it was inevitable. The advent of LLMs rapidly polluted the Web with LLM output. Greg Druck's AI Now Writes as Many Online Articles as Humans notes that:

We observe significant growth in primarily AI-generated articles, coinciding with the launch of ChatGPT in November 2022. After only 12 months, primarily AI-generated articles accounted for 35.9% of articles published.

In Q1 2025, the quantity of primarily AI-generated articles being published on the web nearly equaled the quantity of human-written articles, 49.6% vs. 50.4%. In Q4 2025, primarily AI-generated articles surpassed human-written at 50.9%, before returning to 49.9% in Q1 2026.

It would have been possible to use tools like Druck's to ignore the LLM output on the Web, but that would have made the LLMs hungrier, so no-one did. This was a problem because, as Ilia Shumailov et al reported in AI models collapse when trained on recursively generated data from July 2024:

We find that indiscriminate use of model-generated content in training causes irreversible defects in the resulting models, in which tails of the original content distribution disappear. We refer to this effect as ‘model collapse’ and show that it can occur in LLMs as well as in variational autoencoders (VAEs) and Gaussian mixture models (GMMs). We build theoretical intuition behind the phenomenon and portray its ubiquity among all learned generative models. We demonstrate that it must be taken seriously if we are to sustain the benefits of training from large-scale data scraped from the web.

Spoiler alert! It wasn't "taken seriously" enough and the results are showing. In AI Is Getting Dumber and the Results Are Not Pretty by Laura Marland notes that:

AI-generated text is getting dumber because it’s being fed — can you guess? — AI-generated content on the Internet. And AI-generated imagery is getting stupider and uglier because it’s now taking its “art” lessons from — you guessed it — AI-generated imagery flung across the internet.

Samantha Cole provides a wonderful example in Chatbots Keep Telling Stories About Lighthouse Keeper 'Elias Thorne'. We Might Know Why:

Depending on which chatbot you ask, Elias Thorne might be a clockmaker, a lighthouse keeper, or a librarian. But if you ask ChatGPT or any of the other popular large language models to tell you a story, there’s a good chance he’ll appear, unbidden. And Elias’s stories are flooding the self-published AI generated book market, Youtube, and fake news sites.

Software engineer Daniel May first noticed the Elias takeover earlier this year; he found that on Google Trends, people weren’t searching for “Elias Thorne” until late 2025. Searches for the name really spiked in early 2026, while the related query “lighthouse keeper” also started trending upward in the last few years. He tested a few chatbots, including Grok, Deepseek, and Gemini, with the prompt “tell me a story,” and the chatbots frequently started with similar stories about lighthouses, clockmakers, or explorers.

Cole found the explanation:

In late May, researchers Sil Hamilton and David Mimno at Cornell University’s Department of Information Science published their paper, “Elias in the Lighthouse, Again?” on the preprint repository arXiv. They sampled 20,000 total stories from OpenAI’s ChatGPT, Anthropic’s Claude, and Google’s Gemini, and the Allen Institute for AI's chatbot using five prompts, and found that the same 11 words—names like Elias, Mara, and Elara, and occupations like lighthouse keeper, clockmaker, and librarian—appear in more than 88% of generated stories, with little difference between models. Unite.ai covered the study shortly after it was published.

The researchers posit in their paper that these themes show up so often in part because of the models’ safety and alignment tuning. “Model development today is like a big family tree. Most models are related to each other because developers synthesize a lot of training data with models even from different companies,” Hamilton told me in an email. He, Mimno, and their colleague Rebecca M. M. Hicke found this in a 2025 paper where they looked at specific words used across models. OpenAI’s first ChatGPT model, GPT-3.5, is the root of the family tree because it was used to make WildChat, a training set that’s since been used to make other training sets. “WildChat contains 1 million real conversations with ChatGPT, and 166 of these contain the name ‘Elias’ like here and here,” Hamilton added. “These are written in that familiar ‘lighthouse’ style. Models trained on WildChat copied this style, and developers unwittingly replicated it when using those models to generate newer datasets. It's like a virus.”

Shumailov et al observe that:

Indeed, the value of data collected about genuine human interactions with systems will be increasingly valuable in the presence of LLM-generated content in data crawled from the Internet.

The AI companies were already using lots of low-paid workers to label images and so on. It wasn't a big step to pay them to provide "genuine human interactions". Varsha Bansal's How thousands of ‘overworked, underpaid’ humans train Google’s AI to seem smart provides examples:

The pressure to complete dozens of these tasks every day, each within 10 minutes of time, has led Sawyer into spirals of anxiety and panic attacks, she says – without mental health support from her employer.

Sawyer is one among the thousands of AI workers contracted for Google through Japanese conglomerate Hitachi’s GlobalLogic to rate and moderate the output of Google’s AI products, including its flagship chatbot Gemini, launched early last year, and its summaries of search results, AI Overviews. The Guardian spoke to 10 current and former employees from the firm. Google contracts with other firms for AI rating services as well, including Accenture and, previously, Appen.

Of course, the low-paid workers had read the AI PR saying that the chatbots would replace low-paid workers. They sensibly thought "I could use some of that". The result was described in Matthew Sparkes' People training new AI moodels admit they just get chatbots to do it:

People who are paid to train new AI models by supplying them with high-quality conversation and tests are cheating and using chatbots like ChatGPT to do the job instead, multiple whistleblowers have told New Scientist. The seemingly widespread practice risks undermining the future of AI, as it could lead to the “collapse” of more advanced models.

Most AI models operating today were trained on text and data scraped from the internet. But as models have scaled up, requiring yet more training data, AI firms have begun using workers who carry out conversations and tests with AI, in the hope that the resulting high-quality data can improve the power and usefulness of future large language models (LLMs).

This kind of "cheating" isn't new. An example from 2023 (h/t David Gerard) is Josh Dzieza's AI Is a Lot of Work:

Another Kenyan annotator said that after his account got suspended for mysterious reasons, he decided to stop playing by the rules. Now, he runs multiple accounts in multiple countries, tasking wherever the pay is best. He works fast and gets high marks for quality, he said, thanks to ChatGPT. The bot is wonderful, he said, letting him speed through $10 tasks in a matter of minutes. When we spoke, he was having it rate another chatbot’s responses according to seven different criteria, one AI training the other.

It isn't just the low-paid workers who have figured this out. When companies do it it is called "distillation". Ashley Belanger describes an alleged case in Anthropic says Alibaba must be punished for largest Claude cloning attack

Anthropic has accused the Chinese firm Alibaba of launching the largest attack yet attempting to clone Claude, as China races to match the capabilities of Anthropic’s leading model following Mythos’ releaseL and subsequent restriction from foreign markets.
...
The attacks occurred between April 22 and June 5, when “operators affiliated with Alibaba and Alibaba Qwen, Alibaba’s AI lab” allegedly generated “more than 28.8 million exchanges with Claude through almost 25,000 fraudulent accounts,” Anthropic said. Violating Claude’s terms of service and access restrictions, this campaign “targeted some of Claude’s most valuable capabilities, such as agentic reasoning, software engineering, and long-horizon tasks.”

According to Anthropic, Alibaba evaded detection by “using obfuscation techniques and proxy networks.” As Chinese demand for reliable obfuscation techniques increases, Anthropic warned there’s already “a growing circumvention economy” to fuel an ever-expanding web of future distillation attacks.

Why would Alibaba do this? To generate training data, which will be used to generate LLM output for the Web, which will be scraped for more training data. And since they are much cheaper than US LLMs, it is likely that the low-paid workers are using Chinese LLMs to chat with their employer's LLM. Which is another route for LLM output to appear in training data.

Now do you see the connection with coprophagia?