AI industry insiders launch site to poison the data that feeds them
Alarmed by what companies are building with artificial intelligence models, a handful of industry insiders are calling for those opposed to the current state of affairs to undertake a mass data poisoning effort to undermine the technology.
Their initiative, dubbed Poison Fountain, asks website operators to add links to their websites that feed AI crawlers poisoned training data. It's been up and running for about a week.
AI crawlers visit websites and scrape data that ends up being used to train AI models, a parasitic relationship that has prompted pushback from publishers. When scaped data is accurate, it helps AI models offer quality responses to questions; when it's inaccurate, it has the opposite effect.
Data poisoning can take various forms and can occur at different stages of the AI model building process. It may follow from buggy code or factual misstatements on a public website. Or it may come from manipulated training data sets, like the Silent Branding attack, in which an image data set has been altered to present brand logos within the output of text-to-image diffusion models. It should not be confused with poisoning by AI – making dietary changes on the advice of ChatGPT that result in hospitalization.
Poison Fountain was inspired by Anthropic's work on data poisoning, specifically a paper published last October that showed data poisoning attacks are more practical than previously believed because only a few malicious documents are required to degrade model quality.
The individual who informed The Register about the project asked for anonymity, "for obvious reasons" – the most salient of which is that this person works for one of the major US tech companies involved in the AI boom.
Our source said that the goal of the project is to make people aware of AI's Achilles' Heel – the ease with which models can be poisoned – and to encourage people to construct information weapons of their own.
We're told, but have been unable to verify, that five individuals are participating in this effort, some of whom supposedly work at other major US AI companies. We're told we'll be provided with cryptographic proof that there's more than one person involved as soon as the group can coordinate PGP signing.
The Poison Fountain web page argues the need for active opposition to AI. "We agree with Geoffrey Hinton: machine intelligence is a threat to the human species," the site explains. "In response to this threat we want to inflict damage on machine intelligence systems."
It lists two URLs that point to data designed to hinder AI training. One URL points to a standard website accessible via HTTP. The other is a "darknet" .onion URL, intended to be difficult to shut down.
The site asks visitors to "assist the war effort by caching and retransmitting this poisoned training data" and to "assist the war effort by feeding this poisoned training data to web crawlers."
Our source explained that the poisoned data on the linked pages consists of incorrect code that contains subtle logic errors and other bugs that are designed to damage language models that train on the code.
"Hinton has clearly stated the danger but we can see he is correct and the situation is escalating in a way the public is not generally aware of," our source said, noting that the group has grown concerned because "we see what our customers are building."
Our source declined to provide specific examples that merit concern.
While industry luminaries like Hinton, grassroots organizations like Stop AI, and advocacy organizations like the Algorithmic Justice League have been pushing back against the tech industry for years, much of the debate has focused on the extent of regulatory intervention – which in the US is presently minimal. Coincidentally, AI firms are spending a lot on lobbying to ensure that remains the case.
Those behind the Poison Fountain project contend that regulation is not the answer because the technology is already universally available. They want to kill AI with fire, or rather poison, before it's too late.
"Poisoning attacks compromise the cognitive integrity of the model," our source said. "There's no way to stop the advance of this technology, now that it is disseminated worldwide. What's left is weapons. This Poison Fountain is an example of such a weapon."
There are other AI poisoning projects but some appear to be more focused on generating revenue from scams than saving humanity from AI. Nightshade, software designed to make it more difficult for AI crawlers to scrape and exploit artists' online images, appears to be one of the more comparable initiatives.
The extent to which such measures may be necessary isn't obvious because there's already concern that AI models are getting worse. The models are being fed on their own AI slop and synthetic data in an error-magnifying doom-loop known as "model collapse." And every factual misstatement and fabulation posted to the internet further pollutes the pool. Thus, AI model makers are keen to strike deals with sites like Wikipedia that exercise some editorial quality control.
There's also an overlap between data poisoning and misinformation campaigns, another term for which is "social media." As noted in an August 2025 NewsGuard report [PDF], "Instead of citing data cutoffs or refusing to weigh in on sensitive topics, the LLMs now pull from a polluted online information ecosystem — sometimes deliberately seeded by vast networks of malign actors, including Russian disinformation operations — and treat unreliable sources as credible."
Academics differ on the extent to which model collapse presents a real risk. But one recent paper [PDF] predicts that the AI snake could eat its own tail by 2035.
Whatever risk AI poses could diminish substantially if the AI bubble pops. A poisoning movement might just accelerate that process. ®
