Substack AI detection is now built into the newsletter platform, and some writers are already calling it a “witch hunt.” Substack CEO Chris Best announced the feature Tuesday, saying the company wants to give readers more context as automated writing becomes harder to distinguish from human work online.
The company partnered with Pangram, an AI detection vendor, to let users scan writing and receive a percentage estimate of how much of the text appears to be human-written or AI-generated. Substack also added a “How I make this” statement that publishers can use to describe their process, including whether they use AI tools.
Best said Substack is not trying to ban AI-assisted writing. He said people should be able to choose the tools they use, while readers should have a clearer sense of what they are reading. A Substack spokesperson told 404 Media that the new tools are meant to increase transparency, not punish writers, and said the scans do not affect discovery on the platform.
What is Substack's AI detection tool?
Substack’s tool uses Pangram to make a probabilistic judgment about a piece of writing. In plain English: it is a machine looking at patterns in text and guessing whether another machine likely helped produce it.
That distinction matters because AI detectors do not prove authorship. They produce estimates, and those estimates can be wrong in both directions: human writing can be labeled as AI-generated, and AI writing can be labeled as human. Pangram CEO Max Spero recently told 404 Media that the company works to reduce errors and estimates its false-positive rate at about one in 10,000.
The false-positive problem is the center of the backlash. Alice Lemee, a ghostwriter and digital writing coach, said in a LinkedIn video that AI detectors are widely inaccurate and warned that a single mistaken accusation could seriously damage a writer’s reputation.
Mack Collier, who writes the small Substack Backstage Pass, defended his use of AI in his writing process. Collier wrote that AI helps him structure and improve his work, and said he would not apologize for using it.
Sam Illingworth, a professor and author of Slow AI, criticized the feature on his Substack in a post comparing it to a witch hunt. He argued that Substack is asking a machine to decide whether there is a person behind a text. Illingworth told 404 Media that he supports the new process-disclosure feature, but worries that AI scoring starts from suspicion rather than trust.
Illingworth also raised a concern that has followed AI detectors for years: they may falsely flag writing by non-native English speakers and neurodiverse writers. His argument is that a disclosure box can create a conversation between writers and readers, while a detector score can shut one down before it starts.
Substack says publishers can turn off detection on their posts before or after publication. The company also says creators can report and remove scans of their own work if they believe the result is wrong.
The feature has supporters, too. Some readers want labels for AI-generated material after watching other platforms fill with unlabeled automated text, images, music, and spam. Substack’s problem is the hard part of the whole AI-content fight: detecting synthetic work without falsely branding real writers as fakes.
This story draws on original reporting from 404 Media.
