Cory Doctorow says the useful irritation of computers is that they make humans say the quiet part in executable form. In a June 5 essay on Pluralistic, he argues that software has long exposed the sloppy assumptions buried in everyday categories, from names and addresses to gender fields and alphabetized movie titles.
The point is bigger than bad forms. Doctorow’s claim is that computers have acted as a kind of bureaucratic acid test: if a society encodes a category too narrowly, the person who does not fit has no clerk to persuade, no margin to write in, and no paper form to turn over. The fix becomes systemic, because the machine will enforce the schema until someone with access changes it.
That is the useful part of dumb software. It does exactly what programmers told it to do, including the stupid bits. Doctorow compares this to trying to define the article “A,” noting that the Oxford English Dictionary gives more than a dozen definitions and that the first runs beyond 1,500 words. Apparently even the smallest words are load-bearing.
Software makes hidden assumptions explicit
Doctorow points to the long-running “falsehoods programmers believe” genre, which catalogs bad assumptions about people’s names, street addresses, prices, time, email addresses and phone numbers. The lesson, as he frames it, is that ordinary human categories look tidy until a database demands a field type, a length limit, and a validation rule.
His example is musician Madagascar Slim, whose first name is Randriamananjararadofabesata. Doctorow writes that a paper-era official could deal with a too-short name box by writing in the margin or on the back of the form. A database with a fixed name length cannot improvise unless someone changes the software.
He applies the same mechanism to identity records. People whose names, genders, races or other biographical details change, or do not fit official boxes, have long existed, Doctorow argues. Digitization raises the stakes because a frontline official’s discretion gives way to code that treats fields as fixed. If the database says a gender value is immutable, persuasion at the counter does not update the record.
AI changes the old computer lesson
Doctorow then turns to AI, where his argument gets less comforting for people who want machines to remain boringly literal. Conventional programming forced humans to spell out their reasoning. Deep learning, by contrast, can learn patterns from examples without anyone writing a complete theory of “cat,” “dog,” “tiger” or “tractor” into code.
He treats that as both a technical gain and an intellectual loss. Image classifiers can do useful work without forcing humans to list every visual feature they rely on. The machine can produce the result while skipping the introspection that older software demanded.
Doctorow extends the same point to chess and creativity. Chess software can beat humans and produce moves people would call creative if a person made them, he writes. His conclusion is that this should narrow the definition of creativity, rather than expand personhood to include machines.
For that position, Doctorow cites Ted Chiang’s Atlantic essay, “No, Artificial Intelligence Is Not Conscious”. Doctorow says treating machines as persons would be both a technical mistake and a tactical one, likening it to corporate personhood. He says moral consideration should be expanded to living things such as animals and ecosystems, while excluding artificial constructs.
His analogy comes from Edsger Dijkstra’s old question, “can a submarine swim?” Submarines move through water. Animals swim with goals. Doctorow’s AI version is that machines can generate outputs once associated with creativity, while lacking the intention or meaning that would make the act human.
This story draws on original reporting from Pluralistic.