Risks and regulation of AI: Human-coded crisis

We could all be dead by the end of the decade - that, says AI developer Jacob Coxon, is a serious assessment of the industry and the reason he is quitting Anthropic. 169 million people saw his tweet and it has been on newscasts ever since. Not only Anthropic boss Dario Amodei has now agreed with him, but also Sam Altman, CEO of OpenAI, and tech entrepreneur Elon Musk are in favor of slower development and more regulation.

People are afraid of AI, and that's understandable. Because if AIs solve millennium problems in mathematics today, who's to say they won't help a terrorist create a supervirus tomorrow? Or paralyze critical infrastructure all by yourself? The progress is impressive, but the discussion about the risks overlooks one crucial thing: the danger does not come from abstract “AIs”, but from people.

AI models don't just break out of a security environment to hack another company. They do it because they are given certain tasks, because they are given certain tools, because they have been trained over and over again to behave in a certain way. If you train a dog to hunt for months, you shouldn't be surprised if it ends up killing a rabbit from the neighbors.

And with AI the situation is even clearer: a language model cannot act unless asked. It basically only exists at the moment of the answer. The so-called agents blamed for the incidents are loops of questions and answers that only run when people explicitly write a program that asks question after question. No AI model in the world can wake itself up.

Behind every incident in the headlines are human decisions: training the models for hacking skills, designing the agent loops to run for hours, leaving them unattended. Creating cyber risks and then selling cybersecurity: We usually know this under the mafia term protection racket.

Jacob Coxon knows this, and that's why he denounces the irresponsibility of AI companies. And at the same time, the discussion is great PR for an industry that thrives on making its products look more powerful than they are. And also that responsibility is attributed to the products instead of the people who make them.

What is lost in the Coxon debate: The dangers of uncontrolled systems are absolutely nothing new to humanity. We know cars, trains and airplanes that can cause great damage if the people driving them don't have them under control. We work with poisons, drugs and viruses, and we have to be extremely careful that they do not cause unforeseen side effects or spread autonomously.

We have found ways to deal with this: We check security. We only allow research under strict conditions. Anyone who is now thinking hard about how to enforce a complete stop to development of AI is forgetting all of these mechanisms.

High penalties for negligent behavior, as set out in the EU AI Act, would be enforceable. The review of models by security authorities for AI, which many countries already have - including the USA, if Trump had not gutted them, would be enforceable. Reporting and testing systems such as those in China, where incidentally the speed of innovation does not seem to suffer greatly from regulation, would be enforceable.

In contrast, the US tech industry, which is now negotiating a joint slowdown and self-regulation, seems hypocritical. Amodei is calling for an exception to antitrust law. The companies should set the limits, the state should only approve it. Agreements in the interests of global security simply sound better than: “We can’t manage to build good, safe products.” With the practical side effect that you can secure your technical advantage and no competitor can jeopardize an IPO with a surprisingly good model.

Human cooperation on security issues is complex. It does not protect us from every terrorist, every malicious act and every mistake. But it exists and can also cope with the new risks of the AI ​​industry if the will to enforce it is there. It is people who make every decision behind AI products. That doesn't make them any less dangerous. But we have experience with dangerous people.