Wave of AI Fear: The AI ​​House of Cards

Since this summer at the latest, the wave of fear about artificial intelligence has continued to build. The latest incident that feeds them: The ChatGPT company OpenAI announced on Thursday that an AI model in a test referred to false information that it had previously created itself.

Or when OpenAI and Anthropic revealed that some of their AI models had illegally penetrated company systems and at least one had tried to manipulate people. And in early September, Jacob Coxon, an Anthropic employee, explained his resignation on Platform X, saying: “Neither company is acting responsibly.” And further: “They are racing directly towards a self-improving superintelligence and putting our lives at risk in the process.”

The wave of AI fear was further fueled by AI company bosses asking the US government to regulate them. Or when Anthropic CEO Amodei wrote on his blog shortly after Coxon's post that a so-called superintelligence could take over the entire Internet in 6 to 12 months. He called on the entire industry to slow itself down to make time for research. And the competition agreed, even the right-wing tech billionaire Elon Musk. But US President Trump rejects regulations; AI does not need any guardrails.

But is the fear justified? Not everyone agrees with dropout Coxon, who expects “superhuman systems.” “The scenario of a Terminator AI that is directed against humanity is a Hollywood fantasy,” says Dennis-Kenji Kipker, professor of IT security at the University of Bremen. "This would require superintelligence, i.e. an intelligence that overtakes human intelligence. But we are not at that point yet. Some say it is not possible. Others think it will take until 2050, and still others assume 2100."

After Coxon, Anthropic's security expert Evan Hubinger also spoke up: "We really believe that AI could kill everyone! Personally, I think the probability is ten percent in the next decade."

Thorsten Holz is critical. "That's a subjective opinion from an expert. You can't really quantify an event like the extinction of humanity," says the director at the Max Planck Institute for Security and Privacy. “When someone gives a percentage like that, it’s primarily a tactic to draw attention to the issue.” Nevertheless, Coxon's warning should not be dismissed as pure scaremongering.

Kipker says that it can currently be seen that the large corporations do not have their technologies under control. "We don't even need to assume that the AI ​​acted humanly. It was simply rational. Ethical or moral boundaries play a smaller role for it." Criminals and terrorists would take advantage of this. With AI they could build viruses or develop weapons.

For Kipker, the fact that AI could be exploited in this way is only one of two risks. Another arises from the excessive integration of AI in government institutions, in private life and in industry. “AI is used in energy supply, in logistics, in critical infrastructures that are essential for the community.” One driver is the fear of being left behind by the competition. “The more AI we integrate, the more likely it is that the AI ​​security incident will occur.” This would then have cascading effects and impacts in other areas. In order to stop this or prevent it in advance, expertise is needed. But according to Kipke, it is dwindling. “Also because fewer and fewer people are being trained in the field of computer science.” Companies would rely on AI instead.

There is no need for malicious intent for AIs to cause harm. This is shown by a case from the Indian city of Bengaluru: people have been collecting photographs of temples and coins for decades. Then in July, an AI agent accidentally destroyed 15 percent of the digital archive. Not the only time an AI deleted data.

And using the technology could become even riskier. For example, when a system acts in such a way that people no longer understand it. Or “if we give the AI ​​many options and powers that we can no longer consistently supervise ourselves,” says Kipker. According to him, the use of AI in weapon systems is “high risk”. "Even if the human finally presses the red button, the AI ​​sifts through tens of thousands of possible targets and decides on one. So the button is just an illusion of control." The race takes place not only in industry, but also in war.

Private individuals usually do not assign such large tasks to AI agents. “Normal users interact with ChatGPT, Claude and other models to which the companies have already placed certain restrictions,” explains Holz from the Max Planck Institute. “The models that are currently being criticized are research models and are more powerful.”

Even if the error rate of AI is decreasing, Holz advises distrust. "We don't manage to build systems that are 100 percent trustworthy. We didn't have that before either. Nuclear power plants had accidents, rockets exploded." That's why, according to Holz, critical processes shouldn't just be automated via AI, especially when human lives are at stake, for example in research. "What if we use AI there and cause a disease? Where there could be potentially catastrophic consequences for humanity, there should be an international memorandum not to conduct this research at all. And there needs to be a discussion about the resources that AI uses."

The issue has now reached the US House of Representatives. Das passed a bill on Wednesday that aims to protect private individuals from rising electricity costs caused by the enormous energy consumption of data centers. People in Pennsylvania and Ohio are currently suffering from this. There are protests against data centers across the country.

Despite everything, Holz sees advantages from AI, for example in IT security. "Browser developers are currently patching hundreds of security holes every month that they only found by searching for them using AI. This makes software more trustworthy," explains Holz.

But why are AI companies calling for a deceleration? According to Holz, this could buy time to expand urgently needed data centers. Holz also says: "The warnings are partly marketing. They help keep the AI ​​hype alive."

Peter Ganten suspects another reason. He is chairman of the board of the Open Source Business Alliance. "Open source models and freely available models are quickly improving. Industry and administration can operate them in their own data center. The data, for example medical data, that is used in this way remains with these organizations." This would reduce the risk of someone being able to access it.

However, general regulation of AI would cause problems here: According to Ganten, the tests would not be a major additional effort for billion-dollar companies, but they would be “absurd” for competing open source communities. Also because the models are constantly being changed by the communities. Regulation that looks the same for everyone would “dry up” further development. Ganten wants a “reasonable compromise”. UN Secretary-General António Guterres is now calling for international cooperation on AI regulation: “We need guard rails to make AI safe, transparent and responsible.”

Kipker sees the state as having a responsibility to provide information: “We have to disintegrate AI from areas where we don’t need it.” He wants a debate across society and speaks of an AI house of cards that is being built ever higher. Thorsten Holz compares AI to a hammer. We are constantly looking for nails that can be hammered in with them. “But maybe the hammer isn’t always the right tool.”