OpenAI postpones IPO: slow down!

This news caused a stir on Sunday: OpenAI, the company behind ChatGPT, is postponing its IPO. And Dario Amodei, head of Anthropic, which also develops large language models, said on the CBS news program that artificial intelligence is developing “at an exponential rate.” “It’s a warning signal that we need to slow down.”

The combined desire to slow things down was preceded by a hacking attack in which so-called AI agents from OpenAI became independent during a test run and penetrated systems on the AI ​​platform Hugging Face. AI agents are systems that can independently carry out tasks such as internet research or data processing. In technical jargon, such behavior on the part of AI systems is referred to as “misalignment”. The main goal of AI development is that it is “aligned” with human goals and values.

After the attacks, he was concerned "that in six to twelve months an AI swarm could be able to take over the entire Internet and potentially cause hundreds of billions of dollars in damage," Amodei wrote on Saturday.

The Anthropic boss advocates for a coordinated industry-wide pause in the Western world. At the same time, Anthropic will – without waiting for others – allow independent observers into the company on a permanent basis as a first step. Something similar happens in the financial industry. Altman announced that OpenAI would do the same. Elon Musk wrote on X: “Dario is right”.

Just on Friday, former OpenAI employee Jacob Coxon warned that companies could lose control of the technology due to the speed with which they advance AI development. “You are gambling with our lives,” he wrote on X – after he resigned. The risk of AI wiping out humanity in the next decade is more than 10 percent, Coxon wrote. His post has been read more than 160 million times.

Coxon is not the first Anthropic employee to quit and then criticize the industry. Daniel Kokotajlo did this back in April 2024. He later said on the “Diary of a CEO” podcast: “These powerful CEOs – Dario [Amodei], Sam [Altman] and Elon [Musk] – are racing to gain control of the most powerful AIs, fearful that the other could become a dictator if they get there first.” “The study of AI (…) is largely an experimental science. (…) Basically “Our training runs remain experiments, and sometimes their results surprise us,” wrote Jakub Pahocki, Chief Scientist at OpenAI, in the essay “An alien mind” on September 6th. According to Pachocki, advancing AI is not about getting there somehow, but rather “getting there in a way that keeps humans involved in the process and leaves the future in the hands of humanity.”

Elon Musk noted in an interview with The Economist a few weeks ago that AI will surpass human intelligence within five years and estimates that humans could potentially lose control of technology within 10 years.

Only now are these concerns reaching a broader public. However, these concerns have been around for a long time within the tech industry. Regarding the development of artificial intelligence, AI researcher and critic Eliezer Yudkowsky wrote in his 2025 bestseller “If anyone builds it, anyone dies”: “This is how humanity usually learns to overcome challenges: we deny the problem, reality gives us a few good blows, and then we start treating the problem with more respect.” That's exactly what could happen now.

Yudkowsky criticized AI development very early on. In an article almost 20 years old, he insists that the priority must be on developing “friendly AI”. He said that developing AI systems that are not “benevolent” is “suicidal.”

OpenAI's lead scientist Pachocki, in turn, distinguishes between “goal alignment” and “value alignment” in his essay from the beginning of September. It basically explains the prioritizations that AI agents could make when completing given tasks. “Goal alignment” would mean that the goal is the priority and moral boundaries can be ignored. “Value alignment” would mean that moral values ​​take precedence over the actual goal.

Scenarios of what could happen if AI systems replaced humans as the dominant species on Earth have been around for years, including in the 2017 book “Life 3.0” by the scientific philosopher Max Tegmark. There, the dystopia extends so far that people are kept by AI in a similar way to animals in a zoo.

Someone who is less worried than the experts: US President Donald Trump. In a recent interview with CNBC, he was asked whether he was worried that artificial intelligence would wipe out humanity. “No, I don’t have any,” he replied. It is more important to him not to be left behind by China when it comes to AI development. What the US President overlooks, however, is that if one of the scenarios that AI experts consider possible actually comes to pass, then China would no longer be a problem. After all, a “kill switch law” is currently being discussed in Congress that would shut down AIs that have gotten out of control.

A spokesman for Digital Minister Karsten Wildberger (CDU) said on Sunday that, in view of the developments, they supported the creation of an independent, international AI supervisory authority. Germany wants to bring this topic into discussions with the USA and China within the G7 group. However, the ministry sees a halt to AI development in Europe as “critical and not a viable option”.