The Killer AI Robots Are Already Here

Over the last few weeks, panic about artificial intelligence has reached a fever pitch. AI industry whistleblowers as well as politicians like Bernie Sanders are sounding the alarm about the potential for an AI-driven apocalypse and the need to slow down or pause further development of the technology.

But lest we forget, AI is in fact already killing human beings, thanks to the increasing deployment of autonomous weapons systems in warfare. A recent high-profile incident, covered by the New York Times last month, saw three Ukrainian civilians killed by an AI-guided drone. An AI system has also infamously been used to select targets in Israel’s genocidal assault on Gaza.

Robert Sparrow is a professor of philosophy at Monash University who has written extensively on the ethics of warfare and who has called for a ban on the use of autonomous weapon systems. Jacobin recently interviewed Sparrow about how AI weapons systems are being used now, why he sees their deployment as especially morally disturbing, and what he makes of recent pronouncements about impending doom wrought by artificial intelligence.

Nick French

The New York Times reported last month that a fully autonomous drone killed three people in Ukraine this summer. Start with a definition of what makes a drone or a weapon system fully autonomous. How do the killings in Ukraine differ from previous documented uses of semiautonomous or unmanned weapon systems?

Robert Sparrow

The standard definition of autonomy now is a weapons system choosing its own targets or carrying out some significant function without human oversight. Some people will categorize “fire and forget” munitions as autonomous; other people will demand something more like the kind of agency that we’re seeing in generative AI and these kinds of AI agents that people are talking about.

As far as that news report goes, what seemed genuinely novel was not so much the capability of the weapon. The drone essentially had been designed to home in on gas. It was fired in the general direction of a gas station, and it recognizes and tries to crash into the propane tanks. There’s not an enormous amount of choice going on there. What seemed to be new was that there were reported civilian casualties.

But that capacity is one that systems have had for a long time. There was an Israeli loitering munition called the Harpy, which you fire somewhere where you’re worried that people will turn on air defense radars. It flies around by itself, and then someone switches on an air defense radar, and this thing homes in on and crashes into it. The Americans have a cruise missile called the Long-Range Anti-Ship. Again, you fire it in the general direction of an enemy group of ships. Then it autonomously chooses the highest-value target. It has radar or profiles of the ships, and it’s designed to fly in waves. Once one missile has destroyed the intended target, then it recognizes the next highest-value target.

I’m glad that people are drawing attention to these systems, but in that particular case, what seemed to be new was that it was a Russian, as opposed to a Ukrainian, use of such a drone; and that the Russians used an Nvidia chip that is pretty widely available — showing that this is a capacity that other people could develop — and that it killed civilians.

The Times article draws this distinction between last-mile targeting and fully autonomous weapons. It simply wasn’t clear to me in that particular case [that the drone was fully autonomous], because a human being had chosen that it was going to attack gas cylinders. They’d chosen to fire it; they’d chosen which service station to point it in the general direction of. It seemed like what it was doing was what some of the Ukrainians’ drone weapons do, which is that you can designate a target, then the drone will chase that person or tank in situations where you can’t maintain a human in the loop because there’s too much jamming going on.

But clearly, unless people band together to stop it, this is the future of war: more and more decisions about targets being handed over to machines.

Nick French

What are the major questions raised by autonomous weapons systems, whether it’s something like what was used in Ukraine or something more fully autonomous? With all sorts of weapons, there are unintended consequences. You might drop a bomb or fire a bullet and hit unintended targets, for instance. What is it about autonomous or self-guided weapons that raises special moral issues?

Robert Sparrow

A lot depends here on whether you’re thinking about something like an anti-tank mine. A modern anti-tank mine, in a sense, chooses its target. It’s designed to tell whether what’s driven over the top of it is a tractor or an enemy armored vehicle. Some of them are designed to count targets, so that the enemy tank column is inside the minefield when the first mine explodes. It is hard to get too worried about that.

At the other end, people imagine the Terminator or something like Anthropic’s Claude choosing targets. It’s easier to see that there’s something off about that.

For me, there are two sets of issues. One is, these things raise the risk of accidental war in predictable ways. Imagine a future where you have autonomous submarines — and undersea warfare is a place where this stuff is going to be very influential, because it’s hard to communicate with submarines, and people don’t like being on submarines. But imagine Hunt for Red October–style encounters between autonomous submersibles off the coast of China or off the Pacific coast of the United States: that is not something that you want to hand over to systems where it is quite hard to predict how they will behave in practice.

Then there is a question about what we owe each other, even in warfare, and whether there might be something like a right not to be killed by a robot. People have a reasonable intuition that even in war, there are limits posed by the respect for human life. We don’t directly target civilians. We don’t use weapons that have other effects on the person beyond that required for military necessity. There are prohibitions on, for instance, poison bullets. There are some ways of killing people about which the international community has simply said, “No, we’re not going to do that.”

So the question is: If you are going to fight a war, what do you owe the enemy? One thing you owe them is that a human being makes a decision whether they live or die. Some human being that has to accept responsibility and make that determination — that is something that we owe each other.

This does get complicated. If you think of a future in which I say, “I killed someone with a robot,” then it’s me killing them. And I have made a decision. But if you think of this as a future in which the robot decided who to kill, you would say, “No, that is disrespectful of the humanity of our enemy.”

People mistakenly think that the ethics of warfare is all about winning and the consequences, but it’s simply not. Because otherwise you would target the enemy’s kindergarten; my defense policy could be that if you attack me, I fry your children. That wins a war quickly; it brings about peace. But that is the logic of terrorism.

There is such a thing as “just war theory,” or the international law of war, or military ethics. In that context, we place restrictions on the kinds of weapons that we are willing to see used. I hope that we will add autonomous weapons systems to the list of things that are absolutely prohibited.

Nick French

How exactly do you draw the distinction you just mentioned, between a person using a robot to kill someone versus the robot itself killing someone?

Robert Sparrow

There are two parts to that question. One is a practical component, which is: How much control do I have over the robot’s target, over what it kills? For instance, that Russian system — you could have just fired a lot of drones and hoped that some of them landed on the gas cylinders. Or you could build a system that could recognize gas cylinders, and now you’re more confident about what you are attacking.

There are situations now where the machines clearly outperform human beings. Air defense systems, for instance: Shooting down incoming cruise missiles is something that human beings simply can’t do. If they tried to do it, they would shoot down friendly planes. They would shoot down civilian airliners. You can have a computer-controlled cannon that is quite reliable.

There’s a practical question about how confident you can be that a weapons system will attack a designated target and only that designated target. There are some use cases where you can have a lot of control: anti-submarine warfare, for instance. There are simply not many civilian submarines. My chances of killing civilians, if I’m using a loitering undersea munition, are low. I can accept responsibility; I know that I’ve authorized this. It’s going to go out there, and it’s going to kill a submarine.

Surrender recognition turns out to be a hard problem for these systems. It’s hard to design them so that they recognize surrender. But there are types of conflict in which surrender recognition is already not something that people do. With the exception of surrender recognition, some of these systems can obey what’s called “the principle of distinction,” which is where they’re only going to attack legitimate military targets.

Then you’ve got the case where you say, “Fly around this city and kill anyone who’s my enemy,” or, “Fly around the city and kill anyone who’s holding a rifle.” In some contexts, people carry rifles because that’s what blokes do. In that context, you are not going to have very much control over what it attacks.

That’s the practical component. Can a human being reliably tell what it’s going to attack? If so, then you might say, “Okay, this is an example of a human being killing another human being with a robot.”

There’s also the philosophical question about the nature of the autonomy of the system. If you think about the experience you have when you’re talking to ChatGPT nowadays, it’s easy to fall into a language of agency: “It decided to answer this way,” or “It booked my travel for me.” The more you are using that language about a system, the less it looks as though it’s a human being killing another human being with a robot, and the more it looks as though I’m sending a robot out there to kill people — but that it is doing the killing.

So part of what is necessary here is to pay attention to the language that people are using about these systems and how they describe them. If they are describing them in a way that suggests a human being is not in control and is not responsible for what happens, then that is disrespectful to the enemy’s humanity.

There is a likelihood that command functions are going to be handed over to AI systems. One of the things I’m interested in at the moment is whether the military will be willing to allow decisions that might involve killing their own troops to be handed over to these systems.

The military, for the most part, is on board for the idea that the machine might decide who among the enemy lives or dies. But if you ask, “Are you willing to accept orders from an AI command system, where that AI command system may decide that your job is to go and stand on that hill and make a target of yourself so we can creep around on the left flank and win? Are you up for that?” My suspicion is that the military is going to really balk when their lives are on the line. That tells you something.

Nick French

Let’s talk about this issue of having AI systems in command. You cowrote an article with Adam Henschke in 2023 about the idea of “minotaur warfighting,” which is a very striking image. Could you unpack that metaphor, and explain why you predicted that we’d be seeing more instances of AI commanding human troops?

Robert Sparrow

We’re already seeing it. The film NAZA that just came out discusses Israeli systems that were nominating targets in Gaza — that is precisely what Adam and I were predicting. You have human beings launching weapons at targets that have been selected by machines.

No one ever sells an AI system to skilled professionals by telling them that it’s going to put them out of a job. What you tell them is it will do the grunt work; it will do the stuff that you don’t think of as being a high value-add, and it will free you up to do the important tasks.

There’s a strong Hollywood influence here as well. When the military started planning for a future where robots were playing a role in warfare, they went for a kind of Iron Man model, this idea that you’re going to have a human being surrounded by a cloud of robots, machines. The human being is going to be in command, and the robots will be realizing the human will.

The model that people developed was the idea of “centaur warfighting.” This is an influential concept. The idea is that you have a human head on the body of an animal — that’s the centaur. You have a team of human beings and machines where the machines are following the orders of the human beings. The claim was that this would actually win against fully autonomous systems, because human beings are still capable, in some specific contexts, of superior decision-making.

The idea was that teams of human beings and robots would outcompete either human beings or robots. This was true in the history of chess at one point. People got very excited when the chess computers could beat human beings. But then it turned out that human beings working with chess computers could beat the computers on their own. For a brief period, those human-machine teams would beat either human beings or machines. That’s not actually true anymore in chess; the machines just win now.

Anyway, that’s been the model. It allows the military to reassure the officers that they will still have a job. This is part of a kind of Enlightenment tradition, that the mind is superior to the body; that when it comes to who builds the skyscraper, it’s the architect and not the men on the scaffolding.

Adam and I pointed out that this is quite unlikely. People always thought that the mental stuff, the cognitive stuff was going to be really hard for AI to do. Planning, math, chess: those were all the tests. In fact, the machines got very good at that very quickly. Walking — they couldn’t do. You still can’t build a robot that can walk into your office and empty the rubbish bin. The things that toddlers can do — run around, recognize and pick up objects — turned out to be really hard. And all the cognitive stuff turned out to be doable. But they can’t do the physical stuff.

Think about places where we do have human-machine teams. Think about Uber. For a long while, we’ve had this idea that I’m going to be able to drop a pin and have a robot taxi pick me up. But mostly, what we’ve got is immigrant labor, human beings driving cabs and being told where to go by an algorithm. Or in the big distribution warehouses belonging to Walmart or Amazon, you have people who are essentially the hands of computers. The human labor is to stand and pick things up and put them into a paper bag. [Deciding] where it gets shipped, printing the label, ordering products — all the cognitive stuff is done by AI.

When you look at real-world human-machine teaming, it turns out that the stuff that the infantry can do is much harder. But issuing orders is doable now. So we think that essentially the officers will be replaced, even the generals will be replaced, by computers. But the infantry — the people who have to crawl through the mud and over the barbed wire — they will be human beings. The model there is a monstrous head on a human body, which is the Minotaur. AI commands humans, and humans follow its orders.

We think that these things might actually win, and that people might prefer to be commanded by a machine. I got into this by looking at a project where the US Army wanted to give everyone a sort of Microsoft HoloLens. They’ve all got head-mounted cameras nowadays; they’re all going to have Google glasses. You looked at what people imagined those systems doing; it was threat assessment. You would be told by your glasses, “You’ve got thirty seconds to get over there and shoot that person.” It was very clear that people would become puppets of these threat-assessment algorithms.

People start with the idea that the machines will give you advice. But they also think that they’re capable of superhuman performance. But if my adviser is better than me, I just have to do what it says. The model of second-guessing what the machines say . . .  it’s a very limited period in which that’s true. If they’re genuinely capable of superhuman performance, which is what everyone says, then human beings simply shouldn’t be making these decisions anymore. If the machine tells me to duck, I duck. If I don’t, I die. I’d rather the machine tells me that than a human being who is slower and less reliable.

Again, what we are seeing now, the role that AI is playing in Ukraine or in Gaza, is these algorithms essentially tasking human artillery systems with what to attack. You’ve got an integration of data from every drone, every satellite, and the algorithm is essentially passing targets to people. That’s an early example of Minotaur warfighting.

Nick French

Does that raise distinct moral concerns, if robots are calling the shots? People sometimes raise worries with these sorts of systems that there’s no one to hold responsible for what they do.

Robert Sparrow

The question about responsibility is the same across autonomous weapon systems and AI command functions. What’s interesting is that people respond differently.

If I send a swarm of drones out there to kill enemy tanks and you ask, “Who’s responsible?” I say, I am, because I designed the system and tasked it with killing tanks, and I checked that it worked. Then that should also be true for the people who design the AI command system. But I suspect that people are going to be much less sanguine about the potential of being essentially sent to their deaths by a machine, and that we will expect that some human being is involved there.

There’s a threat of mission creep here too. With something like logistics, the machines are really good at big operations management, with lots of trucks and shipping containers. If the machines are deciding which troops get weapons, then they’re already determining who lives and dies. If the machine has said, “You’re third on our priority list, and you’re not getting any ammunition today. But someone’s got to hold this position while the rest of us retreat, and that person is going to die”; the machine says it’s you. I think people are going to be deeply uneasy about that, even while they’re quite comfortable with sending out these autonomous attack helicopters to destroy enemy tanks or detect enemy troop movements.

This Israeli system, Lavender, that was choosing targets in Gaza: someone built that, and as I understand it, you could move the civilian casualty slider up or down [adjusting the number of acceptable civilian casualties]. People checked the targets, but they spent twenty seconds checking. And someone decided on the level of evidence that was required to establish that someone was a Hamas operative. So there were human beings involved, but at some level, it’s pretty clear that this algorithm was choosing targets, and human beings were just doing what it said.

Nick French

You argue for a ban on autonomous weapons systems, and you’ve said that banning them outright would be more realistic than trying to impose adequate moral and legal constraints. Can you explain your thinking there?

Robert Sparrow

Almost nobody thinks that these things are completely unproblematic. The military itself is really worried: What’s the impact on unit cohesion? How do I know that this thing isn’t going to accidentally kill my troops?

Everyone thinks putting them in charge of a nuclear second strike, for instance — only the real Dr Strangeloves are willing to go there. There is a historical precedent for what’s called “launch on warning.” There’s a period where it seems as though the Russian nuclear arsenal was designed this way. You connect the early warning radars to the computer, to the fire control, because you’re worried that someone will carry out a decapitation strike, and this gives you a reliable second-strike capability. It’s incredibly dangerous, given what we know about the limits of our system.

Most people, when they’re thinking about this stuff, say it needs regulation. But everyone also recognizes that this is a very difficult technology to regulate. You can have a system that is running on remote control, with a human being in the loop, and then someone plugs a USB drive into it, and all of a sudden it’s driving itself.

It seems to me that if you want to have an impact on the future of warfare, drawing some bright lines that make it politically difficult for people to mobilize this technology is more likely to be effective than something that on paper might be more justified. There’s this kind of debate about, when you’re banning land mines: Do you just say no antipersonnel land mines, or do you say no mines that don’t deactivate themselves? There are different technological solutions to civilian casualties from land mines. But if you just say, “Look, none of them are allowed,” that establishes a clear moral expectation and makes it easier, for instance, for people to refuse to work on them.

You really are going to need the engineering profession to say, “We’re not going to build machines that are choosing their own targets.” The more sophisticated attempts to regulate these systems are harder to verify, and they give people wiggle room. It’s like, “Okay, my system isn’t fully autonomous. . . . ” We have to pay attention to what’s called the expressive function of regulation, which is about making clear that we all agree on a prohibition.

If you look at the history of, say, the prohibition on asphyxiating gases, you can find all these people saying “This is a much more humane method of warfare”; “We are going to win wars more quickly”; “What’s the difference between being killed with a gas or with small arms? Death by small arms is horrible.” The blanket prohibition cuts all that stuff short.

This is an argument about how social and political change comes about in this sphere. Big, bold gestures are much more likely to be effective than a creeping, bureaucratic attempt to fine-tune things. That just extends the argument endlessly and allows people wiggle room, such that they don’t confront the true horror of what they’re doing.

Nick French

I wanted to ask you what you make of the recent news cycle of panic about AI — the idea that it poses an existential threat to humanity, and that we need some kind of pause or slowdown of development.

Robert Sparrow

The first thing to say is that it’s incredibly hard for anyone outside of the inner circle in these companies to really understand what the models are capable of. There’s still a lot of stuff about the Hugging Face incident that hasn’t been revealed, because there’s quite a bit of internal censorship by the organizations.

So much of how we think about the future has been determined by science fiction. The engineers set out to build futuristic stuff. What does the future look like? It’s like Star Trek from the 1970s. There’s a clear influence of science fiction here that includes the idea of robot apocalypse. There’s a really interesting race and class politics to robots here too, because the whole idea of a robot originally was essentially of a kind of worker. Fears of robot uprisings were fears about socialism.

All of that stuff makes it hard to know how grounded any of these panics are. That said, if I said there was a 10 percent chance that al-Qaeda would kill everyone in the world — if you imagine this were a terrorist threat and people in the defense industry said, “We think there’s a 10 percent chance that this terrorist organization would kill everybody” — no one would go, “La la la, let’s see what what happens.”

So, I’m inclined to take them at their word. Major figures in the history of AI have come out and said this stuff is an existential threat. On the one hand, there’s nothing so stupid that you can’t find a professor who believes it. Just because you’re a clever physicist or computer scientist doesn’t mean that you are well qualified to talk about the future, because the future is not just a technical question. It’s political — it’s something that we can change.

But given the number of these people who say that this is a danger, that this will kill us all, I think we should close all the labs. Just shut it all down. I think the idea that you would proceed cautiously or have a pause is really strange, given what they’ve said about the nature of the threat.