Skip to content
Faultline Prefer us on Google

Migration taz

Computer scientist on AI in asylum procedures: “As reliable as randomly asking people on the street”

The federal government wants to use AI in asylum procedures. Computer science professor Alexander Steen sees hardly any advantages – and many dangers. more...

Deutsch (original) ·Auto-translated to English

taz: Mr. Steen, the federal government wants to use AI programs in asylum procedures. A corresponding draft law is currently being discussed by the Bundestag. Is that a good idea?

Alexander Steen: It's a good idea to speed up the procedures and digitalize them sensibly. But what is in the draft law seems ill-thought out and gives the authorities involved a blank check for the unreflective use of technology.

Born in 1990, is a professor of computer science at the University of Greifswald and spokesman for the AI ​​department of the Society for Computer Science.

Steen: Even the choice of words is remarkable. An important area of ​​application is referred to in the draft as “monitoring”. That's more of a euphemism. Monitoring implies that asylum decisions are about passive observation. What actually seems to be meant is an active intervention in the asylum procedure.

taz: Specifically, the draft law stipulates that AI should analyze completed asylum procedures and make the results available to employees of the Federal Office for Migration and Refugees (Bamf). What's so bad about that?

Steen: People sometimes trust recommendations from automated systems more than their own assessment. The technical term for this phenomenon is “automation bias”. Scientific studies also prove this for human-AI interactions. Even if in the end there are still people who formally decide on asylum applications, there is a risk of a situation in which AI recommendations are increasingly being approved without critically questioning them.

taz: It has to be possible to counteract that.

Steen: The automation bias can even be observed when the AI ​​systems explicitly point out possible sources of error or provide explanations when outputting data. The draft law does not provide for any protective mechanisms here.

Steen: There is no uniform architecture of such systems defined and no usage process prescribed. Each authority should decide autonomously how to use AI. The draft also ignores the costs of necessary training for the decision-makers who are supposed to interact with the systems.

taz: As long as the data that the AI ​​outputs is correct, it's not so bad.

Steen: Even if the facts are correct, that doesn't mean that all the details of a case are evaluated and weighed up. AI models could detect and reproduce statistical relationships that have no causal basis without being noticed. People attribute an objectivity to automatically collected data that they do not have to this extent.

taz: How should you imagine this in practice?

Steen: An example would be that AI discovers a connection in the files according to which a certain group of people submitted forged documents at an above-average rate.

taz: For example, people of a certain nationality, region or ethnic group.

Steen: If someone from such a group of people submits an application and the AI ​​system points out the connection in the data, then suspicion would of course immediately fall on this person. In case of doubt, this could lead to decision-makers no longer examining the application objectively, consciously or unconsciously. But a connection calculated in this way is not a probability that says anything about an individual.

taz: But such data is not entirely without significance, is it?

Steen: AI systems make mistakes. And data sets can contain statistical biases that can lead to irrelevant or incorrect relationships. Another factor is that the AI ​​systems are trained with data that may itself be biased. Detecting and preventing something like this is expensive and time-consuming.

taz: The draft law should also allow the authorities to have dubious information that asylum seekers provide during the procedure automatically checked online. At least that makes sense?

Steen: The same applies to people: wanting to verify personal data on the Internet is about as reliable as asking random people on the street about it. Even 20 years ago it was easy to fake photos, texts and profiles on the Internet. With today's technical possibilities it is much easier.

taz: Isn't that exactly an argument for using AI?

Steen: If someone takes photos from my website and uses them to create a fake Facebook profile of me, then even an AI system cannot reliably determine that I am not behind it. I could have created a profile. There is also a risk that AI systems will misinterpret data. Most recently, there was the case of a chief physician from the Rhineland who was falsely suggested by the AI ​​mode of the Google search that he was a sexual offender. The program had mixed the head physician's data with that of another employee at the clinic.

Steen: Much smaller errors in interpreting the data are enough. If the system falsely claims that I went on vacation to my home country, that alone could endanger my asylum application. Not to mention the data protection issues.

Steen: AI systems use search engines, just like people do. The data streams that go to the search engine operators allow conclusions to be drawn that they come from the asylum authorities. And that in turn makes the data interesting, for example, for repressive regimes that want to persecute people even after they have fled to Germany.

taz: AI programs must have some use in the asylum process.

Steen: When in doubt, the risk is greater than the benefit. In any case, precise specifications are needed as to how and for what they should be used. The aim should not be for machines to do the work and for humans to only come in at the very end of the process and then make a decision. People must be involved in all sub-processes and therefore be forced to deal with the content of the data and the decision.

Steen: I currently see little potential there, but I also don't know the specific specialist procedures of asylum administration. But digitalization doesn’t always have to mean “AI”. It is important to find sensible and safe optimization options and implement them.

taz: Can AI programs at least be used sensibly by other authorities?

Steen: When it comes to vulnerable groups of people, the unreflective use of AI can be very dangerous. But there are also areas in which AI systems can be used sensibly. Especially where work is carried out according to a clear scheme and there is less discretionary decisions. “AI” does not necessarily have to mean that it is about statistical learning processes, which is currently being talked about so much. Fixed, logical instructions for action can also be used. Current research focuses in particular on the combination of both approaches. But at the end of the day, people have to stay involved.

Originally published bytaz
Read the original