Indiscreet language models | AI chatbots pass on content from conversations to companies

The researchers found hardly any differences in terms of indiscretion between free and paid – and therefore mostly ad-free – pro versions of the language models. Photo: dpa/Philip Dulian Anyone who tells a language model about illnesses, money worries or relationship problems or uses it for work assumes that it is a confidential conversation between two parties - even if the other person is an AI chatbot. A study by the Madrid research institute Imdea Networks now shows that this assumption does not apply to many large services - at least with regard to private conversations.

ChatGPT, Claude, Perplexity, Grok, Gemini, Mistral AI, Meta AI, DeepSeek and Microsoft Copilot were examined. Six of the nine web versions and three of the eight Android apps transmitted conversation-related data to tracking and advertising services, including Google, Meta and Tiktok.

It is common, for example, to pass on a summary title of conversations with the AI ​​chatbot. This is assigned automatically by the systems so that the topic can be found more easily in the history. The question about Parkinson's symptoms becomes the title "Parkinson's symptoms in the early stages", and a mortgage question becomes a title with salary and loan amounts. According to the study, Gemini, Grok and Mistral passed on such titles to third parties.

netzpolitik.org, among others, reports on the results of the study. Researcher Narseo Vallina-Rodriguez told the portal that even seemingly harmless chat titles could reveal very personal preferences, concerns and life circumstances. If the titles are transmitted together with email addresses or user IDs (also disguised as so-called hashes), it is also easier to identify which account the information belongs to. According to the study, addresses at Anthropic, Mistral and Perplexity went to third parties, and user IDs at ChatGPT, Claude, Le Chat and Perplexity also went to third parties.

The study also shows that web and app versions differ. In the apps, for example, the summary titles are transmitted less frequently, but other identifiers are sent to commercial third parties. However, four apps transmitted account-related IDs, two email addresses and three a number sequence that is actually only used to count advertising and can be reset. At Grok and Perplexity, this advertising identifier was transmitted along with persistent user or installation IDs, according to the study. The protection provided by the reset evaporates.

The sharing is particularly far-reaching with Grok, the chatbot from Elon Musk's xAI company. In the browser, Grok transferred chat title and chat link to Meta, Tiktok, Google and Twitter Analytics, among others. By default, these links are visible to everyone in both the free and paid versions - unless users switch them off. Third parties can view the entire chat history including entries and replies.

Internal links that are created by the chatbots for every conversation are also a security gap for supposedly private communication. ChatGPT, Claude, Grok, Mistral and Perplexity also transmitted these to third parties.

Another way in which data gets out is the conscious sharing of a chat by the users themselves. Grok and other providers offer so-called share links for this purpose. Anyone who releases such a conversation only expects it to be viewed by selected people. The researchers examined which information flows to which companies. In shared Grok conversations, a particularly large number of details were passed on to third parties. Meta received the title of the chat and the last prompt entered by the person using it. Tiktok also received a screenshot of the conversation.

To check whether such links are actually accessed in Grok, the researchers incorporated decoy links into shared conversations. They then registered 70 accesses from 70 IP addresses from 14 countries, sometimes only days later. Two thirds of the hits came from the USA, even though the conversations took place in the European Union.

Across all nine services, the researchers found 44 institutions that receive data, including 34 advertising and tracking services. Google was most strongly represented with Firebase, but also Meta and the well-known Internet service Datadog. During DeepSeek, Imdea Networks came across Chinese tracking services that were hardly known in this country.

The results of the study also show the limits of self-help against AI chatbots that are willing to provide information: Even after rejecting all unnecessary cookies, 80.8 percent of the observed third-party trackers remained active in the web versions. The researchers also found hardly any differences in terms of indiscretion between free and paid – and therefore mostly ad-free – Pro versions. ChatGPT is an exception: On the website, the paid version supports permanently saved consent settings - however, around 95 percent of users work with the free version of ChatGPT.

The researchers also emphasize that their measurements are a snapshot and that the mere presence of a third party does not prove that data is flowing for advertising purposes in every situation. The study by the non-profit organization Imdea Networks has not yet been independently verified; However, such a peer review is announced.

Other media have corporations or billionaires behind them. We have the nd.Genossenschaft and readers like you. That makes us independent and that’s exactly how we want to remain. But it also means: without you the calculation won't work. We need 300,000 euros by the end of the year.

You can donate directly here, one-time or regularly, so that nd stays.