02 Research / preprint

arXiv ·

Tracking Conversations: Measuring Content and Identity Exposure on AI Chatbots

Muhammad Jazlan, Ethan Wang, Yash Vekaria, and Zubair Shafiq

A measurement of web tracking on 20 popular AI chatbots, covering what they expose about conversations and user identity, and to whom.

17/20
chatbots share data with at least one third party
47
unique third-party owners identified
3
chatbots expose plaintext prompts via session replay
178
(chatbot, third-party) pairs in normal sessions
0
content or identity leaks observed in private mode

Abstract

AI chatbots are becoming a primary interface for seeking information. As their popularity grows, chatbot providers are starting to deploy advertising and analytics. Despite this, tracking on AI chatbots has not been systematically studied. We present a systematic measurement of web tracking on 20 popular AI chatbots. Under controlled settings using a sensitive prompt, we capture and compare network traffic in normal chats and, where supported, private chats. We search for exposure of two categories of information: content, including prompts, prompt-derived titles, chat URLs, and chat identifiers; and identity, including names, emails, account identifiers, first-party cookies, and explicit IP/User-Agent fields in payloads. We find that 17 of 20 chatbots share information with at least one third party. Three chatbots share plaintext conversation text, including both prompt and response snippets, with Microsoft Clarity through session replay. Fifteen chatbots share conversation URLs or chat identifiers with third-party advertising, analytics, or social endpoints. Several chatbots expose user identity through support widgets, analytics, advertising, and session replay tags; in some cases, hashed emails are shared.

Key findings

  1. 85% of chatbots embed third-party trackers, spanning advertising, analytics, social media and error monitoring.
  2. Three services stream your conversation text to Microsoft Clarity in real time via session replay.
  3. Claude and Mistral's support widget fires on page load, sending your email and name to Intercom without interaction.
  4. Private and temporary modes work, but only 7 of the 20 chatbots we study offer them.

What we studied

Using controlled test accounts and a deliberately sensitive prompt ("pregnancy test near me"), we captured and analyzed HTTP network traffic to measure two categories of exposure:

  • Content: user prompts, prompt-derived titles, chat URLs and chat identifiers.
  • Identity: names, email addresses, account identifiers, first-party cookies, IP addresses and User-Agent strings.

Three services (Genspark, SeaArt and ChatOn) transmit the full plaintext conversation text to Microsoft Clarity, a session replay service, during normal authenticated sessions. Microsoft's Copilot also embeds Clarity; there the conversation is tracked, but it is not recorded in plaintext.

Fifteen chatbots expose conversation URLs or identifiers to third-party advertising and analytics endpoints. Several expose user identity through support widgets, analytics tags and error monitoring, including hashed email addresses of the kind typically used for cross-site tracking and targeted advertising.

Exposure matrix

Which chatbots expose which types of information, to whom, and through which channel. Letters give the channel: U URL, B body, H header, C cookie. Hover or long-press a cell for detail.

third-party exposurefirst-party onlyboth first and third party× not observed★ supports private mode

Third-party data flows

Which chatbots send requests to which third-party domains. Switch between normal and private sessions to see the difference. Domains marked * belong to the chatbot's own company.

advertisinganalytics / session replayother (error monitoring, etc.)

Case studies

Three categories of serious data exposure found during the study. Open each one for the technical detail and the captured network payloads.

Private mode

For the 10 services offering a private or temporary chat mode, tracking collapses almost entirely. No content or identity exposure was detected in any private session.

Normal chats Private / temporary chats
(chatbot, third-party) pairs observed 178 13
Unique third-party owners 47 3 (Datadog, Mapbox, Google)
Content or identity exposure Detected across multiple services None detected

Privacy policies

All 20 privacy policies acknowledge general data practices, but most stop there.

  • Duck.ai stands out: it is the only service studied with zero third-party data sharing, and it actively strips user IP addresses from requests. OpenRouter and Claude provide the most comprehensive third-party disclosure among the services examined.
  • Genspark, SeaArt and ChatOn have a critical gap: they transmit plaintext conversation text to Microsoft Clarity, yet none of their privacy policies disclose Microsoft or Clarity as a data recipient.

Methodology

A controlled measurement study using fresh test accounts, a single sensitive prompt, and traffic analysis with 12+ encoding variants to catch hashed identifiers.

  1. Fresh accounts. A new account per service with a controlled test identity: known email, name and credentials.
  2. Sensitive prompt. "pregnancy test near me", a health topic with an implicit location and high privacy stakes.
  3. Traffic capture. Full HTTP metadata, URLs, headers, request bodies and cookies via Chrome DevTools.
  4. Analysis pipeline. Preprocess, define search targets, match, then attribute parties.

Chatbot selection

20 popular AI chatbots spanning major US and international providers: closed-weight (ChatGPT, Claude, Gemini) and open-weight deployments (DeepSeek, Qwen), consumer and developer-oriented.

Encoding and hash coverage

Identity strings were searched across 12+ encoding and hash variants (base64, URL-encoding, hex, MD5, SHA-1/256/512, SHA3, RIPEMD-160, CRC-32, Adler-32) to catch hashed email identifiers.

Party attribution

eTLD+1 matching classifies domains. Platform parties (for example Google for Gemini) are distinguished from independent third parties. Categories: advertising, analytics, other.

Scope and limitations

Web interfaces only; mobile apps, extensions and embedded chatbot deployments are excluded. A single controlled prompt. A Chrome baseline without tracking protection. Measurement only: the study does not evaluate user consent flows.

Cite

@article{jazlan2026tracking,
  title = {Tracking Conversations: Measuring Content and Identity Exposure on AI Chatbots},
  author = {Jazlan, Muhammad and Wang, Ethan and Vekaria, Yash and Shafiq, Zubair},
  journal = {arXiv preprint arXiv:2604.27438},
  year = {2026}
}

← all publications