When a company starts using AI tools, one of the first questions it should ask is: what data are we sending outside the company, to whom, and under what terms?
The question is harder than it looks, because data flows in ways that aren’t always obvious. An employee who pastes a client contract into ChatGPT to get a summary is sending confidential information to OpenAI’s servers. A customer service system built on an external language model processes conversations containing customers’ personal data on a third party’s systems.
This article explains the specific risks, how to assess them, and which measures let you manage privacy without blocking AI adoption.
The most common privacy risk vectors
Sending sensitive data to third-party APIs. Most of today’s generative AI tools (ChatGPT, Claude, Gemini) work by sending data to the provider’s servers to process the request. For non-sensitive data — public documentation, generic text, code without proprietary information — the risk is low. For customers’ personal data, confidential financial information, health data, or company strategy, you need to check whether the provider’s privacy policy is compatible with the GDPR and with your confidentiality commitments.
Training data that includes personal information. If a company builds its own model and trains it on historical customer data, GDPR obligations apply: a legal basis for processing, data minimization, and possibly pseudonymization of the training data to reduce risk in the event of an incident.
Models memorizing data. Large language models can sometimes “memorize” fragments of their training data and reproduce them in responses. With AI providers that train on their users’ data (something enterprise plans usually rule out contractually), this can become a channel for information leaks.
Conversation logs and storage. AI platforms often store conversations by default to improve the service. Enterprise plans offer no-retention options. For uses involving sensitive data, check what the provider stores and for how long.
The GDPR applied to AI projects
The GDPR doesn’t prohibit using personal data in AI systems. It sets out the principles and obligations for doing so lawfully:
Legal basis. Processing personal data requires a legal basis. The most common in business settings are performance of a contract (for customer data needed to provide the service), legitimate interest (in some internal analytics cases), or explicit consent.
Data minimization. Use only the data strictly necessary for the system’s purpose. If you don’t need customers’ names or addresses to predict churn, training the model without them is better practice.
Transparency. If the AI system makes decisions that affect people (customers, employees), those people have the right to know the system exists and, in some cases, to request human review of decisions that affect them.
Data processing agreements. When you use third-party services (AI APIs, cloud platforms), you need to sign the relevant data processing agreements (DPAs) to ensure the provider complies with the GDPR.
Options that reduce risk
Enterprise versions of the tools. OpenAI, Anthropic, and Google offer enterprise versions of their models with extra guarantees: your data isn’t used to train future models, conversation retention can be turned off, and data processing agreements are available. For companies handling sensitive data, enterprise plans are the starting point.
Open-source models deployed on premises. Tools like Ollama and LM Studio let you run high-capability language models on your own infrastructure, without sending data to external servers. Performance is somewhat below the best API models, but it’s enough for many business use cases, and data privacy is fully guaranteed.
Pseudonymizing and anonymizing training data. For projects that build their own models with customer data, pseudonymization — replacing direct identifiers with codes — reduces risk in a security incident and may be enough to meet the GDPR’s minimization principle.
An internal AI use policy. Many companies are now defining which types of data can and can’t be sent to external AI tools. A clear policy stops employees from deciding on their own which data is appropriate to share with which tools.
Frequently asked questions
Do you have to tell customers if AI processes their data? It depends on the use. If AI processes customers’ personal data to make decisions that affect them, transparency is a GDPR obligation. If AI is used for internal analysis that doesn’t directly affect customers, individual notification isn’t necessarily required, although the company’s privacy policy should reflect its use of AI.
Are open-source AI models always safer for privacy? Not necessarily. Privacy depends on where and how the model is deployed. An open-source model running on external cloud servers carries the same risks as a proprietary one. The privacy advantage appears when the model runs on the company’s own infrastructure.
Does the EU AI Act add privacy obligations? Yes, especially for high-risk systems, which have data governance obligations that complement the GDPR: training data must be relevant, representative, and free of inappropriate bias. These requirements are in addition to the GDPR, not instead of it.