What actually leaves your network when you use an AI model

When a team member pastes a contract into an AI assistant, "the AI" is not a feature inside your network. It is a service run by someone else, and the text travels to it. Most privacy questions about AI reduce to one: what leaves, where does it go, and how long does it stay?

What a single request actually carries

A request to a hosted model usually includes four things:

1. The prompt the user typed.

2. Attachments such as PDFs, spreadsheets and screenshots.

3. Retrieved context. If the tool searches your documents to ground its answer, those excerpts are added to the request.

4. History. Earlier turns of the conversation are resent so the model can follow the thread.

An assistant wired into your file store therefore sends passages from files the user never opened. That is why the access rules on the underlying documents matter: Microsoft's documentation says Copilot surfaces only organizational data a user already has at least view permission to see ^[3], which makes sloppy sharing settings a privacy problem, not just an IT tidiness one.

Training, retention and residency are three separate questions

Vendors tend to answer the comforting question (are you training on my data?) and leave the others to the fine print.

Training. OpenAI states that data sent to its API is not used to train or improve its models by default, unless the customer opts in ^[1]. Microsoft says prompts, responses and data accessed through Microsoft Graph are not used to train the foundation models behind Copilot ^[3].

Retention. Not training is not the same as not storing. OpenAI says abuse-monitoring logs, which can include prompts and responses, are generated by default and kept for up to 30 days, longer in some cases ^[1]. Anthropic says API inputs and outputs are automatically deleted from its backend within 30 days, with exceptions: content flagged for usage-policy violations may be kept for up to two years ^[2].

Residency. OpenAI describes data residency as a project setting that stores eligible content at rest in a chosen region, while noting that in-region inference is available only in some regions and that account and billing data may be processed elsewhere ^[1]. Storage in one place does not guarantee processing in the same place.

The persistence trap

Feature choices change retention. OpenAI's table lists several endpoints, including stored conversations, vector stores, batches, fine-tuning jobs and files, that are not eligible for zero data retention and keep application state until it is deleted ^[1]. A team that builds a document-search feature on a vector store has made a retention decision, whether or not anyone wrote it down.

Web search is another quiet exit. Microsoft notes that when Copilot uses web search it generates a search query from the user's prompt and sends it to the Bing search service ^[3]. A prompt that contains a client name can become a search term.

Why this is a policy problem as much as a technical one

In 2023 Samsung restricted employee use of generative AI after an engineer accidentally uploaded internal source code to ChatGPT; the company's stated worry was that data shared with chatbots sits on external servers where it is hard to retrieve or delete ^[4]. The episode illustrates a pattern: the exposure here came from a person pasting something, not from an attacker.

Four questions to settle before sensitive data goes in

1. Which tier are we on? Business or API tier, not a personal account.

2. What is retained, and for how long? Get the number from the vendor documentation, including logs and any stored files.

3. Is zero retention or regional storage available, and what does it exclude? Approval requirements and endpoint exceptions apply ^[1].

4. What does the tool pull in on its own? Connectors, retrieval and web search all widen what leaves.

Classify your data first (public, internal, confidential, restricted), then decide which classes may go to which tools. For most businesses, the practical implication is that the first AI policy should be a one-page data-handling rule tied to approved, business-tier tools rather than a blanket ban.