How Public-Cloud LLM Telemetry Can Leak Classified Information

An analyst at a sovereign institution pastes a paragraph of a draft policy into a commercial chat assistant to get a quick rephrase. The text never trains the model, the vendor's marketing assured everyone of that. Six weeks later, the same paragraph appears verbatim in a foreign legal-discovery production, attributed by IP and timestamp to a session originating from the institution's office. The model did not leak it. The telemetry did. This is the failure mode that "we don't train on your data" was never built to prevent, and it is the reason classified and sovereign-restricted workloads do not belong on public-cloud LLM endpoints, regardless of how strongly the privacy page is worded.

What "telemetry" actually contains in commercial LLM APIs

Telemetry on a commercial LLM endpoint is a much wider surface than most buyers assume. It is not a single log line. It is the set of every byte the vendor's infrastructure observes, derives, or stores while serving the request, plus everything the operations team needs to keep the service running, billed, and abuse-free. A short, non-exhaustive list:

• Prompt logs. The full text of the user's prompt, including any pasted documents, code, names, and identifiers, captured at the API gateway before the model sees it.
• Completion logs. The full text of the model's response, stored on the same path as the prompt for replay, debugging, and abuse analysis.
• Latency, token counts, and shape metadata. Even when the body is redacted, input-token count, output-token count, time-to-first-token, total latency, and request size remain. With enough samples, this is itself a side channel.
• Error stacks and partial completions. A request that triggers a server-side error often persists in error-tracking systems with the offending input attached, on a different retention clock.
• Model version IDs, routing decisions, and tenant metadata. Which checkpoint served the request, which region, which tenant key, which abuse-monitoring policy matched.
• Classifier scores and abuse-monitoring outputs. Real-time safety classifiers run on every request. Their outputs are stored, often on longer retention than the prompt.

Each of these is a legitimate engineering need for the vendor. None of them is something a classified workload can tolerate leaving the perimeter.

Three concrete leak channels

The risk is not abstract. Three concrete channels carry classified content out of the institution and into systems the institution does not control.

Prompt-level metadata. Even if a vendor honours a "do not store the body" agreement, request metadata persists for billing, capacity planning, and incident response. A pattern of long-context requests at 02:00 Muscat time, against a specific embedding namespace, with a specific model version, can fingerprint a sensitive workflow even when no body is retained. Adversaries with access to network metadata, lawful-process metadata, or vendor-side analytics can reconstruct considerable structure without ever reading a prompt.

Model-output retention. By default, OpenAI's API retains inputs and outputs for up to 30 days for abuse monitoring under its standard data-controls policy. Anthropic now retains API logs for 7 days, reduced from 30, with prompts flagged by safety classifiers held up to 2 years and the classifier scores up to 7 years, per its privacy centre. These windows are reasonable for commercial use. They are catastrophic for a draft cabinet memo, a classified intelligence brief, or a court-sealed financial filing that gets pasted into a prompt.

Abuse-monitoring queues. Every major LLM endpoint runs automated abuse detection. AWS Bedrock's abuse-detection documentation confirms that classifier metrics derived from inputs and outputs may be compiled and shared with third-party model providers, even when the raw text is not. Anthropic's policy explicitly preserves flagged content for up to two years to improve detection. The intent is benign. The effect, for a sovereign workload, is that a single mis-routed prompt can sit in a foreign vendor's review pipeline, accessible to its trust-and-safety staff, for years.

Why "we don't train on your data" doesn't address it

The training pledge has become the headline privacy commitment of the major LLM vendors, and it is genuinely meaningful. It rules out the worst case where a competitor or adversary later prompts the model and recovers a verbatim quotation of your sensitive text. Buyers, especially in sovereign and regulated markets, hear it and conclude the privacy question is settled. It is not.

Training and telemetry are two different bargains. The training bargain is about what enters the next checkpoint. The telemetry bargain is about what touches the vendor's infrastructure, gets logged, gets indexed for abuse review, gets cached, gets archived for incident response, gets disclosed under lawful process, and gets read by trust-and-safety staff. A vendor can honestly say it does not train on your data while every telemetry-side exposure remains alive. The risk surface for a classified workload is the telemetry surface, not the training surface, and that surface is governed by retention windows, abuse-monitoring policies, and the vendor's home-jurisdiction legal regime. For the jurisdictional side, see our pillar on GCC sovereign data CLOUD Act compliance.

Even Zero Data Retention agreements, where they exist, narrow the surface but do not eliminate it. The data still transits the vendor's network, is processed on shared infrastructure, and lives, briefly, inside a foreign legal jurisdiction. For commercial work that is fine. For classified work it is not.

The on-prem invariant: zero outbound, signed-bundle updates

The architecture that resolves this is straightforward, and it is the only one that resolves it cleanly: the model and every byte of its telemetry stay inside the institution. An on-premise sovereign appliance has zero outbound network connectivity. Logs land on disks the institution owns, are read by the institution's auditor, and are deleted on the institution's own retention schedule. There is no abuse-monitoring queue in a foreign datacentre because abuse monitoring, when needed, runs locally. There is no classifier-score archive that a third-party trust-and-safety team can read because there is no third party in the loop.

Updates flow the other direction, inbound only. New model checkpoints arrive as signed weight bundles delivered on tamper-evident media or across an inbound-only data diode. The institution verifies the signature against a published hash, runs a local acceptance suite, and promotes the bundle through a documented two-person change window. The patch lag is real and accepted: the enclave is days or weeks behind public release, never hours. That latency is the price of the invariant, and for sovereign workloads it is the right trade. Related architectural detail is in our pieces on air-gap network architecture for AI and hyperscaler AI data residency in Oman.

If your institution is evaluating where its AI workload should live and the telemetry question is on the table, the next step is a closed conversation. Email [email protected] or message +968 9889 9100 to arrange a one-hour briefing in Muscat or at your site anywhere in the GCC. Pricing is by quotation, sized to your concurrency and classification target.