Secrets and sensitive data in LLM applications

An LLM application handles two families of sensitive material, and they fail in different ways. The credentials it uses leak through a file, an environment dump or a log. The content it sends — the prompt, the documents retrieved into it, the answers — leaves the machine and lands in someone’s retention policy, or in your own logs. Both need a decision rather than a default; neither is settled by the model.

Credentials: where they live

The convention is old and unambiguous: config, including credentials, is separated from code. The twelve-factor formulation lists “credentials to external services” among the things that vary between deploys and states the requirement as “strict separation of config from code”, with a litmus test worth remembering — “whether the codebase could be made open source at any moment, without compromising any credentials” — and the recommendation to store config in environment variables. The security guidance adds the specific prohibition: secrets “should never be hardcoded”, including through container build arguments that “can easily leak with the container”, and it lists what is in scope — “API keys, database credentials, IAM permissions, SSH keys, certificates” and the like.

Beyond “not in the repository”, the same guidance describes a small set of practices that scale:

centralise the “storage, provisioning, auditing, rotation and management” of secrets rather than scattering copies;
least privilege, with fine-grained access controls, so that “engineers should not have access to all secrets in the secrets management system”;
rotate, and automate the rotation, “significantly reducing the risk of compromised credentials”;
audit access, and monitor where and by whom a secret is used, treating an unexpected source as a compromise signal;
encrypt at rest in the store, and revoke credentials when they are “no longer required or potentially compromised”.

An engineering term worth knowing here is the cryptoperiod: in key-management vocabulary it is “the time span during which a specific key is authorized for use” — the formal framing of the question “how often do we rotate this, and why”. Its counterpart in practice is the distinction between machine credentials, which can rotate on a schedule, and user credentials, which the same guidance excludes from routine rotation and rotates on suspicion or evidence of compromise.

The local case inverts the default. A local inference engine normally runs without authentication at all: in a llama.cpp server, --api-key KEY is documented with “default: none”, with --api-key-file for a list of keys and --ssl-key-file / --ssl-cert-file for TLS. So the key is not a hardened setting to relax — it is a control you have to switch on, and until you do, anything that can reach the port can use the model. The health endpoint is deliberately “public (no API key check)”, which is appropriate for a readiness probe and worth knowing when you design what sits in front of the server.

On the client side the failure is mundane and constant: credentials reproduced in logs, in error messages, in support tickets and in shared screenshots. Whatever the store, the rule that does the most work is that a secret has exactly one place it lives and is never echoed.

Content: what leaves the machine

The second family is the data that flows through the request. Four statements are worth having in mind, each of which is somebody’s policy rather than a property of the technology:

A remote service receives the prompt and the response. The provider’s own statements are the only authority on what happens next, and they are specific. OpenAI states that, “As of March 1, 2023, data sent to the OpenAI API is not used to train or improve OpenAI models (unless you explicitly opt in to share data with us)”, that abuse-monitoring logs “are generated for all API feature usage and retained for up to 30 days, unless longer retention is required by law”, and that Zero Data Retention, which also treats the store parameter as false, is available to eligible customers on approval. Anthropic states that “By default, we will not use your inputs or outputs from our commercial products (e.g. Claude for Work, Anthropic API, Claude Gov, etc.) to train our models”.
Sensitive information disclosure is a named risk, not a hypothetical. OWASP’s LLM02 entry covers the leakage of sensitive information through model outputs — personal data, credentials, business data — and its section on mitigations contains the sentence that matters most for prompt design: such restrictions “may not always be honored”. A policy that consists of telling the model not to reveal something is not a control; it is a request, in the same category as any other instruction in the prompt.
Data privacy is a recognised risk class in AI systems generally. The NIST generative-AI profile lists “Data Privacy” among its risks, describing it as “impacts due to leakage and unauthorized use, disclosure, or de-anonymization of biometric, health, location, or other personally identifiable information or sensitive data”.
Local does not mean safe by omission. Sending nothing to a third party removes one exposure path and leaves the others: the engine’s own logs, your client’s logs, the prompt history on a shared machine, and the raw model output kept “just while debugging”.

A minimal checklist

Classify before sending. Anything that would be unacceptable in an email to the provider is unacceptable in a prompt — including the contents of a document you pasted for summarisation.
Never put credentials in prompts. They are the easiest thing to leak and the hardest to notice; a key pasted into a prompt belongs in the revocation queue, not in the log.
Know the policy you are relying on, and its date. Retention and training statements change; the ones quoted here are recorded with their reading date in the sources register.
Keep the key out of the code path that logs. Redact headers and configuration dumps; a stack trace with an Authorization header in it is a credential disclosure.
Give the local engine a key the moment it stops being local — --api-key plus TLS, or a reverse proxy that terminates both.

Level and prerequisites. L2 — operational: the reader must be able to place a credential, decide what may be sent, and name the policy the decision depends on. The risk framing for personal data belongs to the L1 sheet on privacy and data disclosure in this area; this sheet is about handling. Prerequisites: the placement sheet of this batch (where inference runs) and the API sheet (where the key is presented).

Where to go next

Automation & AI — the macro-area this sheet belongs to.
Operations and risk — the node this sheet sits in.

References

OWASP — Secrets Management Cheat Sheet — the scope of “secrets”, centralisation, least privilege, rotation, auditing, encryption at rest and revocation.
The Twelve-Factor App — III. Config — strict separation of config from code, credentials among config, and the open-source litmus test.
NIST — SP 800-57 Part 1 Rev. 5 — the definition of a cryptoperiod.
OWASP — Top 10 for LLM Applications (2025), LLM02 — sensitive information disclosure and the limits of prompt-level restrictions.
NIST — AI 600-1 — Data Privacy as a risk class.
OpenAI and Anthropic — policy pages — the retention and training statements quoted here.
ggml-org — llama.cpp server — --api-key (default none), --api-key-file, the TLS options and the public health endpoint.