Start with request facts, not prompt text
A failed AI request needs an explanation, but your logs do not need a copy of the conversation. Prompts can contain customer names, private documents, source code, and credentials pasted by mistake. Saving them turns an ordinary debugging tool into another place where sensitive information can escape.
The safest first lap is simple: record what happened without recording what was said. Keep enough information to identify failures, understand token usage, and connect related events. Leave prompts, model responses, uploaded files, and tool results out of routine logs.
Build a small allowlist of log fields
An allowlist names the fields your application is permitted to log. Everything else stays out. This is easier to reason about than collecting an entire request and trying to remove every possible secret afterward.
- Request identifier: Generate a random identifier for each request. Do not derive it from the prompt, email address, or API key.
- Timestamp and environment: Record when the event happened and whether it came from development, testing, or production.
- Model and operation: Record the requested model and a fixed operation label, such as document summarization. Do not build labels from user input.
- Outcome: Store the HTTP status, an application-defined error category, and the retry count.
- Usage: Record input and output token counts when the API returns them. Treat missing usage as unknown, not zero.
For example, a log entry can say that a document summarization request using Claude Sonnet 5.5 failed authentication. That points you toward credential configuration without revealing the document itself.
Block the places where prompt text sneaks in
Removing an explicit prompt field is not enough. A request body can appear in SDK debug output, HTTP middleware, tracing tools, crash reports, and reverse-proxy logs. Check each layer that touches the request, including local development tools.
Disable body capture for AI endpoints. Exclude authorization headers, cookies, and API keys. Avoid logging full URLs when query strings might contain sensitive values. Keep response bodies out too: a model can repeat private information from the prompt.
Error handling deserves special attention. An exception message might contain part of a request or a tool result. Translate failures into controlled categories such as authentication failure, rate limit, or upstream error instead of copying raw exception objects into logs.
Use SwiftRelay without widening your logging scope
SwiftRelay is a pay-as-you-go relay for multiple model providers and works with OpenAI-compatible SDKs. It can provide a consistent integration point, but routing requests through a relay does not make prompt logging safe. Keep the same metadata-only policy in your application and review the data-handling practices of every service involved.
For an OpenAI-compatible SDK, configure the base URL as https://api.fastapirelay.com/v1. Keep the API key in a secret store or environment variable, and never print the client configuration. Sign up for SwiftRelay to try this setup with the small gift credit available to new accounts.
Run a fake-secret test before the green light
Send a test prompt containing a distinctive fake secret and ask the model to repeat it. Exercise a successful request, a rejected request, a timeout, and a retry. Then search application logs, traces, crash reports, and exported monitoring data for that marker.
If it appears anywhere, fix the capture point and repeat the test. Do not use real credentials or customer data for this check. Add the test to your release checklist so a new middleware package does not quietly restore body logging.
Finally, limit who can read logs and set a short retention period that matches your operational needs. Even metadata can reveal activity patterns. If a difficult incident genuinely requires content capture, use a separate, explicitly approved process with restricted access and automatic deletion. Keep routine logs useful, predictable, and free of conversation content.