A model request times out after leaving your server. Did generation fail, finish, or remain in progress? Sending it again might produce another answer and another billable generation. This sprint builds an application-level retry workflow that prevents avoidable duplicates without pretending every timeout is safe to retry.
1. Give each generation intent a stable key
Create an operation identifier before the first model call, and retain it across browser retries, worker restarts, and queue redeliveries. Scope it to the authenticated account and the intended action, such as generating a particular document revision.
Do not create a fresh key for each network attempt. A deliberate regeneration gets a new key; recovering the original operation keeps the existing one. Treat keys as identifiers, not secrets, and enforce authorization whenever someone retrieves an operation.
2. Bind the key to one immutable payload
Store a fingerprint of the effective request beside the key. Include the model, messages, generation settings, tool definitions, and any other inputs that affect the result. Use deterministic serialization so object field ordering does not change the fingerprint.
If the same key arrives with a different fingerprint, reject it as a conflict. Silently accepting changed input turns deduplication into a source of incorrect answers. Also freeze routing choices for the operation: switching models during recovery is a new execution decision, not transparent replay.
3. Claim the operation in durable storage
Use a database uniqueness constraint on the account and operation key. Atomically insert a record before dispatching the request. In-memory maps cannot coordinate multiple workers or survive a restart.
- Pending: one worker owns the dispatch attempt.
- Completed: the saved response can be replayed.
- Failed: a definitive failure is recorded with its retry eligibility.
- Unknown: dispatch may have reached the provider, but completion is unconfirmed.
Return the saved result for completed operations. For pending duplicates, return an operation status rather than starting another generation. Use ownership tokens when updating records so an expired worker cannot overwrite a newer owner's state. An expired lease alone does not prove redispatch is safe.
4. Configure one deliberate retry layer
SwiftRelay is a practical relay for an OpenAI-compatible SDK: configure its base URL as https://api.fastapirelay.com/v1 and keep the API key on your server. Sign up for SwiftRelay to try this workflow with the small gift credit available to new accounts.
Start with a fixed model, such as Claude Haiku 4.5, while testing recovery behavior. Inspect the SDK's retry settings and disable automatic retries when your application owns the policy. Otherwise, one application attempt can hide several transport attempts.
Do not assume an idempotency header is supported. Verify documented behavior for your exact endpoint and route, including key scope, retention, concurrent requests, and response replay. Your database ledger prevents local duplicates; it does not establish provider-side deduplication.
5. Separate retryable failures from uncertain outcomes
Authentication errors and invalid requests need correction, not backoff. Rate-limit responses and temporary service failures may justify retries when endpoint semantics make replay safe. Honor a valid Retry-After value, otherwise use capped exponential backoff with jitter, an attempt limit, and an overall deadline.
A read timeout, broken stream, or lost connection after sending the request is different: execution may already have happened. Without documented upstream idempotency or a reliable status lookup, mark the operation unknown instead of automatically resending it. Let the caller explicitly authorize another generation, with the possibility of additional token charges made clear.
6. Save the result before acknowledging completion
Persist the response and completed state atomically before reporting success to your client. Keep enough response metadata to reproduce the application result, plus available request identifiers for support and reconciliation.
A crash after provider completion but before persistence remains an uncertainty window. Local transactions cannot close it across a remote API. Streaming requires particular care: displaying partial text does not mean you have a complete response to replay.
7. Test duplicate delivery before the green light
Send concurrent requests with the same key, restart a worker after dispatch, simulate a lost response, and reuse a key with changed input. Verify that completed results replay, conflicts fail clearly, and uncertain operations do not silently regenerate. Apply the same discipline separately to tool side effects: deduplicating generation does not make a payment or email idempotent.