A rate limit error should change your request schedule, not crash your application. HTTP 429 means the server is rejecting a request because a limit has been reached. The right response depends on which limit you hit, whether the operation is safe to repeat, and how long your user can wait. Follow this numbered checklist to build a recovery path that stays useful under pressure.
1. Identify the limit behind the 429
Capture the status code, response body, request identifier, and relevant response headers. A 429 can reflect a request limit, a token limit, a concurrency restriction, or an account quota. Read the provider's documented error fields instead of assuming every 429 means the same thing.
Separate temporary throttling from exhausted quota. A temporary limit may clear after a waiting period. A depleted balance or account restriction usually needs an account change rather than another identical request. Keep credentials, prompts, and sensitive response content out of diagnostic logs.
2. Honor Retry-After before calculating backoff
When present, the Retry-After header can contain either a number of seconds or an HTTP date. Parse both formats. For a date, calculate the remaining delay using your current clock, accounting for clock skew where possible.
Treat a valid Retry-After value as a minimum wait. If it exceeds the request's remaining deadline, stop the interactive attempt and offer a deferred retry instead. Never shorten the server's requested wait just to fit a local backoff cap. If the header is absent or invalid, use your own bounded backoff policy.
3. Add exponential backoff with jitter
For temporary throttling without a usable server delay, increase the waiting window after each failed attempt. One practical policy doubles the window each time, up to a configured ceiling, then chooses a random delay between zero and that window. This is commonly called full jitter.
Randomness prevents many clients from retrying together. When Retry-After is available, you can add a small randomized delay after its minimum wait. Avoid immediate retry loops: they create more traffic while the service is already asking you to slow down.
4. Set one retry budget across your stack
Define both a maximum attempt count and a total elapsed-time budget. Count the initial request as an attempt, and include waiting time in the elapsed budget. Stop when either limit is reached. Respect cancellation when the user leaves or cancels the task.
Check whether your SDK already retries 429 responses. An application loop wrapped around SDK retries can multiply attempts unexpectedly. Choose one layer to own retries, or explicitly coordinate their budgets. For streamed responses, do not silently restart after delivering output: you could duplicate text or repeat downstream actions.
5. Reduce pressure before retrying more work
Backoff handles an individual failure; traffic control prevents the next wave. Put requests behind a bounded queue and cap concurrent calls. If several workers share an account, coordinate throttling across them rather than letting each worker assume it owns the full allowance.
- Reduce concurrency when throttling persists, then increase it cautiously after recovery.
- Limit requested output tokens and trim unnecessary context when token limits are involved.
- Reject or defer excess queued work instead of allowing an unbounded backlog.
- Retry operations only when repetition is safe; use documented idempotency support where available.
6. Wire the policy into your SwiftRelay client
SwiftRelay is a practical option for accessing models through an OpenAI-compatible SDK with per-token billing. Set the SDK base URL to https://api.fastapirelay.com/v1, then apply your retry and concurrency policy around the client's documented error handling. Claude Code and Gemini CLI use the bare host, https://api.fastapirelay.com.
Sign up for SwiftRelay to try this workflow with the small gift credit available to new accounts. A relay does not make rate limits disappear: inspect actual responses and avoid assuming that switching models will resolve an account-wide restriction.
7. Test recovery before real traffic arrives
Simulate missing headers, date-formatted Retry-After values, repeated 429 responses, cancellation, and exhausted budgets. Verify that retries stop, queues remain bounded, and users receive a clear message when work is deferred. Track throttled requests, retry attempts, queue depth, and final outcomes. Graceful recovery means controlled behavior, not endless persistence.