Your first lap starts with response headers
Rate limits are rules about how much API work your application can request within a particular window. Response headers are the small pieces of metadata returned alongside the answer. When an API exposes rate limit information there, those headers can tell you what allowance remains and when more capacity may become available.
The useful habit is simple: inspect rate limit headers on every response, not just after an error. A successful request can still leave your application close to its allowance. Reading the headers early gives your application a chance to pause new work before requests start failing.
Find the limit, remaining allowance, and reset
Header names vary across providers and endpoints. You might encounter names such as x-ratelimit-limit-requests, x-ratelimit-remaining-requests, and x-ratelimit-reset-requests. Treat these as examples, not a promise that every API returns them. HTTP header names are case-insensitive.
- Limit: The stated allowance for a particular resource and window, such as requests or tokens.
- Remaining: The allowance reported as available after the server accounts for the request, according to that API's rules.
- Reset: Information about when capacity becomes available again. Its format and meaning depend on the API.
- Retry-After: A wait instruction that may accompany a rate limit response. Standard forms are a number of seconds or an HTTP date.
Never assume a reset value is a timestamp. It could be a duration, an epoch value, or another documented format. Check the documentation for the endpoint you actually call, including the units.
Track request limits and token limits separately
An application can have room for another request but insufficient token allowance for the work it wants to submit. A long prompt and a generous output budget can put different pressure on the limits than a short exchange.
Keep separate records for each exposed limit. Do not combine request counts and token counts into one remaining number. Also check the scope: a limit might apply to an account, project, model, or endpoint. A local counter for one worker does not describe everything other workers are doing.
Think of remaining allowance as a useful signal, not a reservation. Other requests can consume shared capacity after your application reads a response.
Read successful responses before queuing more work
Put header inspection in a shared response handler so individual features do not have to remember it. After each response, record the status, relevant limit values, and observation time. Use an allowlist of headers to avoid logging unrelated or sensitive metadata.
If your SDK normally returns only the generated content, look for its raw-response interface. For streaming requests, inspect the headers when the response opens rather than waiting for the stream to finish. Do not assume the stream will provide updated quota information throughout generation.
Missing headers are not unlimited permission. Preserve the distinction between an unknown allowance and a reported zero, and keep a conservative fallback policy.
Handle a rate limit response without a retry loop
A 429 status commonly indicates rate limiting, but read the error details to understand the cause. If Retry-After is present and valid, respect it. Otherwise, use documented reset information where applicable, or capped exponential backoff with a small random delay.
Set a maximum number of retries. Coordinate waiting across workers when they share a limit, and avoid automatically replaying requests with side effects unless replay is safe. Repeated retries cannot fix an exhausted allowance that requires account action.
Check the headers on your SwiftRelay route
SwiftRelay is a pay-as-you-go relay for multiple model providers that works with OpenAI-compatible SDKs. Its API host is https://api.fastapirelay.com; OpenAI SDK configurations add /v1. Sign up for SwiftRelay to try your integration with the small gift credit available to new accounts.
Inspect the headers actually returned by your chosen route rather than assuming every provider's native headers pass through unchanged. Your first-run checklist is straightforward: capture available headers, verify their meaning, track separate limits, and make every response inform the next request.