A failed request can still consume tokens
You send a prompt, your app shows an error, and you press retry. It feels like restarting an unfinished task. But an error on your screen does not necessarily mean the model did no work.
The model may already have processed your input or generated part of an answer before the connection broke. Depending on the provider’s billing rules and what happened upstream, that attempt may have incurred charges. The next attempt can create another billable round of processing.
A retry is another attempt, not automatically a free replacement. Understanding that distinction is the first step toward keeping a small integration from quietly becoming an expensive one.
Find every place that can repeat a request
Retries often hide below the button you clicked. An SDK may retry certain failures automatically. Your application may have its own retry loop. A background job runner may repeat a task when it does not receive confirmation.
These layers can multiply each other. If your application makes three attempts and each application attempt allows three SDK attempts, one user action could produce as many as nine upstream attempts. The exact behavior depends on your configuration, but the lesson is simple: count total attempts, not just visible clicks.
- Check the SDK: Read its retry defaults and identify which errors trigger another attempt.
- Check your code: Look for retry loops, fallback calls, and catch blocks that resend prompts.
- Check the job system: Find out whether failed or interrupted tasks are automatically restarted.
- Check the interface: Prevent repeated clicks from launching duplicate requests while one is still active.
Watch long prompts and interrupted streams
Suppose your request contains a large document and asks Claude Sonnet 5.5 to extract a short checklist. If the request is repeated from scratch, the document may be processed again. A tiny final answer does not make the repeated input free.
Long conversation histories create the same trap. Your app might resend the entire conversation on every attempt, including material that is no longer relevant. Caching can change input pricing where supported, but do not assume every retry receives a cache discount.
Streaming adds another wrinkle. You might receive part of an answer before the connection closes. Starting over can generate overlapping output and additional charges. A local timeout is not proof that upstream generation stopped. Treat an interrupted stream as an uncertain outcome, not an automatically unbilled failure.
Give retries a small, explicit budget
Retries are useful for temporary failures. The goal is to make them deliberate rather than endless. Choose one layer to own retry decisions wherever practical, and place a hard limit on total attempts for each user action.
- Retry selectively: Follow documented guidance for temporary failures. Fix invalid requests and authentication errors instead of repeatedly resending them.
- Back off between attempts: Use exponential backoff with jitter, and respect retry instructions when supplied.
- Limit repeated payloads: Send only the context the task needs, and set an appropriate output limit.
- Bound the whole task: Apply a spending ceiling across retries and model fallbacks, not separately to each call.
- Handle uncertain outcomes: Check any available request status before resubmitting. Do not assume an arbitrary request identifier prevents duplicate processing.
Log attempts before your next paid run
Record a task identifier, attempt number, model, error category, available usage totals, and any provider request identifier. Avoid logging sensitive prompt contents unnecessarily. Compare your records with billing data; a missing usage response does not establish that an attempt was free.
For a pay-as-you-go setup, SwiftRelay provides per-token access to multiple model providers through compatible development tools and SDKs. Sign up for SwiftRelay and use the small gift credit to test your retry behavior before scaling usage. Keep your own attempt tracking in place regardless of which provider you call.
Your final pit-stop check is straightforward: one deliberate retry policy, a bounded task budget, and enough records to explain every repeated call. That turns retries from an invisible expense into a controlled recovery mechanism.