A long conversation does not cost only the tokens in its newest message. When your application resends earlier messages, those messages become input again. This first-run checklist turns that growing transcript into a measurable budget, with a worked example you can adapt to your own application.
1. Inventory everything sent with each chat request
Start with the complete request, not just the conversation visible in your interface. Input can include system instructions, developer instructions, tool definitions, retrieved documents, previous messages, tool results, and the latest user message. Message formatting and non-text content can add tokens too.
Count what your client actually sends. Some applications resend the full transcript; others trim it or use provider-managed conversation state. A persistent conversation identifier does not, by itself, tell you which tokens are billed. Check the provider’s accounting behavior and inspect returned usage.
2. Measure tokens with the selected model’s tokenizer
Character counts are useful for an early estimate, but they are not a dependable billing calculation. English prose, source code, structured data, and unusual identifiers tokenize differently. Use a compatible tokenizer or token-counting endpoint where available, then validate your estimate against actual response usage.
Keep separate counters for input, output, and any reported cache categories. If your model reports reasoning tokens or other usage details, check how those categories relate to billed output rather than adding them twice. A model change can also change token counts, so repeat this check when switching models.
3. Calculate one growing-history request
Assume your fixed instructions use 1,000 tokens. Each user message adds 200 tokens, and each assistant reply adds 300. These are illustrative measurements, not promises about typical conversations.
Before generating the tenth reply, the request contains the fixed instructions, ten user messages, and nine previous assistant replies:
Tenth-request input = 1,000 + (10 × 200) + (9 × 300) = 5,700 tokens.
The tenth reply then adds another 300 output tokens. Its tokens are output now and, if retained, input on subsequent requests. That second role is what makes long histories increasingly expensive.
4. Sum the entire session, not just its last request
In this example, request number n contains 1,000 + 200n + 300(n − 1) input tokens, which simplifies to 700 + 500n. Across ten requests, the input counts run from 1,200 to 5,700 in increments of 500.
Total input = 10 × (1,200 + 5,700) ÷ 2 = 34,500 tokens. Total output is 10 × 300 = 3,000 tokens.
The final transcript contains only 6,000 tokens, including the fixed instructions. Yet processing the session consumed 34,500 input tokens because earlier content was submitted repeatedly. With similarly sized turns and full-history resending, cumulative input grows approximately quadratically.
5. Convert token totals into a billing estimate
Read the current rates for your selected model and route. For rates quoted per million tokens, use:
Estimated cost = (input tokens × input rate + output tokens × output rate) ÷ 1,000,000.
If caching applies, split input into the provider’s billable categories and apply each category’s rate. Do not assume repeated history automatically receives a cache discount; eligibility, cache writes, and cache reads can have different rules.
SwiftRelay offers pay-as-you-go, per-token access, making it practical to test this calculation against a small real workload. Sign up for SwiftRelay, use the small gift credit available to new accounts, and compare your recorded usage with your estimate.
6. Reserve context space before history fills it
A context window is a capacity limit, not a spending target. For an illustrative 32,000-token window, reserving 4,000 tokens for output and 2,000 for counting uncertainty leaves a 26,000-token input budget. Confirm the selected model’s actual limits and any separate output cap.
When approaching that budget, remove irrelevant tool results, retrieve only useful excerpts, or replace older turns with a compact summary. Summarization itself costs tokens and can lose details, so preserve decisions, constraints, and unresolved questions.
7. Log usage and enforce a history budget
Record estimated input, reported input, reported output, cache usage, and cost after every request. Recalculate the next request after trimming history. Enforce both a context threshold and a spending threshold: a conversation can remain within the context window while repeated history quietly exhausts its budget.