Your next message carries earlier messages
A short question can create a surprisingly large API request. When an app includes earlier messages to keep a conversation coherent, those messages become input again. You are not just sending the latest sentence; you may be sending much of the conversation behind it.
That is the central rule of long-chat token math: count what each request sends, not just what you newly type. Once that clicks, usage reports become much easier to understand.
Tokens are text pieces, not exact word counts
A token is a small unit of text processed by a model. It might be a word, part of a word, punctuation, or another text fragment. As a rough English estimate, one token often represents about four characters, but code, unusual names, and formatting can change that relationship.
Use that estimate for planning, not billing reconciliation. For actual counts, use the model provider's tokenizer or the API response's usage fields. Different models can tokenize the same text differently.
- Input tokens: the instructions, messages, and other content supplied to the model.
- Output tokens: the content generated by the model, with additional accounting details possible for some models.
- Context window: the capacity available for a request, generally shared between input and generated output, subject to model-specific limits.
A four-turn chat can process 3,800 tokens
Imagine a simple conversation with 200 tokens of opening instructions. Each new user message adds 100 tokens, and each assistant reply adds 200. Assume the app sends the full history on every request, without caching or extra metadata.
- Request one: 200 instruction tokens + 100 new user tokens = 300 input tokens.
- Request two: 200 instructions + 300 previous conversation tokens + 100 new user tokens = 600 input tokens.
- Request three: 200 instructions + 600 previous conversation tokens + 100 new user tokens = 900 input tokens.
- Request four: 200 instructions + 900 previous conversation tokens + 100 new user tokens = 1,200 input tokens.
Total input is 3,000 tokens. Four replies contribute another 800 output tokens. Across the four requests, the model processes 3,800 tokens, even though the stored conversation after the final reply contains only 1,400 tokens, including the opening instructions.
Nothing mysterious happened: earlier messages were counted again whenever they were resent. With similarly sized turns and full-history requests, cumulative input grows roughly with the square of the number of turns.
Turn token counts into a cost estimate
Input and output commonly have different prices. If rates are quoted per million tokens, estimate cost as: input tokens divided by 1,000,000, multiplied by the input rate, plus output tokens divided by 1,000,000, multiplied by the output rate.
For illustration only, suppose input costs $1 per million tokens and output costs $4 per million. Our example costs $0.003 for input plus $0.0032 for output, totaling $0.0062. These are hypothetical rates, not a SwiftRelay price quote.
Cached input, reasoning tokens, and tools can introduce additional billing categories. Read the selected model's pricing and usage details rather than assuming every token has one price.
Trim repeated context before your next request
- Keep a compact working summary. Preserve decisions, constraints, and unresolved questions instead of replaying every exchange.
- Retrieve relevant excerpts. Send the document sections needed for the current question, not an entire archive.
- Limit oversized tool results. Logs and file contents can quietly dominate input usage.
- Reserve output space. A request near the context limit may leave insufficient room for the answer.
Summaries can lose details, so keep original material available when accuracy matters. Also check whether your client automatically trims history or uses prompt caching; visible chat length alone does not reveal billable usage.
Check real usage after a short test conversation
SwiftRelay is a practical option for pay-as-you-go access across models, with per-token billing and support for OpenAI-compatible SDKs. Sign up for SwiftRelay to try a short conversation with the small gift credit for new accounts, then compare each request's reported input and output usage. That small test makes your own application's history behavior visible before a much longer chat grows expensive.