A cached prompt is not free input. Creating a reusable cache can carry a write charge, and reusing it can carry a read charge. The useful question is whether those charges beat processing the same input repeatedly. Walk through this checklist before treating caching as a guaranteed discount.
1. Separate ordinary input, cache writes, and cache reads
Start with three billing categories. Ordinary input is prompt content processed without a cache discount. Cache writes create reusable cached content. Cache reads reuse eligible content already stored. Generated output remains a separate charge; caching your instructions does not make the model's answer free.
These categories are not universally priced or reported the same way. Some models expose explicit cache creation charges; others use automatic caching with different rules. Check the selected model's pricing and caching documentation rather than carrying assumptions across providers.
2. Put reusable content at the front
Imagine an assistant that receives the same reference manual on every request, followed by a different question. The manual is your potential cached prefix. The changing question is ordinary input unless it independently qualifies for caching.
Keep stable instructions, reference material, and tool definitions consistent and ahead of changing content where the model's caching rules allow. Adding a timestamp or request identifier near the beginning can prevent later content from matching. Similar meaning is not enough: cache matching generally depends on a matching prefix, subject to provider-specific rules.
3. Price the first cache write explicitly
Use a deliberately hypothetical rate card: ordinary input costs $1 per million tokens, cache writes cost $1.25 per million, and cache reads cost $0.10 per million. These are arithmetic examples, not SwiftRelay prices or a quote for any model.
For a 100,000-token reusable prefix and a 5,000-token changing question, the first eligible request costs:
- Cache write: 100,000 divided by one million, multiplied by $1.25, equals $0.125.
- Ordinary input: 5,000 divided by one million, multiplied by $1, equals $0.005.
- Total input charge: $0.130. Output charges are additional.
Without caching, those 105,000 input tokens would cost $0.105. In this example, creating the cache initially costs more than processing the input normally.
4. Price the next matching cache read
Now send another question with the same eligible prefix while the cache remains available. The 100,000-token prefix receives the hypothetical read rate, while the new 5,000-token question receives the ordinary input rate.
- Cache read: 100,000 divided by one million, multiplied by $0.10, equals $0.010.
- Ordinary input: $0.005.
- Total input charge: $0.015. Output charges still apply.
Across these two requests, cached input costs $0.145 instead of $0.210. That saves $0.065 before output charges. The first write only pays off because a later request actually reuses it.
5. Budget for misses and expiration
A changed prefix, an expired retention window, an unsupported request shape, or insufficient eligible content can prevent a read. Depending on the model, that request may incur ordinary input charges, a new write charge, or a mixture across different token segments.
Check minimum cacheable lengths, retention options, and any pricing differences between retention settings. Do not assume a cache survives a model change or provider change. For workloads with little repeated content, ordinary input can be cheaper than repeatedly creating caches that nobody reads.
6. Verify usage without counting tokens twice
Record ordinary input, cache creation, cache reads, and output using the response's documented usage fields. Some APIs report cached tokens inside an overall input total. Adding both totals directly would double-count those tokens.
With SwiftRelay, a pay-as-you-go relay, check the selected model's applicable rates and how its usage maps to your bill. Sign up for SwiftRelay to test a representative repeated-prefix workload, then reconcile the reported charges with your calculation.
7. Keep caching only where reuse pays
Measure total spend across the whole sequence, not just the cheapest cache hit. Include the initial write, subsequent reads, misses, changing input, and generated output. The winning pattern is stable content reused enough times to repay its creation cost. If that pattern disappears, revisit the cache design before increasing traffic.