What a cache hit changes on the monthly bill
Prompt caching is not a coupon you clip once and forget. It is a change to the shape of every request you send, and your monthly bill follows that shape. When the beginning of a request matches something the provider has already processed, the model can reuse that work and bill the reused portion at a cached rate instead of the full input rate. Nothing about the answer changes. What changes is which tokens you pay full price for, every single turn.
The effect is easiest to see in an agent loop. A coding agent sends the same system prompt, the same tool definitions, and the same conversation history on every step, then adds a little new text at the end. Without caching, you pay full input price for that entire stack dozens of times per task. With caching, the repeated stack becomes a cache hit after the first request, and only the genuinely new tokens are billed at the standard input rate. On a heavy agent workload, that difference is often the largest single lever on the monthly bill.
Two details deserve respect before you plan around the savings. First, writing to the cache is not always free: some models charge a cache-write rate that sits above the normal input rate, so the first request in a session is the expensive one. Second, cache entries expire. If your traffic has long gaps, or if the prefix shifts slightly between calls, you keep paying the write cost and never collect the read savings. The goal of this checklist is to make sure the hits outnumber the misses.
The pre-flight checklist before your first caching run
Run through these items before you change any code, so that your first measurements actually mean something.
- List your static prefix. Write down everything that repeats across calls: system prompt, developer instructions, tool or function schemas, retrieved documents, style rules, file trees. This is the candidate cache.
- Count its tokens. Estimate the token size of that repeated block. A prefix of a few dozen tokens is not worth caching; a prefix of many thousands of tokens repeated across dozens of turns is exactly where caching earns its keep.
- Confirm the model supports it. Caching behaviour is per model family. Check the model you actually plan to run, not the one in an old example.
- Confirm your transport passes the request through unchanged. A relay or wrapper that reorders fields, injects a timestamp, or re-serializes your payload can silently break prefix matching.
- Find the cache fields in the response. Decide up front where you will read cached input token counts from, so you can log them from day one.
- Pick a measurement window. Choose a task set and a request range you will compare before and after, so the bill change is attributable to caching and not to a busier week.
That last item matters more than it sounds. Prompt caching savings show up as a ratio: cached tokens versus uncached tokens across a burst of requests. If you cannot see that ratio in your logs, you are guessing, and guesses make bad budgets.
Keep the cached prefix byte-for-byte stable
Cache matching is literal. The provider compares the start of the request against what it stored, and any difference ends the match at that point. A single changed character early in the prompt forfeits the savings for everything after it. This is the mistake that quietly turns a caching setup into a cache-write generator.
Put all volatile content at the end of the request, never at the front. Timestamps, usernames, random request IDs, per-run goal lines, and environment dumps belong after the static block. Keep tool schemas in a fixed order and with stable field ordering. If you build prompts by iterating over a dictionary or map, sort the keys so the output is deterministic. If you inject retrieved documents, cache the long-lived instructions and let only the retrieval results vary.
Also watch for version churn. Editing the system prompt in a deploy is fine, but remember it invalidates the stored prefix for every caller, and the next wave of requests pays cache-write pricing again. Batch prompt edits together, and treat a prompt version bump as a small cost event rather than a daily habit. In long agent sessions, keep appending to the conversation instead of rebuilding the whole message array from scratch, because a rebuilt history that differs in formatting counts as a different prefix.
Choosing a model that rewards a stable prefix
Caching pays best on workloads with a fat static prefix and many repeated turns. Long agent sessions with heavy tool use are the classic case, and models tuned for agentic work make the pattern easy to express. Claude Sonnet 5.5 is a natural fit here, since agent loops built around a large instruction block and a fixed tool list repeat almost everything except the latest step.
If your workload is closer to bulk analysis over a fixed policy or schema, look at the fast tiers where input volume dominates the spend. DeepSeek V4 Flash is worth a look for pipelines that replay the same rules across many documents, since caching lets you pay once for the rules and reuse them per document. Whichever model you pick, write the cache fields from its responses to your own ledger first, and let real ratios guide the switch.
Read the bill like a pit board
Once the first run is done, look at three numbers together: total input tokens, cached input tokens, and cache-write tokens. The share of input that lands in the cached column is your hit rate, and it is the number to optimise. A low hit rate with a high write count means your prefix is unstable or your requests are too far apart. A high hit rate with a modest write count means the shape is right and you can extend caching to more of your traffic.
Segment the numbers by workload. Agent sessions, batch jobs, and one-shot completions have completely different reuse patterns, and an average across all three hides both the wins and the waste. The same goes for errors and retries: a request that fails and is retried with a re-serialized body may miss the cache entirely, so fixing flaky calls is a caching improvement too.
SwiftRelay makes this loop straightforward to run. It is a pay-as-you-go relay for Claude, GPT, Gemini, DeepSeek, Kimi, GLM, Qwen, Grok and other models, billing is per token with no subscription, and usage is visible per request so cached counts are easy to chart. It works with Claude Code, Codex, Gemini CLI and any OpenAI-compatible SDK: Claude Code and Gemini CLI take the bare host https://api.fastapirelay.com, while OpenAI SDKs add /v1. New accounts get a small gift credit, inviting friends earns 14% of their top-ups, and you can sign up and run the first caching experiment on that credit before committing real budget.
Common cache-busters to clear before the green light
Do a final sweep for the patterns that quietly reset your savings:
- Leading timestamps or request IDs. Move them to the end of the prompt or into metadata.
- Unstable serialization. Random key order, whitespace drift, and changing indentation all create new prefixes.
- Per-call scaffolding at the front. Inject task context after the static instructions, not before them.
- Frequent prompt edits. Batch changes so each invalidation is deliberate.
- Gaps long enough to expire entries. Space interactive sessions and batch windows so work arrives in bursts rather than trickles.
- Retries with re-built bodies. Reuse the original serialized request on retry so the cached prefix still matches.
Clear these, log the cached versus uncached split, and let the ratio set your next move. Prompt caching rewards discipline more than cleverness: a stable prefix, repeated turns, and a ledger you actually read will do more for the monthly bill than any single prompt rewrite. Start small, measure one workload, and scale the pattern to the rest of your traffic once the hits are real.