Your system prompt rides along with repeated API requests, so unnecessary instructions can become recurring input costs. The goal is not the shortest possible prompt. It is the smallest prompt that reliably preserves your application’s behavior. Use this seven-step pit stop to remove waste without weakening the rules your product depends on.
1. Record the full request before trimming
Capture a representative request, including the system prompt, conversation history, tool definitions, and injected context. Count tokens with the target model’s tokenizer or a compatible counting tool, then check the API’s reported usage. Character counts are useful for spotting bulk, but they are not billing measurements.
Save the original prompt as your baseline. Record the model, generation settings, input tokens, output tokens, and cost where available. Keep sensitive customer information out of your test fixtures.
2. Mark the rules that must survive
Separate essential behavior from explanatory padding. Before deleting anything, identify instructions that protect correctness, security, and your application’s contract.
- Keep: permission boundaries, privacy requirements, tool restrictions, and required output fields.
- Consolidate: repeated tone guidance, overlapping formatting instructions, and duplicated definitions.
- Question: background stories, motivational language, and examples that no longer match the task.
Do not move essential system-level rules into user messages just to make the system prompt look smaller. That changes their authority rather than simply reducing their cost.
3. Replace repeated prose with direct instructions
Look for multiple sentences that express the same requirement. A prompt might say, “Be concise,” “Avoid unnecessary explanation,” and “Keep every response brief unless additional detail is requested.” Usually, one clear instruction can carry that intent.
Before: You are a helpful assistant who writes customer support replies. Always maintain a professional tone. Your replies should sound professional and should not include unnecessary introductory language.
After: Write professional customer support replies without unnecessary introductions.
Keep qualifications that affect behavior. “Use only verified account data” is not redundant with “be accurate.” Avoid cryptic abbreviations: a smaller prompt that needs interpretation can produce more mistakes and retries.
4. Remove examples that no longer earn their tokens
Examples are valuable when they clarify an unusual format or a difficult decision. They are expensive clutter when they merely repeat an obvious instruction. Test removing them individually rather than deleting the entire example section at once.
For structured responses, retain the required field names, types, and constraints. Where supported, use structured output features to enforce the response shape, and test their behavior separately. Remember that schemas and tool definitions can also consume input tokens; moving text there does not automatically eliminate its cost.
5. Load task-specific guidance only when needed
A universal prompt often accumulates instructions for billing, onboarding, refunds, and troubleshooting. Most requests need only one of those workflows. Keep a compact shared policy and attach the relevant workflow through your application’s trusted instruction mechanism.
Choose that workflow using validated application state when possible. Treat retrieved documents and user-supplied text as data, not trusted instructions. Conditional loading should reduce irrelevant context without allowing untrusted content to rewrite your rules.
6. Test each edit against a fixed case set
Run the baseline and revised prompts against the same normal, ambiguous, and adversarial requests. Include missing information, malformed inputs, permission violations, and attempts to override instructions. Change one section at a time so failures are easy to trace.
Keep the model fixed, such as Claude Sonnet 5.5, while evaluating prompt edits. Check required fields, factual grounding, tool choices, and refusal behavior. Repeat important cases because outputs can vary. Reject a trim if cheaper inputs lead to costly corrections or extra calls.
7. Compare billed cost and ship a reversible change
Measure complete task cost, including outputs, retries, and tool-related follow-up requests. Account for provider-specific cache pricing: a shorter prompt does not guarantee lower billed cost if the change disrupts an existing cache. Keep stable shared content consistent where caching applies.
SwiftRelay offers pay-as-you-go, per-token access across supported models, making it a practical place to run controlled prompt experiments. Sign up for SwiftRelay to use the small gift credit available to new accounts, then compare usage under the applicable model pricing.
Version the winning prompt, deploy it gradually, and retain a rollback path. Your finish line is a lower-cost request that still passes the behavior checks—not simply a smaller block of text.