Smaller System Prompts Save Money Only After Testing

A system prompt is a recurring expense disguised as documentation. Teams add rules after incidents, keep examples after migrations, and repeat instructions across application layers. Eventually, every request carries material that nobody has recently tested. Trimming that material can reduce input costs, but the useful target is lower cost per successful task, not the shortest possible prompt.

Step one establishes the real billing baseline

Collect a representative sample of production requests before editing anything. Record provider-reported input and output tokens, cache usage, retries, and task outcomes. Separate system instructions from user content and tool definitions where your instrumentation permits it. A large system prompt may be an obvious target without being the largest billable component.

This walkthrough is published by BestRelay, a quality-first API relay that forwards each request to the official model named in it, without substitution. Its pay-as-you-go billing uses provider-reported token counts, with cache reads and writes priced separately. That distinction matters: removing a thousand cached tokens does not necessarily save as much as removing a thousand uncached tokens.

Step two gives every instruction a purpose

Read the prompt sentence by sentence and classify each instruction as a behavioral requirement, output contract, safety boundary, task fact, example, or repetition. Attach each requirement to an observable outcome. If a sentence cannot be connected to a testable behavior, mark it for review rather than automatically preserving it.

Look first for duplicated role descriptions, obsolete product details, repeated formatting rules, and motivational language. Preserve authorization boundaries, tool restrictions, and required output constraints. Move changing task facts into the relevant request when that improves relevance, but recognize that moving text does not eliminate its token cost.

Step three compresses wording without compressing meaning

Replace overlapping instructions with one explicit rule. Several paragraphs requesting brevity can often become a single sentence defining the expected answer length and necessary exceptions. Prefer direct language over ceremonial introductions. Remove explanations written for prompt maintainers unless the model needs them to follow the instruction.

Do not replace clear prose with cryptic abbreviations merely to reduce characters. Tokenization varies, and shorter text is not always fewer tokens. Measure with the target model's tokenizer where available, then confirm actual usage through provider reports. Keep useful examples until evaluation shows that removing them preserves the behavior they teach.

Step four tests behavior before counting savings

Build an evaluation set containing ordinary requests, edge cases, malformed inputs, and attempts to override application rules. Include tasks that previously required retries or manual correction. Compare the original and trimmed prompts with the same model, settings, tools, and inputs.

Define acceptance criteria before examining results. Check output validity, factual accuracy, tool selection, refusal boundaries, and completion rate as appropriate. Repeat tests when generation variability could conceal regressions. A cheaper request that creates another request, a support ticket, or an incorrect action is not necessarily a cheaper workflow.

Step five accounts for cache behavior explicitly

Calculate cost using separate uncached-input, cache-read, cache-write, and output quantities under the applicable rates. As a hypothetical example, removing 600 uncached input tokens across 100,000 requests eliminates 60 million input tokens. The monetary saving still depends on pricing, cache eligibility, and whether output length or retry frequency changes.

Keep reusable instructions stable when the provider's caching mechanism benefits from a shared prefix. Editing that prefix can reduce cache reuse temporarily. Compare both the transition period and steady-state operation rather than treating a cold-cache test as the permanent result.

Step six rolls out the smallest proven change

Release the trimmed prompt to a limited traffic share, monitor quality alongside spending, and retain a rollback version. BestRelay works with Claude Code, Codex, Gemini CLI, and OpenAI SDK clients, making existing workflows a practical place to gather measurements. Create a BestRelay account to measure your own baseline.

Keep incentives separate from technical conclusions: members can earn referral commission when invited users top up, but that income is not evidence of prompt savings. The durable saving comes from a tested reduction in unnecessary instructions, verified against actual usage and successful outcomes.