Your Prompt’s First Line Sets the Price

Prompt structure establishes the first billing boundary

Prompt caching becomes useful when a request repeats a substantial prefix across many calls. That prefix might contain system instructions, tool definitions, policy text, reference material, or a large conversation history. The provider can store that repeated material and reuse it instead of processing every token as fresh input.

The important detail is placement. A stable prefix should appear before changing content, because most caching systems match from the beginning of the eligible prompt. A small edit near the top can shift the matching boundary and turn an otherwise identical request into a new write.

Separate stable instructions from changing conversation content

Begin the walkthrough by dividing the request into two conceptual regions. Put durable material first: role instructions, documentation, schemas, and repeated examples. Put the user question, timestamp, request identifier, or other changing values afterward.

This arrangement gives the provider a predictable candidate for caching. It also makes the prompt easier to inspect. If a cache hit rate falls, you can examine the first changing token rather than searching through the entire request.

• Stable prefix: instructions and reference material shared by many calls.
• Variable suffix: the task, user input, state, and response-specific details.

Send the first request as a deliberate cache write

The first request containing a new stable prefix normally has no reusable entry. The provider processes that prefix and may record it as a cache write. A write is not free merely because it creates future savings; it has its own price, retention rule, and eligibility conditions.

Record the provider's reported input tokens, cache-write tokens, output tokens, and any other usage fields returned for the request. Do not infer these values from character counts or from the length of your local prompt. Tokenization and cache accounting belong to the provider.

Repeat the prefix and watch the cache read appear

Send a second request with the same eligible prefix and a different suffix. If the provider recognizes the stored material, the usage report should identify cache-read tokens. Those tokens represent reused input rather than newly processed input, and they normally carry a different rate from ordinary input tokens.

A cache read can reduce both cost and latency, but neither benefit is guaranteed for every request. Providers may impose minimum prefix lengths, expiration windows, model-specific rules, or exact-match requirements. Treat the usage report as the source of truth for each model.

Trace the bill across writes, reads, and fresh input

Build the bill from separate token categories instead of applying one input rate to the total prompt. A useful mental model is fresh input tokens multiplied by the input rate, cache-write tokens multiplied by the write rate, cache-read tokens multiplied by the read rate, and output tokens multiplied by the output rate.

The exact rates vary by provider and model. A cache read may be substantially cheaper than fresh input, while a cache write may cost more than ordinary input because it creates reusable state. A short-lived workload can therefore lose money if it pays for many writes but produces too few reads.

Measure the break-even point before changing architecture

Count expected repetitions for each stable prefix. One write followed by many reads usually creates a stronger case for caching than one write followed by a single completion. Include expiry in the calculation, because a prefix that disappears before its next use must be written again.

Compare the measured totals with a no-cache baseline. Include latency, engineering effort, and prompt maintenance, but keep token charges separate. This prevents an attractive hit rate from hiding expensive writes or a low hit rate from being mistaken for a model pricing problem.

Verify relay accounting against the provider usage report

BestRelay is a quality-first API relay that forwards each request to the official model named in it, without substitution, across Claude, GPT, Gemini, DeepSeek, Kimi, GLM, Qwen, Grok, and other supported families. Its pay-as-you-go billing uses the token counts reported by the provider, with cache reads and cache writes priced separately.

That arrangement makes the usage fields especially important. When testing a cached workflow through BestRelay, compare the model response metadata with the relay billing record. A mismatch may indicate a provider-specific accounting convention, an expired cache entry, or a changed prompt prefix rather than a mysterious surcharge.

For a concrete test, Claude Sonnet 5.5 can serve as one model under examination. Keep the model name fixed, send one stable prefix repeatedly, vary only the final task, and record each reported category. Then repeat the experiment with a small edit near the beginning to see the cache boundary move.

Turn reliable cache hits into a controlled operating habit

Monitor cache-read share, write frequency, prefix size, expiry behavior, and total cost per completed task. Alert on sudden drops in reads, because prompt templates often change silently when teams add instructions or reorder fields.

Create a BestRelay account when you are ready to run the same measurement against your production-shaped traffic. Members can also earn a referral commission when people they invite top up, although the central financial decision remains the measured relationship between writes, reads, and useful completions.