Cached Input Costs Less Because Earlier Work Still Counts

Cached input preserves work the model already performed

An AI request can contain much more than your latest question. It may include instructions, reference documents, tool definitions, and earlier conversation messages. Sending that material repeatedly gives the model familiar text to process again. Prompt caching lets a provider retain reusable results from processing some of that input, so a later request can avoid repeating part of the work.

That is the central reason cache reads often cost less than fresh input: the provider is reusing computation instead of performing all of it again. The discount does not mean the words have become less important. It reflects a different path through the system serving your request.

Fresh input requires computation before any answer appears

Before a model generates an answer, it processes the input tokens. Tokens are pieces of text, not necessarily whole words. During this stage, often called prefill, the system builds internal representations that help the model use the supplied context while generating its response.

For a long prompt, that preparation can require substantial computation. With a supported cache, the provider can preserve reusable internal state associated with a matching prompt prefix. On a later request, it can load that state and process the new material that follows it. Reading stored state still consumes resources, but it can be cheaper than calculating the same state again.

A cache hit reuses context rather than answers

Prompt caching is not usually a lookup table of finished replies. The model still generates an answer for the current request. You might reuse the same handbook while asking a different question each time. The handbook portion may qualify for cached processing, while the question remains fresh input and the answer remains newly generated output.

This distinction matters for expectations. A cache hit does not guarantee an identical answer, eliminate output charges, or make the entire request free. It may reduce preparation time, but total response time also depends on answer length, system load, and other factors.

Stable prompt prefixes make reuse more likely

Caching rules differ between providers and models. Some systems manage caching automatically; others support explicit cache controls. Minimum prompt lengths, retention periods, supported content, and matching requirements can all vary. A repeated passage does not automatically qualify for a discount.

A useful general practice is to put stable instructions and shared reference material before changing details. Keep those stable sections consistent across requests. In prefix-based caching systems, a change near the beginning can prevent reuse of material that follows it. Timestamps, reordered tool definitions, or small edits can therefore reduce cache hits.

Do not add unnecessary text merely to pursue a discount. Cached tokens can still cost money, and retaining sensitive material deserves a separate review of the provider's privacy and retention terms.

Separate billing categories reveal the actual savings

Think of a request as potentially containing several billing categories: fresh input, cache writes, cache reads, and generated output. A cache write establishes reusable state and may carry a different price from ordinary input. A cache read uses previously stored state. The exact categories and rates depend on the provider.

The useful comparison is total cost across repeated requests, not the cache-read rate alone. A large shared document used many times may produce meaningful savings. A document used once may provide little benefit, particularly when creating its cache carries an extra charge.

Provider usage reports keep relay billing understandable

BestRelay, which publishes this article, is a quality-first API relay that forwards each request to the official model named in it, without substitution. It works with Claude Code, Codex, Gemini CLI, and OpenAI SDK clients. Its pay-as-you-go billing uses provider-reported token counts, with cache reads and writes priced separately.

Create a BestRelay account to try it with your own workload. Members also earn referral commissions when people they invite top up; that is separate from cache pricing.

The practical habit is simple: inspect reported usage, check the applicable rates, and measure several representative requests. Cheaper cache reads reward reusable work, but your actual savings depend on successful reuse and the full bill.