Long Conversations Spend More Before Answers Begin

A short answer can arrive with a surprisingly large bill. The response is only one part of the work: the model also processes instructions, conversation history, retrieved documents, and tool results. Understanding that split makes API spending easier to predict and gives engineers better optimization targets than simply asking for shorter replies.

Step one counts everything entering the request

Start with the complete request, not just the latest message. Input tokens can include system instructions, developer instructions, earlier conversation turns, attached text, tool definitions, and results returned by external tools. A tokenizer divides this material into model-specific units; tokens are not consistently equivalent to words or characters.

In a coding assistant, a one-sentence request may accompany substantial repository context. In a support application, a brief question may bring along several retrieved articles. Those additions can improve accuracy, but they also increase the amount of input processed. For images, audio, and other modalities, consult the applicable provider's accounting rules rather than estimating from visible text alone.

Step two separates generated tokens from visible prose

Output tokens represent generated material under the provider's accounting rules. That can include ordinary text and generated tool-call arguments. Some models also account for reasoning tokens that are not displayed as part of the final answer, so visible response length is not always a reliable measure of billed output.

An output limit is a ceiling, not a promise that every request will consume that amount. Lowering it can constrain runaway generation, but setting it too aggressively may truncate useful answers or interrupt structured responses. Measure actual usage before changing limits across an application.

Step three applies separate rates to each category

Use a hypothetical rate card to make the arithmetic concrete. Suppose ordinary input costs three dollars per million tokens and output costs twelve dollars per million. A request with 20,000 input tokens and 2,000 output tokens costs six cents for input and 2.4 cents for output, totaling 8.4 cents before other applicable charges.

The output rate is four times higher, yet input still contributes most of this bill because its volume is ten times larger. This is the central budgeting lesson: the largest rate is not necessarily the largest expense. Multiply each category's measured token count by its own rate, using the same billing unit throughout.

Step four treats cache traffic as distinct usage

Prompt caching can change that calculation when a provider recognizes an eligible reusable prefix. Suppose 15,000 of those input tokens qualify as cache reads at thirty cents per million, while 5,000 remain ordinary input. Input then costs 1.95 cents, and the same output brings the total to 4.35 cents.

That example excludes cache-write charges. Providers differ in cache eligibility, retention periods, and write pricing. Follow their category definitions to avoid counting cached tokens twice. Stable instructions and reusable context can improve cache opportunities, but a cache hit should be verified in usage data rather than assumed from repeated text.

Step five follows the entire multi-request workflow

A single user action may trigger several model requests. Tool use, retries, agent loops, and follow-up checks can each add usage. Earlier generated text may also return as input on a later turn. Track the cost of the completed task, not merely the cost of its final response.

This matters across Claude Code, Codex, Gemini CLI, and applications using an OpenAI SDK, all supported by BestRelay. The relay forwards each request to the official model named in it without substitution, across providers including Claude, GPT, Gemini, DeepSeek, Kimi, GLM, Qwen, and Grok.

Step six reconciles savings against completed task quality

BestRelay uses pay-as-you-go billing based on provider-reported token counts, with cache reads and writes priced separately. Reconcile those categories with your application's task records. Reduce irrelevant context, control repeated tool output, and request concise responses where appropriate, then check whether completion quality holds.

Create a BestRelay account to start tracking your API usage. Members also earn a referral commission when people they invite top up; keep that account benefit separate from request-cost calculations. The useful engineering metric remains cost per successful task, not the lowest token count in isolation.