Long Chats Rebill More History Than You Expect

A chat window makes conversation look like a growing transcript. An API usage report reveals something less intuitive: earlier messages can become input again on every later request. Under a full-history pattern, the input for each turn grows linearly, but cumulative input grows quadratically. The distinction matters when estimating an assistant’s cost.

Step one: Identify the context sent repeatedly.

Start with the actual request, not the latest message visible in the interface. A typical chat application sends system instructions, previous user messages, previous assistant replies, and the new user message. Tool definitions, retrieved documents, and tool results can add more input.

Some APIs manage conversation state for you, but stored history does not automatically mean free context. Check the provider’s accounting rules and usage fields. For this walkthrough, assume every request includes the complete transcript, with no truncation, summarization, caching, or hidden overhead.

Step two: Calculate three turns before extrapolating.

Suppose the system instructions contain 1,000 tokens, each user message contains 100 tokens, and each assistant reply contains 200 tokens. These fixed sizes are deliberately simple; real token counts vary with content and tokenizer.

• Turn one: Input is 1,000 system tokens plus 100 user tokens, totaling 1,100. The assistant generates 200 output tokens.
• Turn two: Input includes the same instructions, the first exchange, and the new message, totaling 1,400. Another 200 output tokens follow.
• Turn three: Input reaches 1,700 tokens because both earlier exchanges remain in context. Output adds another 200 tokens.

The third request does not contain just the newest 100-token message. It contains that message plus 1,600 tokens of instructions and history. An assistant reply generated once can subsequently count as input many times.

Step three: Add requests instead of measuring transcripts.

After ten turns, the retained transcript contains 4,000 tokens: 1,000 instruction tokens, 1,000 user tokens, and 2,000 assistant tokens. Yet cumulative input is 24,500 tokens. Add the 2,000 generated output tokens, and total processed input plus output reaches 26,500.

At twenty turns, cumulative input reaches 79,000 tokens, with another 4,000 output tokens. Doubling the conversation length therefore increases total processed tokens from 26,500 to 83,000, rather than simply doubling them.

This is not a universal billing formula. It is the result of repeatedly sending an expanding transcript. Context limits, history pruning, caching, and changing message lengths alter the outcome. The practical finding remains: transcript size is not cumulative usage.

Step four: Separate token volume from monetary cost.

Input and output often have different prices. Cached input may have a discounted read price, while creating a cache entry can have a separate write price. Cache eligibility, expiration, and matching rules vary by provider, so repeated history does not guarantee a cache hit.

Track ordinary input, cache reads, cache writes, and output as distinct categories wherever the provider reports them. Follow each provider’s field definitions to avoid counting cached tokens twice. A single blended token total can conceal both savings and unexpected expense.

Step five: Measure the complete application workflow.

BestRelay, which publishes this walkthrough, is a quality-first API relay that forwards requests to the official model named, without substitution. Its coverage includes Claude, GPT, Gemini, DeepSeek, Kimi, GLM, Qwen, Grok, and others, with compatibility for Claude Code, Codex, Gemini CLI, and OpenAI SDK clients.

Its pay-as-you-go billing uses provider-reported token counts, with cache reads and writes priced separately. Members also earn a referral commission when invited users top up; that is separate from token accounting. Create a BestRelay account to get started.

For any application, log usage per request and aggregate it by user task. Include retries, tool loops, and background calls. Reduce repeated context through selective retrieval, shorter tool results, or carefully tested summaries. Then compare cost with task quality: the goal is not merely fewer tokens, but less repeated work without losing essential context.