Polite Retries Turn Rate Limits Into Controlled Work

A retry is another request competing for the capacity that just ran out. When an API returns HTTP 429, an immediate repeat can deepen congestion, waste a deadline, and synchronize your application with thousands of other clients. Polite retries treat rate limiting as feedback: pause, reduce pressure, and resume only while the work remains useful.

Step one identifies the limit before retrying

Start with the response status, headers, and structured error details. HTTP 429 means too many requests, but the underlying constraint might involve request frequency, token throughput, concurrency, or an account allowance. A short traffic burst and an exhausted spending allowance need different remedies. Repeating a request cannot replenish a balance.

Classify the failure before scheduling another attempt. Follow documented provider guidance, and treat an unclear error conservatively. Preserve request identifiers for investigation, but avoid recording prompts, credentials, or sensitive response content in routine retry logs.

Step two respects the server's requested pause

Look for Retry-After. Its standard forms are a nonnegative number of seconds or an HTTP date. Parse both rather than assuming every value is a duration. Date-based values require attention to clock skew; synchronized clocks and the response's Date header can help interpret them.

For a retryable request, treat a valid server-directed delay as a minimum wait. If it exceeds the operation's remaining deadline, stop or defer the work instead of shortening the pause. Missing or malformed guidance calls for a local backoff policy, not an immediate retry loop.

Step three adds randomness without shortening guidance

Use capped exponential backoff when the server supplies no usable delay: increase the waiting window after each failed attempt, up to a configured maximum. Apply jitter by choosing a random delay within that window. This spreads clients across time instead of having every worker return at the same instant.

When combining local backoff with valid server guidance, never let randomization pull the next attempt ahead of the server's minimum. One approach is to wait for the greater of the server delay and the local backoff, then add a small randomized spread. A cap on local backoff must not override a longer server-directed pause.

Step four bounds retries by useful lifetime

Set both a total-attempt limit and an overall deadline. Count the original request explicitly so operators understand the configured budget. Include waiting time and request execution time in that deadline, and cancel scheduled retries when the caller disconnects or cancels.

Choose one layer to own the policy. An SDK, application wrapper, queue worker, and gateway can each retry independently, multiplying traffic unexpectedly. Inspect SDK defaults and coordinate them with the application budget. When attempts run out, return a clear failure or move eligible background work into a bounded queue.

Step five checks replay safety before resending

A retry is safe only when repeating the operation is acceptable. For operations with side effects, use documented idempotency support where available; do not assume an arbitrary key guarantees deduplication. A rejected request and an interrupted request with an uncertain outcome deserve different handling.

Streaming needs particular care. Once output reaches a consumer, silently restarting can duplicate content or produce a different continuation. Define whether the application discards partial output, reports interruption, or offers an explicit restart. Do not promise exactly-once execution or zero billing impact without a documented guarantee.

Step six reduces pressure across shared workers

Backoff alone cannot fix sustained overload. Bound concurrency, smooth dispatch, and coordinate limits across workers sharing credentials. Track 429 frequency, queue age, attempts per operation, and eventual success. Reduce admission when throttling rises, then recover gradually rather than releasing the entire backlog together.

Step seven keeps relay behavior and accounting explicit

BestRelay, which publishes this walkthrough, is a quality-first relay that forwards each request to the official model named in it without substitution. It covers Claude, GPT, Gemini, DeepSeek, Kimi, GLM, Qwen, Grok, and more, and works with Claude Code, Codex, Gemini CLI, and any OpenAI SDK. Verify where throttling originated rather than assuming every 429 shares one quota.

Billing is pay-as-you-go using provider-reported token counts, with cache reads and writes priced separately. Members also earn referral commissions when invited users top up. Create a BestRelay account to try your workflow. Whatever the connection, keep retry policy bounded, observable, and respectful of upstream capacity.