Hardening begins where the prototype feels finished

A prototype proves possibility, not readiness

Every AI feature begins as a hopeful experiment, and that is exactly where it should begin. A prototype answers one narrow question: whether a model can produce something useful for the task at hand. It usually runs on a laptop, on a small sample of data, and on a generous supply of patience. Prompts are tuned by hand, outputs are read by the person who wrote them, and failures are remembered rather than recorded. This is a fine way to discover whether an idea has life in it.

Production readiness is a different property altogether. It asks whether the feature behaves acceptably on the oddest input a stranger will send, whether it degrades gracefully when a provider is slow or unavailable, and whether the team can explain a bad answer after a user reports it. A prototype can be brilliant and still fail all three questions. The gap between the two states is a change of mindset: the earlier you ask whether the system can be trusted, the less painful the answer becomes.

One useful habit is to treat the prototype as evidence rather than as a foundation. Keep the sample inputs that produced the best and the worst results. Write down the moment each output surprised you. Turn every surprise into a question that the production design must answer in writing. Teams that do this carry their learning forward, while teams that skip it rebuild the same lessons under deadline pressure.

Design evaluation before the first production request

Evaluation is a small, deliberate sample of the work a feature will really do, paired with a judgement about whether each result is acceptable. It is not a demo. A demo shows the happy path, while an evaluation includes the awkward middle, the ambiguous request, and the input designed to make the model careless. Build this set before tuning continues, because a team without one cannot tell whether a change improved the product or merely improved a mood.

Keep the rubric short enough to use and specific enough to argue with. Instead of asking whether an answer is good, ask whether it is correct, whether it covers the points that matter, whether it stays within scope, and whether its tone fits the audience. Have two people score a sample and compare their disagreements; the gaps in that conversation are where the feature definition is still fuzzy.

• Hold out examples the prompt author never saw, so tuning cannot quietly memorize the test.
• Run the full set on every meaningful change, and treat a score drop as a blocking signal.
• Record the model name and the prompt version beside each result, so a regression can be traced.

Automated judgement can help at scale, but only after it agrees with people on a known sample. A judge that looks precise while drifting from human opinion will quietly approve work that no user wants. Recheck that agreement whenever the task, the audience, or the model changes.

Reliability grows from budgets, limits, and graceful failure

Reliability in an AI feature is mostly the reliability of everything around the model. Requests time out. Providers return errors under load. A user submits a paragraph where the design assumed a sentence. The system must decide, in advance, what happens in each of those moments instead of improvising under pressure.

Put a budget on every dependency: a timeout for the model call, a limit on retries so a failure cannot amplify itself, and a ceiling on concurrency so a traffic spike cannot exhaust credentials or funds. Retries deserve particular care, because a model call may be costly and may not be safe to repeat. Decide which failures are worth repeating, which should surface to the user immediately, and which should quietly move work to a queue that can drain later.

• Return a clear, honest message when generation fails rather than an empty or invented result.
• Stream partial output when the experience allows it, so slow generation feels like progress.
• Keep a small, tested path that serves something useful while the full pipeline is recovering.

Graceful failure is a craft detail that users feel without naming it. The feature that admits it is temporarily unavailable keeps trust, while the one that quietly returns nonsense spends it.

Observability turns model behavior into engineering evidence

You cannot harden what you cannot see. For an AI feature, observability means capturing enough context to reconstruct a bad outcome: the request as the user sent it, the prompt and its version, the model name, the timing, the token counts, the output, and the user's reaction to it. Capture what you need and redact what you must, because logs that expose personal data become their own incident.

Aggregates matter as much as individual cases. Watch error rates by model and by prompt version, latency percentiles instead of averages, cache behavior, and the share of outputs users accept, edit, or reject. A calm average can hide a failing tenth of traffic for weeks.

Every signal should feed a loop. A bad output becomes an evaluation example, an evaluation gap becomes a rubric change, and a rubric change becomes a prompt revision reviewed with the same care as any other change. Versioning is the thread that holds this loop together, since a result you cannot attribute to a specific prompt, model, and setting is folklore rather than evidence.

Cost, privacy, and change control shape the launch

Cost is a reliability property too, because a feature that spends unpredictably will be throttled or switched off by the people who fund it. Pay-as-you-go billing based on the token counts the provider reports, with cache reads and cache writes priced separately, keeps unit economics visible at exactly the moment when traffic becomes real. Set a per-request budget, alert before it is breached, and decide in advance which features are worth a long, expensive generation and which must answer cheaply.

Model choice and routing belong in the production contract as well. Tools such as Claude Code, Codex, Gemini CLI, and any OpenAI SDK let teams keep familiar interfaces while they decide which model serves which workload, and two candidates such as Claude Sonnet 5.5 and Gemini 3.1 Pro can be compared on the same evaluation set before either earns a place in production. A quality-first relay such as BestRelay forwards each request to the official model named in it, with no substitution, which keeps those comparisons honest and makes a regression investigation unambiguous. Its pay-as-you-go billing follows the provider's own token counts, members who invite others earn a referral commission when those people top up, and that benefit is simply worth knowing while the technical choice stays yours.

Finish with a controlled launch: move a small share of real traffic first, watch the evaluation, the errors, and the bills together, and keep a rollback that one person can trigger. Review the feature on a schedule rather than only after complaints, because inputs drift and models change underneath a stable interface. Create a BestRelay account when you are ready to send real traffic.