Reliable Answers Begin With Narrow Retrieval

Define the assistant's answer boundary before retrieval begins

Start with a narrow promise: the assistant answers questions from an approved knowledge collection and clearly declines requests that the collection cannot support. This boundary is more important than model selection because retrieval cannot correct an undefined scope. Decide whether the assistant serves customers, employees, or both, then list the source types it may use. Establish a response policy for missing, conflicting, outdated, and sensitive information.

Write the policy as observable behavior. A supported answer should state the result and identify the relevant evidence. An unsupported answer should explain that the available material does not establish a reliable answer. A sensitive request should follow a separate escalation path rather than receiving a confident completion.

Prepare source material for precise evidence retrieval

Collect the documents that already represent the organization's current policy, product behavior, and support decisions. Remove duplicate pages, obsolete versions, navigation fragments, and boilerplate before indexing. Preserve titles, section names, publication dates, owners, product areas, and access labels as metadata. These fields later support filtering, ranking, citations, and maintenance.

Chunk documents at meaningful boundaries such as procedures, policy sections, troubleshooting steps, or individual question-and-answer entries. Very small fragments lose the conditions surrounding an answer, while very large fragments dilute search relevance. Keep enough neighboring context to preserve qualifications, exceptions, and prerequisites.

Choose retrieval units that preserve useful context

Create an embedding for each chunk and store it with its metadata in a vector-capable search system. Add lexical search when exact terms matter, especially for error messages, product names, plan labels, and policy identifiers. A hybrid retriever usually handles real support language better than semantic search alone.

Normalize the user's question before searching. Resolve obvious spelling variation, retain important product terms, and generate a small number of search formulations only when the original wording is ambiguous. Apply access filters before ranking results so the assistant never receives evidence the user is not allowed to see.

Rank evidence before asking the model to answer

Retrieve a broad candidate set, then use a reranking stage to prioritize passages that directly answer the question. The reranker should reward topical relevance and penalize passages that merely share vocabulary. Deduplicate overlapping chunks and keep a compact evidence set within the model's context budget.

Pass each selected passage with its source metadata and a clear instruction to rely only on the supplied evidence. Ask the model to distinguish direct statements from reasonable implications. Require source references in the final response when the interface supports them, because traceability makes review and correction much faster.

Separate answer generation from unsupported inference

Give the generator explicit rules for uncertainty. It should answer when the evidence is sufficient, ask a clarifying question when multiple interpretations remain plausible, and decline when the evidence is absent or contradictory. It should never fill a gap with general model knowledge merely because the question sounds familiar.

Keep the response format predictable. A useful answer normally contains a direct conclusion, a short explanation, and evidence references. For procedural questions, include prerequisites and important exceptions. For account-specific or operational questions, route the user to an authenticated workflow rather than pretending that public documents contain private state.

Evaluate the assistant with difficult real questions

Build an evaluation set from resolved support tickets, search logs, policy reviews, and deliberately adversarial questions. Label each item for answer correctness, evidence support, refusal quality, citation accuracy, and access compliance. Include paraphrases, misspellings, incomplete questions, conflicting documents, and questions whose answer changed over time.

Review failures by stage. Poor retrieval usually shows irrelevant or incomplete evidence. Poor generation shows unsupported claims despite good evidence. Poor policy handling shows confident answers to unanswerable questions. Track these categories separately so improvements target the actual bottleneck.

Operate the system with visible failure signals

Log retrieval scores, selected sources, model responses, latency, token usage, refusals, and user feedback under appropriate privacy controls. Alert on sudden changes in empty-result rates, citation coverage, or answer length. Assign owners to source collections and schedule reindexing whenever approved content changes.

Choose a model according to the task's reasoning, latency, and cost requirements, then test that choice against the evaluation set. BestRelay can route each request to the official model named in it without substitution, including models such as Claude Sonnet 5.5 and Gemini 3.1 Pro. Its relay works with Claude Code, Codex, Gemini CLI, and any OpenAI SDK through its API base. Billing follows provider-reported token counts, with cache reads and writes priced separately. Members can also earn referral commission when invited users top up.

Add BestRelay after the workflow proves reliable

Create a BestRelay account when the retrieval and evaluation workflow is ready for controlled usage. Keep provider selection explicit in application settings, record it with each evaluation run, and compare quality before optimizing cost. This preserves a clear relationship between the model requested, the evidence supplied, and the answer produced.

A retrieval-augmented FAQ assistant becomes dependable through disciplined boundaries, carefully prepared evidence, measured ranking, and explicit uncertainty. The model is only one component; the surrounding retrieval and evaluation process determines whether users receive answers they can trust.