Stream Boundaries Are Not the Boundaries That Matter

A streamed response can arrive smoothly and still be read incorrectly. The common mistake is treating each network chunk as a complete message. Server-sent events, or SSE, define their own framing above the transport. Reliable clients preserve that distinction from the first byte through the final completion signal.

Step one separates transport delivery from event meaning

Begin with three layers: incoming bytes, SSE events, and provider-specific payloads. A read from the response body returns whatever bytes are available, not necessarily one event. A chunk might contain half a line, several events, or the middle of a Unicode character. HTTP chunk boundaries have no application-level meaning.

This distinction matters equally when connecting directly or through a relay. BestRelay is a quality-first API relay that forwards each request to the official model named in it, without substitution. Its coverage includes Claude, GPT, Gemini, DeepSeek, Kimi, GLM, Qwen, and Grok. Routing preserves model choice; it does not make transport chunks into messages.

Step two validates the response before decoding bytes

Check the HTTP status and response content type before interpreting the body as SSE. A rejected request might return an ordinary JSON error or an intermediary-generated HTML page. Feeding either into an event parser can hide the actual failure. A successful HTTP status also does not guarantee successful generation: the stream can report an application error later.

Decode the body as UTF-8 using a stateful decoder. Preserve incomplete character sequences between reads, then flush the decoder when input ends. Decoding every chunk independently can corrupt characters split across chunks. SSE also permits an initial byte-order mark, which a conforming implementation ignores.

Step three accumulates lines until an event closes

Keep a text buffer across reads and recognize the SSE line endings: carriage return, line feed, or their combination. Handle a carriage return split from its following line feed without inventing an extra blank line. An empty line closes an event block; a chunk ending does not.

Within each block, interpret field names exactly. A line beginning with a colon is a comment, often used as a heartbeat. For other lines, split at the first colon and remove at most one following space from the field value. Repeated data fields contribute separate lines, joined with newline characters. Remove the final appended newline before dispatching the assembled data.

Do not parse each data line independently as JSON. JSON belongs to the completed event payload, if the provider uses JSON at all. Also distinguish an empty data field from a block containing no data field: the former can dispatch an empty payload, while the latter does not dispatch a message. An unfinished event at end of input is not implicitly completed.

Step four interprets payloads through the provider contract

Once framing produces an event, apply the endpoint’s documented schema. Event names can distinguish content updates, tool activity, usage information, and errors. Some endpoints use a textual completion sentinel; others use structured terminal events. Neither convention belongs to SSE itself.

Append only fields defined as incremental content. Tool arguments may arrive as fragments that are not valid JSON until assembled, while some fields may be snapshots rather than additions. Avoid rendering every payload as assistant prose. Applications using Claude Code, Codex, Gemini CLI, or an OpenAI SDK should check their client’s supported streaming abstraction before adding another parser. BestRelay works with these clients, but endpoint semantics still matter.

Step five verifies completion before accepting the result

Track transport closure separately from application completion. A cleanly closed connection can still lack the provider’s expected terminal signal. Preserve partial output when useful, but label interrupted generations accurately. Cancellation, timeouts, parser failures, and provider errors deserve distinct outcomes.

Do not automatically replay a generation after interruption. SSE event identifiers and retry fields do not guarantee that an inference endpoint supports resumable generation or duplicate suppression. Follow the endpoint’s recovery contract, and retain final usage metadata when available. BestRelay bills pay-as-you-go from provider-reported token counts, with cache reads and writes priced separately; visible text is not a billing ledger.

Create a BestRelay account to test your streaming client. Members can also earn referral commission when invited users top up. For implementation quality, however, the decisive test remains simple: the same bytes must produce the same events regardless of where transport chunks split.