Streaming AI Responses With Server-Sent Events (SSE)

Streaming makes an AI interface feel responsive by showing useful text before generation finishes. It also turns a single HTTP result into an ongoing conversation between your server and the browser. That conversation needs framing, cancellation, and an explicit ending, not just a loop that appends whatever bytes arrive.

For a documentation assistant, Server-Sent Events provides a practical text-based envelope. Your server consumes the provider's stream and emits a smaller application protocol. The browser does not need provider credentials or knowledge of every upstream event type, and you retain control over errors and logging.

Choose the browser transport deliberately

Server-Sent Events, or SSE, is a format for sending events over a long-lived HTTP response. The native browser EventSource interface conveniently parses that format and can reconnect automatically. However, its constructor does not accept an arbitrary POST body or custom authentication headers.

For a question submitted as JSON, a common choice is fetch-based streaming: send a POST request, then read the response body as a stream containing SSE frames. You implement parsing and reconnection policy yourself. Another option is to create a job with POST and connect EventSource to an authenticated GET endpoint for that job.

Do not put sensitive prompts or access tokens into query strings to accommodate EventSource. URLs can appear in logs and browser history. Use your existing session model, enforce cross-origin rules, and apply CSRF protections where cookie-based authentication requires them.

The MDN SSE guide describes event framing and EventSource behaviour. Choose the transport according to authentication and request shape, not because a demo happens to use one constructor.

Define an application event contract

Keep provider events behind your server adapter. Your browser usually needs a run identifier, text additions, a completion signal, and a recoverable error message. It does not need internal reasoning content, raw provider diagnostics, or tool arguments that may contain sensitive information.

An SSE frame consists of fields separated by line endings, followed by a blank line. Put JSON inside the data field so escaping is predictable. For example, your application might emit these frames:

event: delta
data: {"runId":"r42","sequence":1,"text":"Start with "}

event: delta
data: {"runId":"r42","sequence":2,"text":"the installation guide."}

event: done
data: {"runId":"r42","lastSequence":2}

Document whether text events are deltas or snapshots. Appending a full snapshot as if it were a delta duplicates the answer. Sequence numbers help detect duplicate or missing messages, but do not automatically provide resumability; the server must retain and replay events for that feature.

Use the JSON formatter to inspect a redacted event payload when debugging escaping or unexpected fields. Format the JSON data value, not the entire SSE frame. Pair the contract with the structured JSON guide when the final answer must also satisfy a schema.

Parse frames rather than network chunks

A network chunk is not an event. One read can contain half an event, several events, or a split multibyte character. Decode bytes incrementally, buffer incomplete lines, and only dispatch an event after its blank-line delimiter arrives.

A production parser must understand carriage-return and line-feed variations, comments, empty fields, and multiple data lines. Concatenate multiple data fields with newline separators according to SSE rules. Prefer a maintained parser with tests over a quick split operation that only handles the provider's current happy path.

With fetch, verify the HTTP status and expected content type before starting the event parser. An authentication failure may return ordinary JSON or an HTML error page. Treating that body as an empty stream hides the useful error and leaves the loading indicator running indefinitely.

Use incremental decoding through a streaming TextDecoder or TextDecoderStream. The MDN Streams documentation explains the underlying read model. Flush decoder state at the end, and regard an unfinished frame or missing completion event as an interrupted response rather than quietly claiming success.

Make the server and infrastructure cooperate

Send the appropriate text/event-stream content type and configure caching so generated answers are not accidentally shared. Flush headers promptly where the server framework permits it. Some reverse proxies, compression layers, and hosting platforms buffer small writes, so correct application code can still arrive as one large batch.

Test through the production delivery path, including the CDN and load balancer. A local development server proves little about intermediary buffering or maximum connection duration. Check the hosting platform's streaming support and timeout limits before designing a multi-minute workflow.

Heartbeats can keep some idle connections active, but they do not override every proxy timeout. SSE comments are suitable heartbeat frames because compliant parsers ignore them. Choose an interval compatible with infrastructure limits and avoid treating a heartbeat as evidence that generation is progressing.

Respect backpressure when writing. A slow browser should not cause unlimited in-memory buffering while the provider continues producing tokens. Bound queues, handle failed writes, and stop upstream work when the request disconnects. Keep connection counts and accumulated output size within limits even for authenticated users.

Handle cancellation, errors, and completion visibly

A stop button should abort the browser request and signal cancellation upstream where supported. Cancellation does not guarantee that all already-started provider work disappears or becomes free. Record it distinctly from server failures so normal user behaviour does not inflate your error rate.

Model the interface with explicit states: connecting, receiving, complete, stopped, and failed. Preserve partial text after an interruption, but label it as incomplete. Announce status changes accessibly without sending every token into a live region; continuous announcements can overwhelm assistive technology.

After headers have been sent, the server cannot replace the response with a new HTTP error status. Emit a sanitised error event when possible, close the connection, and include a support identifier. Never expose credentials, raw stack traces, or the full upstream request in the browser.

Avoid blind reconnection. Native EventSource retries can restart an operation unless your job protocol supports event replay and stable identifiers. For fetch-based streams, provide an explicit retry action and decide whether it restarts generation or resumes stored output. The LLM API basics guide covers deadlines and upstream retry costs.

Test split delimiters, Unicode boundaries, duplicate sequences, disconnects, provider refusal, and completion without text. Also test a browser tab that becomes inactive. MDN's AbortController reference explains cancellation primitives, but application tests must verify that cancellation reaches your server and provider adapter. A stream is reliable only when its ending is as well defined as its beginning.

Key takeaways

• Choose EventSource or fetch according to authentication and request needs.
• Publish a small event contract with explicit completion and error signals.
• Decode incrementally and parse complete SSE frames, not read chunks.
• Test buffering, connection limits, and cancellation through production infrastructure.
• Preserve incomplete answers honestly and avoid automatic duplicate generation.