# Handle LLM streaming responses

## Check support before enabling a stream

Streaming lets a client consume partial output. It does not guarantee lower total latency, lower cost or a complete answer. Check the selected route's declared capabilities in `/v1/models` and its model guide. If streaming is not declared, start with a non-streaming request and confirm support before enabling it.

For an OpenAI Chat Completions route with verified streaming support, set `MODEL_ID` to its exact public ID. This command requires `jq` and makes one billable generation request; it intentionally has no automatic retry.

```bash
jq -n --arg model "$MODEL_ID" \
  '{model: $model, messages: [{role: "user", content: "Explain HTTP in three steps."}], stream: true}' \
  > request.json
curl --no-buffer --fail-with-body --max-time 120 \
  "https://newrouters.com/v1/chat/completions" \
  -H "Authorization: Bearer $API_KEY" \
  -H "Content-Type: application/json" \
  --data-binary @request.json
```

`--no-buffer` changes curl output buffering; it does not configure the model. The command is for inspecting the event stream, not a production parser. Keep `request.json` out of logs if you replace the demonstration prompt with sensitive content.

## Parse events, not network chunks

A network chunk may contain half an event or several events. Use an SSE parser or the selected protocol's SDK with correct endpoint configuration. Preserve incomplete data between reads, decode UTF-8 incrementally, and dispatch only complete events. Comments and empty deltas must not become user-visible answer text.

| Protocol | Streaming entry point | What to parse |
| --- | --- | --- |
| Chat Completions | `/v1/chat/completions`, `stream: true` | SSE data, choice deltas and terminal `[DONE]` |
| Responses | `/v1/responses`, `stream: true` | Named lifecycle and output events |
| Claude Messages | `/v1/messages`, `stream: true` | Message/content-block events and stop state |
| Gemini | `/v1beta/models/{model}:streamGenerateContent?alt=sse` | Candidate parts and finish information |

Only use an entry point supported by the selected route. Do not apply the Chat Completions `[DONE]` parser to every protocol. Usage may arrive separately from text; keep it separate and treat absent usage as unknown, not zero.

## Make partial output explicit

Track connecting, receiving, complete and interrupted states. Mark an answer complete only after the expected protocol completion signal. A closed socket or HTTP 200 alone is insufficient. Surface truncation or refusal distinctly from a successful complete answer.

When a client disconnects, stop rendering and release client resources. Cancellation does not prove that upstream generation stopped or that no usage was incurred. Do not restart the entire request automatically after partial text: this can duplicate output and spend. Follow [errors and retries](/docs/llm/errors-and-retries) and measure time to first text separately from total time in your [evaluation](/docs/llm/evaluation).