> ## Documentation Index
> Fetch the complete documentation index at: https://runinfra.ai/docs/llms.txt
> Use this file to discover all available pages before exploring further.
# Streaming
> Stream chat completion deltas over Server-Sent Events, and ask for the usage frame.
For a model whose page lists streaming, set `stream: true` on `POST /v1/chat/completions` to get a Server-Sent Events response. Add `stream_options.include_usage` when your client needs the usage frame. `/v1/messages` uses a different, event-named stream grammar described in [Anthropic Messages](/docs/api-reference/anthropic-messages#streaming).
```python Python theme={"dark"}
import os
from openai import OpenAI
client = OpenAI(
base_url="https://api.runinfra.ai/v1",
api_key=os.environ["RUNINFRA_GATEWAY_KEY"],
)
stream = client.chat.completions.create(
model="nemotron-3-5-lightning-30b",
messages=[{"role": "user", "content": "Count from one to three."}],
max_tokens=64,
stream=True,
stream_options={"include_usage": True},
)
for chunk in stream:
if chunk.usage:
print(f"\nusage: {chunk.usage}")
continue
print(chunk.choices[0].delta.content or "", end="", flush=True)
```
```typescript TypeScript theme={"dark"}
import OpenAI from "openai";
const client = new OpenAI({
baseURL: "https://api.runinfra.ai/v1",
apiKey: process.env.RUNINFRA_GATEWAY_KEY,
});
const stream = await client.chat.completions.create({
model: "nemotron-3-5-lightning-30b",
messages: [{ role: "user", content: "Count from one to three." }],
max_tokens: 64,
stream: true,
stream_options: { include_usage: true },
});
for await (const chunk of stream) {
if (chunk.usage) console.error("usage:", chunk.usage);
process.stdout.write(chunk.choices[0]?.delta?.content ?? "");
}
```
```bash cURL theme={"dark"}
curl -N https://api.runinfra.ai/v1/chat/completions \
-H "Authorization: Bearer $RUNINFRA_GATEWAY_KEY" \
-H "Content-Type: application/json" \
-d '{"model":"nemotron-3-5-lightning-30b","messages":[{"role":"user","content":"Count from one to three."}],"max_tokens":64,"stream":true,"stream_options":{"include_usage":true}}'
```
## What arrives, and when
## Frame shapes
The response carries `Content-Type: text/event-stream`, `Cache-Control: no-cache` and `Connection: keep-alive`. Content arrives in OpenAI-compatible `data:` frames, and partial output is on `choices[].delta`.
```text theme={"dark"}
data: {"id":"chatcmpl_example","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"content":"One"}}]}
```
The usage frame has an empty `choices` array. On a hosted model it also carries the request's cost and the count of input tokens billed at the cached rate, both described in [Chat completions](/docs/api-reference/chat-completions#the-cost-of-the-request-and-its-cached-input-on-its-usage):
```text theme={"dark"}
data: {"choices":[],"usage":{"prompt_tokens":12,"completion_tokens":4,"total_tokens":16,"prompt_tokens_details":{"cached_tokens":0},"cost":0.0000028,"runinfra":{"cost_microcents":280,"cached_input_tokens":0}}}
```
A completed stream ends with `data: [DONE]`. A stream the gateway terminates early ends with an error frame and no `[DONE]`, so treat a missing `[DONE]` as an incomplete response (see [Errors](/docs/api-reference/errors)).
```text theme={"dark"}
data: [DONE]
```
## Is usage always included?
No. The gateway always asks the model for usage so it can settle the request, but it forwards the usage frame to you only when your request carried `stream_options.include_usage: true`. Omit it and content frames still stream while usage-only frames stay hidden. A usage frame is never synthesized: if the model reports no usage for a request, none is sent.
There is one exception, so you can account for generated tokens even when no answer was delivered. When every choice finishes without final answer content, the gateway forwards one annotated usage frame whether or not you opted in.
## When a stream ends without an answer
A choice that reaches its generation limit before producing answer content carries a machine-readable status on its terminal frame:
```text theme={"dark"}
data: {"choices":[{"index":0,"delta":{},"finish_reason":"length","runinfra":{"output_status":{"code":"generation_limit_reached_before_answer","message":"The model reached its generation limit before producing answer content. Increase max_tokens or max_completion_tokens if the model limit allows, or shorten the prompt, then retry."}}}]}
```
If another terminal `finish_reason` produced no answer, the code is `no_answer_content` instead. Inspect the preserved `finish_reason` before deciding whether to retry.
The annotated usage frame then reports how much of the generated output was not an answer:
```text theme={"dark"}
data: {"choices":[],"usage":{"prompt_tokens":12,"completion_tokens":64,"total_tokens":76,"prompt_tokens_details":{"cached_tokens":0},"cost":0,"runinfra":{"output_token_accounting":{"visible_answer_tokens":0,"non_answer_completion_tokens":64,"sources":{"visible_answer_tokens":"proxy_classified","non_answer_completion_tokens":"provider_reported"}},"cost_microcents":0,"cached_input_tokens":0}}}
```
A stream that produced no answer settles at zero, so this frame says `cost: 0` and `cached_tokens: 0` whatever the model reported: nothing was priced, at either rate.
The usual cause is an output budget too small for a model that reasons first. See [give reasoning models room](/docs/api-reference/chat-completions#give-reasoning-models-room).
An `Idempotency-Key` on a streaming request stops a retry from generating and charging twice, but delivered tokens are never stored and never replayed. Read [Idempotent retries](/docs/api-reference/idempotent-retries) before retrying a stream.
## Related
The request contract every stream starts from.
Retry a dropped stream without paying twice.
Every status, code, and caller action.