Streaming

Set stream: true to receive the answer as the provider produces it.

stream = client.chat.completions.create(
    model="openai/gpt-oss-20b",
    messages=[{"role": "user", "content": "Write a haiku."}],
    stream=True,
)
for chunk in stream:
    print(chunk.choices[0].delta.content or "", end="")

The wire format#

Server-sent events (text/event-stream). Each event is a data: line carrying an OpenAI chat.completion.chunk JSON frame, forwarded incrementally as the provider's chunks arrive — not buffered into one lump at the end. Two fields are rewritten on every frame:

• id is the AI Grid invocation id (the same value as the X-AIGrid-Invocation response header);
• model is the Grid's product id, not the provider's internal name.

The stream ends with a terminal chunk carrying the usage block, then:

data: [DONE]

Standard OpenAI semantics — any OpenAI-compatible client handles it without changes.

Mid-stream errors#

A failure after the stream has opened arrives as a data: frame, not an HTTP status:

data: {"error": {"code": "provider_failed", "message": "…"}, "id": "<invocation id>"}

The id on the frame is the invocation id, so even a truncated answer can be reconciled against its usage record.

Disconnects and billing#

Settlement runs after the stream ends, on a context detached from the caller. If your client disconnects mid-answer, the server keeps draining the provider, records the usage and settles — a disconnect can neither strand a hold nor lose a charge. A stream that ends without a usage block (a provider that stops mid-answer) retains its reservation as Reconciling for operator review, exactly like the non-streaming path.

A streamed call is metered exactly like a buffered one: a conservative estimate is reserved at admission, then settled against the actual token counts from the terminal usage block, and the difference released. Streaming changes delivery, not price.

Next#

• Chat completions — the full request shape
• Usage and metering — how a call is charged