Streaming

Server-sent events streaming for chat completions

Overview

Set stream: true in the request
Response is Server-Sent Events (SSE)
Compatible with OpenAI SDK streaming, Vercel AI SDK, etc.
Usage data included in the final chunk automatically (including usage.cost)
For sansa-auto requests, the sansa routing metadata appears on the first chunk and again on the final usage-bearing chunk (see Models)

Quick Example

See the code panel for streaming examples in Python, TypeScript, curl, and raw fetch with SSE parsing.

SSE Format

Each event is a JSON object followed by a blank line.

Note: Line breaks added for readability. Actual events are single-line.

data: {
  "id": "...",
  "object": "chat.completion.chunk",
  "created": 1700000000,
  "model": "openai/gpt-5.4-mini",
  "choices": [{
    "index": 0,
    "delta": { "content": "Hello" },
    "finish_reason": null
  }]
}

Stream ends with:

data: [DONE]

The first chunk of a sansa-auto stream additionally carries the sansa routing object:

{
  "id": "...",
  "object": "chat.completion.chunk",
  "created": 1700000000,
  "model": "openai/gpt-5.4-mini",
  "choices": [{
    "index": 0,
    "delta": { "role": "assistant" },
    "finish_reason": null
  }],
  "sansa": {
    "routed": true,
    "routed_model": "openai/gpt-5.4-mini",
    "routing_latency_ms": 287
  }
}

The final chunk before [DONE] includes usage (with billed cost) and may also repeat the sansa routing fields:

{
  "id": "...",
  "model": "openai/gpt-5.4-mini",
  "choices": [{
    "index": 0,
    "delta": {},
    "finish_reason": "stop"
  }],
  "usage": {
    "prompt_tokens": 12,
    "completion_tokens": 84,
    "total_tokens": 96,
    "cost": 0.00081
  },
  "sansa": {
    "routed": true,
    "routed_model": "openai/gpt-5.4-mini",
    "routing_latency_ms": 287
  }
}

Streaming Chunk Shape

interface ChatCompletionChunk {
  id: string;
  object: "chat.completion.chunk";
  created: number;
  
  // The underlying model that served the request.
  model: string;
  
  choices: {
    index: number;
    delta: {
      // Present in first chunk only.
      role?: "assistant";
      
      // Incremental text content.
      content?: string;
      
      // Incremental tool call data.
      tool_calls?: ToolCallChunk[];
      
      // Reasoning blocks (when enabled).
      reasoning_details?: ReasoningDetail[];
    };
    // Reason for stopping: "stop", "length", "tool_calls", "content_filter", or "error".
    finish_reason: string | null;
  }[];
  
  // Present in final chunk only.
  usage?: {
    prompt_tokens: number;
    completion_tokens: number;
    total_tokens: number;
    // Total billed cost for this completion in USD.
    cost: number;
  };

  // Present on the first chunk of a sansa-auto stream and on the final
  // usage-bearing chunk. Omitted on direct model requests and in between.
  sansa?: SansaCompletionExtension;
}

stream_options is accepted and has no effect. See The sansa response object for its fields.

Error Handling

Pre-Stream Errors

If validation fails before streaming starts (bad API key, invalid params, insufficient credits), a standard JSON error response is returned:

{
  "error": {
    "code": "insufficient_credits",
    "message": "Insufficient credits. Please add credits to continue."
  }
}

Mid-Stream Errors

If an error occurs after streaming has begun (HTTP 200 already sent), the error is sent as an SSE event:

data: {
  "error": {
    "code": "model_error",
    "message": "Provider disconnected"
  },
  "choices": [{
    "finish_reason": "error"
  }]
}

data: [DONE]

Mid-stream errors always have finish_reason: "error" and are followed by [DONE].

Streaming with Tool Calls

See Tools.

Streaming with Reasoning

When reasoning tokens are enabled:

delta.reasoning_details appears before delta.content
Reasoning blocks stream incrementally
Content starts streaming after reasoning completes

Note: The OpenAI SDK does not currently auto-accumulate reasoning_details from streaming deltas, so reasoning continuity across turns is not available when streaming. If reasoning continuity matters (e.g., multi-turn tool calling), use non-streaming requests. See the Reasoning docs for details.

Streaming with Structured Outputs

Structured outputs (response_format: { type: "json_schema" }) work with streaming. The model streams valid partial JSON that forms a complete valid response when done.