Streaming protocol
Applies to:
/v1/chat/completions,/v1/responses,/v1/messages,/v1/jobs/:id/events
Nexith streaming responses are uniformly text/event-stream (SSE). Text generation uses
OpenAI-compatible chat.completion.chunk frames; image jobs use job event frames; and a few
nexith.* side-channel frames appear only when the caller explicitly enables the corresponding
nexith_* extension.
1. Text stream (Chat Completions)#
Request#
{
"model": "nexith-core",
"messages": [{"role": "user", "content": "写一首短诗"}],
"stream": true,
"stream_options": {"include_usage": true}
}Frame format#
data: {"id":"chatcmpl-...","object":"chat.completion.chunk","created":1795055220,"model":"nexith-core","choices":[{"index":0,"delta":{"role":"assistant"},"finish_reason":null}]}
data: {"id":"chatcmpl-...","object":"chat.completion.chunk",...,"choices":[{"index":0,"delta":{"content":"春"},"finish_reason":null}]}
data: {"id":"chatcmpl-...","object":"chat.completion.chunk",...,"choices":[{"index":0,"delta":{},"finish_reason":"stop"}]}
data: {"id":"chatcmpl-...","object":"chat.completion.chunk",...,"choices":[],"usage":{"prompt_tokens":12,"completion_tokens":34,"total_tokens":46}}
data: [DONE]Key points:
- Sent as
data: <json>\n\n, always terminated bydata: [DONE]. - With
stream_options.include_usage=true, a frame withchoices: []+usageis sent before[DONE]; billing follows that frame. Without it, there is no usage frame (legacy OpenAI behavior). - Tool calls are delivered incrementally as
delta.tool_calls; the concatenation rules match OpenAI. - The server keeps the connection alive (SSE comment heartbeat); the typical proxy timeout is 300s.
Errors#
- Before the first token: a normal HTTP error is returned (4xx/5xx + JSON body); a 200 is never sent first.
- After output has started: the status code can no longer change; an error object frame is sent inside the stream and the stream then ends. Clients should treat the content received so far as incomplete.
2. Side-channel frames (Nexith extensions)#
These appear only when the request carries the corresponding flag. Pure OpenAI clients never receive them.
nexith.sources (web search sources)#
Sent before the first answer frame so frontends can render citation cards first:
{
"object": "nexith.sources",
"sources": [
{"title": "…", "url": "https://…", "content": "摘要片段"}
]
}nexith.step (deep research progress)#
Each deep research stage (planning, retrieval, fetching, compression) is replayed frame by frame,
before nexith.sources:
{"object": "nexith.step", "label": "检索", "model": "…", "content": "…", "items": [...]}nexith.image_request (image-generation intent)#
When a request carries nexith_detect_media: true and L1 classifies it as an image-generation
intent, the gateway does not enter the text stream and returns non-streaming JSON directly:
{
"object": "nexith.image_request",
"capability": "image.generate",
"target_model": "nexith-image",
"original_prompt": "画一只小猫",
"enhanced_prompt": "A cute kitten, …",
"size": "1024x1024"
}The frontend should then call POST /v1/images/generations to submit an async job. Editing intent
corresponds to nexith.image_edit_request (which additionally carries reference image information).
nexith.routing_decision (Responses adapter)#
/v1/responses may return this structure when the caller needs to supply a capability
(status=requires_capability); it tells the client which capability tier the request needs instead
of fabricating a text answer.
3. The nexith object in non-streaming responses#
Non-streaming Chat Completions response bodies may carry a top-level nexith object (again, only
when extensions are enabled):
{
"id": "chatcmpl-...",
"object": "chat.completion",
"choices": [...],
"usage": {...},
"nexith": {
"memory": [{"id": "...", "content": "...", "score": 0.83}],
"sources": [{"title": "...", "url": "...", "content": "..."}],
"steps": [{"label": "...", "model": "...", "content": "..."}]
}
}All fields are optional: return_memory → memory; nexith_web_search → sources;
nexith_deep_research → steps.
4. Job event stream (Jobs SSE)#
GET /v1/jobs/:id/events returns a job event stream whose frame format differs from text streams:
data: {"event":"queued","payload":{},"ts":"2026-11-19T02:00:00Z"}
data: {"event":"started","payload":{},"ts":"..."}
data: {"event":"completed","payload":{},"ts":"..."}- Event types:
queued/started/error/completed/failed/cancelled. - Each connection replays from event 0, so a reconnecting client catches up on all past events.
- The server sends an SSE comment heartbeat every 15 seconds to keep proxies from dropping idle connections.
payload.errorinerror/failedis an opaque fixed message and contains no upstream model name or raw internal error text.- After
completed/failed/cancelled, the server ends the stream.
5. Client guidance#
- With the official SDKs, do not parse SSE manually; just use
stream=true. - Custom clients need to handle: half frames spanning chunk boundaries, the
[DONE]terminator, and empty-line heartbeats. - Reconnecting and retrying: text streams have no resumption, so a retry creates a new billable request; job streams can be safely reconnected.