NNexithDocs
StatusDashboard ↗
Overview & quickstartModel catalogGuidesAPI referenceIntegrationsOAuth2 / OIDC

Streaming protocol

Applies to: /v1/chat/completions, /v1/responses, /v1/messages, /v1/jobs/:id/events

Nexith streaming responses are uniformly text/event-stream (SSE). Text generation uses OpenAI-compatible chat.completion.chunk frames; image jobs use job event frames; and a few nexith.* side-channel frames appear only when the caller explicitly enables the corresponding nexith_* extension.


1. Text stream (Chat Completions)#

Request#

{
  "model": "nexith-core",
  "messages": [{"role": "user", "content": "写一首短诗"}],
  "stream": true,
  "stream_options": {"include_usage": true}
}

Frame format#

data: {"id":"chatcmpl-...","object":"chat.completion.chunk","created":1795055220,"model":"nexith-core","choices":[{"index":0,"delta":{"role":"assistant"},"finish_reason":null}]}

data: {"id":"chatcmpl-...","object":"chat.completion.chunk",...,"choices":[{"index":0,"delta":{"content":"春"},"finish_reason":null}]}

data: {"id":"chatcmpl-...","object":"chat.completion.chunk",...,"choices":[{"index":0,"delta":{},"finish_reason":"stop"}]}

data: {"id":"chatcmpl-...","object":"chat.completion.chunk",...,"choices":[],"usage":{"prompt_tokens":12,"completion_tokens":34,"total_tokens":46}}

data: [DONE]

Key points:

  • Sent as data: <json>\n\n, always terminated by data: [DONE].
  • With stream_options.include_usage=true, a frame with choices: [] + usage is sent before [DONE]; billing follows that frame. Without it, there is no usage frame (legacy OpenAI behavior).
  • Tool calls are delivered incrementally as delta.tool_calls; the concatenation rules match OpenAI.
  • The server keeps the connection alive (SSE comment heartbeat); the typical proxy timeout is 300s.

Errors#

  • Before the first token: a normal HTTP error is returned (4xx/5xx + JSON body); a 200 is never sent first.
  • After output has started: the status code can no longer change; an error object frame is sent inside the stream and the stream then ends. Clients should treat the content received so far as incomplete.

2. Side-channel frames (Nexith extensions)#

These appear only when the request carries the corresponding flag. Pure OpenAI clients never receive them.

nexith.sources (web search sources)#

Sent before the first answer frame so frontends can render citation cards first:

{
  "object": "nexith.sources",
  "sources": [
    {"title": "…", "url": "https://…", "content": "摘要片段"}
  ]
}

nexith.step (deep research progress)#

Each deep research stage (planning, retrieval, fetching, compression) is replayed frame by frame, before nexith.sources:

{"object": "nexith.step", "label": "检索", "model": "…", "content": "…", "items": [...]}

nexith.image_request (image-generation intent)#

When a request carries nexith_detect_media: true and L1 classifies it as an image-generation intent, the gateway does not enter the text stream and returns non-streaming JSON directly:

{
  "object": "nexith.image_request",
  "capability": "image.generate",
  "target_model": "nexith-image",
  "original_prompt": "画一只小猫",
  "enhanced_prompt": "A cute kitten, …",
  "size": "1024x1024"
}

The frontend should then call POST /v1/images/generations to submit an async job. Editing intent corresponds to nexith.image_edit_request (which additionally carries reference image information).

nexith.routing_decision (Responses adapter)#

/v1/responses may return this structure when the caller needs to supply a capability (status=requires_capability); it tells the client which capability tier the request needs instead of fabricating a text answer.


3. The nexith object in non-streaming responses#

Non-streaming Chat Completions response bodies may carry a top-level nexith object (again, only when extensions are enabled):

{
  "id": "chatcmpl-...",
  "object": "chat.completion",
  "choices": [...],
  "usage": {...},
  "nexith": {
    "memory":  [{"id": "...", "content": "...", "score": 0.83}],
    "sources": [{"title": "...", "url": "...", "content": "..."}],
    "steps":   [{"label": "...", "model": "...", "content": "..."}]
  }
}

All fields are optional: return_memory → memory; nexith_web_search → sources; nexith_deep_research → steps.


4. Job event stream (Jobs SSE)#

GET /v1/jobs/:id/events returns a job event stream whose frame format differs from text streams:

data: {"event":"queued","payload":{},"ts":"2026-11-19T02:00:00Z"}

data: {"event":"started","payload":{},"ts":"..."}

data: {"event":"completed","payload":{},"ts":"..."}
  • Event types: queued / started / error / completed / failed / cancelled.
  • Each connection replays from event 0, so a reconnecting client catches up on all past events.
  • The server sends an SSE comment heartbeat every 15 seconds to keep proxies from dropping idle connections.
  • payload.error in error / failed is an opaque fixed message and contains no upstream model name or raw internal error text.
  • After completed / failed / cancelled, the server ends the stream.

5. Client guidance#

  • With the official SDKs, do not parse SSE manually; just use stream=true.
  • Custom clients need to handle: half frames spanning chunk boundaries, the [DONE] terminator, and empty-line heartbeats.
  • Reconnecting and retrying: text streams have no resumption, so a retry creates a new billable request; job streams can be safely reconnected.