mirror of
https://github.com/ollama/ollama.git
synced 2026-09-21 13:38:14 -05:00
Bruce MacDonald
8edecb5c69
Reworked our /v1/chat/completions streaming to match what api.openai.com actually sends,
chunk-for-chunk, based on captures I took of real OpenAI traffic.
What changed:
- finish_reason now goes on its own chunk with an empty delta {}, instead of riding on the last content
chunk. Precedence is length > tool_calls > the response's done reason > stop.
- role is only sent on the first chunk of a stream, not on every chunk.
- With stream_options.include_usage, usage goes out on its own chunk with choices: [] after the finish
chunk.
- A truncated response keeps finish_reason: "length" even when tool calls were streamed — it used to get
overwritten with "tool_calls". Fixed in both streaming and non-streaming paths.
- The metrics-only trailer response (empty message at end of stream) no longer produces a stray
delta:{"content":""} chunk before the finish chunk. A wholly empty completion still opens with a role
chunk.
- Every chunk in a stream shares one timestamp, from the response's CreatedAt.