HTTP client and SSE parsing.
callModel is the single outbound call: it sends the messages array as an OpenAI-compatible chat completions request, reads the Server-Sent Events stream chunk by chunk, and returns a completed assistant JsonNode plus a Usage record.
XML tool call recovery handles a gpt-oss quirk: some nvidia-hosted variants leak the model's native <tool_call> chat template into delta.content instead of the OpenAI tool_calls field. When xmlToolCalls is set for a combo, callModel promotes those tags to synthetic tool_calls so the rest of the pipeline sees a uniform shape.
Types
ApiStreamHooks = object beforeCall*: proc (lastPromptTokens, window: int): string {.closure.} afterCall*: proc () {.closure.} progress*: proc (baseLabel: string; slurped: int) {.closure.} setStatusLabel*: proc (label: string) {.closure.} startSpinner*: proc (label: string) {.closure.} stopSpinner*: proc () {.closure.} providerActivity*: proc () {.closure.} reasoningDelta*: proc (reasoning, baseLabel: string; slurped: int; contentStarted: bool) {.closure.} contentDelta*: proc (chunk, baseLabel: string; slurped: int): bool {.closure.} contentFinished*: proc (fullContent, baseLabel: string; slurped: int): bool {. closure.} trimTrailingContent*: proc (fullContent, baseLabel: string; slurped: int) {. closure.} afterLiveContent*: proc (baseLabel: string; slurped: int) {.closure.} finalUsage*: proc (usage: Usage; window, elapsed: int; assistantContent: string; streamedLive: bool) {.closure.} noUsage*: proc (elapsed: int) {.closure.}
FetchModelsHook = proc (url, key: string): (seq[string], string) {.closure.}
VerifyProfileHook = proc (p: Profile): (bool, string) {.closure.}
Consts
QuietRecvWakeMs {.intdefine.} = 500
- How long each blocking recv may stall before waking. With streamhttp's readTimeoutMs set to this, readLine raises StreamTimeoutError periodically so the stream loop can re-check the quiet/interrupt flags. Must be well under QuietTooLongMs. Kept at 1s so user interrupts (Ctrl-C) are honored within ~1s; the extra poll overhead is negligible for trickle-rate streams.
QuietTooLongMs {.intdefine.} = 180000
- If a streaming response goes this long with no data from the provider, the turn is aborted. posix.shutdown(fd) from another thread does not reliably wake a blocked TLS recv, so the stream loop instead relies on QuietRecvWakeMs-bounded reads to wake up and observe this threshold.
Procs
proc applyGenerationDefaults(p: Profile; body: JsonNode) {....raises: [], tags: [], forbids: [].}
- Known-good generation policy. Temperature is intentionally hardcoded for now; later a user override can resolve before this writes the field.
proc applyReasoning(p: Profile; body: JsonNode) {....raises: [], tags: [], forbids: [].}
-
Per-family wire mapping for Profile.reasoning. Adding a new family means: (1) set reasoning in the known-good combo table,
- write an applyXReasoning proc, (3) add a case branch.
proc applyStreamingOptions(p: Profile; body: JsonNode) {....raises: [], tags: [], forbids: [].}
-
Provider-specific additions for SSE fidelity.
Z.ai's first-party API (provider names zai, zai-coding, zaicode) gets tool_stream: true, which streams tool-call arguments as per-token deltas. This was disabled for a long time as a workaround for a streamhttp TLS read bug that truncated the per-token deltas mid-stream; that bug is fixed in streamhttp >= 0.2.0 (the recv loop drains OpenSSL's internal buffer before polling), so streamed tool args now arrive complete. Reasoning/thinking streams live regardless (gated by stream:true + thinking:enabled, not tool_stream).
proc buildBatchAssistantMsg(message, reasoning: string; toolCalls: JsonNode): JsonNode {. ...raises: [], tags: [RootEffect], forbids: [].}
- Build an assistant message from a non-streaming completion's choices[0].message. Mirrors buildStreamAssistantMsg's shape so the rest of the pipeline (history replay, tool dispatch) sees a uniform object regardless of transport. Returns nil only when the message is genuinely empty (no content, no tool_calls, no reasoning) — the caller treats that as an error to surface.
proc buildStreamAssistantMsg(content, reasoning: string; tools: OrderedTable[int, JsonNode]; usage: Usage; wasInterrupted = false): JsonNode {. ...raises: [KeyError], tags: [], forbids: [].}
- Build the assistant message reconstructed from an SSE stream. Returns nil when the stream produced no assistant data.
proc classifyRetry(exc: ref CatchableError; code: int): string {....raises: [], tags: [], forbids: [].}
- Returns "server" for network errors and 5xx, "rate" for 429, "" for anything else (not retryable). Pure-logic helper for the callModel retry block.
proc clearInterrupted() {....gcsafe, raises: [], tags: [], forbids: [].}
proc clearNetworkQuiet() {....gcsafe, raises: [], tags: [], forbids: [].}
proc closeCachedStreamConn() {....raises: [Exception], tags: [RootEffect], forbids: [].}
proc fetchModels(url, key: string): (seq[string], string) {. ...raises: [Exception, Exception], tags: [RootEffect, ReadDirEffect, ReadEnvEffect, ReadIOEffect, WriteIOEffect, TimeEffect], forbids: [].}
- GET /models on the provider. Returns (models, error) — error is empty on success. Callers are responsible for displaying the error.
proc installInterruptHook() {....raises: [], tags: [], forbids: [].}
proc isInterrupted(): bool {....gcsafe, raises: [], tags: [], forbids: [].}
proc isNetworkQuiet(): bool {....gcsafe, raises: [], tags: [], forbids: [].}
proc markNetworkQuiet() {....gcsafe, raises: [], tags: [], forbids: [].}
proc parseUsage(u: JsonNode): Usage {....raises: [], tags: [RootEffect], forbids: [].}
- Parses an OpenAI-compatible usage object. Cached-token accounting differs by provider: OpenAI/DeepInfra/Anthropic report it under prompt_tokens_details.cached_tokens; DeepSeek reports it flat as prompt_cache_hit_tokens. We accept either.
proc parseXmlToolCalls(content: string): tuple[cleaned: string, calls: seq[JsonNode]] {....raises: [], tags: [], forbids: [].}
-
Extract GLM/Qwen native <tool_call>NAME<arg_key>K</arg_key> <arg_value>V</arg_value>...</tool_call> blocks from content and promote them to OpenAI-style tool_calls entries. Returns the content with those blocks removed and the synthesized calls.
Some endpoints (e.g. nvidia z-ai/glm4.7 mid-turn) leak the model's chat-template tokens into the SSE content stream instead of parsing them into tool_calls deltas. This parser is the fallback.
proc requestTurnInterrupt() {....gcsafe, raises: [], tags: [], forbids: [].}
- One cancellation path for signal hooks, buffered prompt keys, and stream/tool stdin watchers. Setting the flag alone is not enough: blocking HTTP reads must be woken and active tool subprocesses must be signalled, otherwise Ctrl-C appears to do nothing until the provider or command produces output.
proc retryCategory(errMsg: string; assistantMsg: JsonNode; statusCode: int): string {. ...raises: [], tags: [RootEffect], forbids: [].}
proc setApiStreamHooks(hooks: ApiStreamHooks) {....raises: [], tags: [], forbids: [].}
proc setInterrupted(value: bool) {....gcsafe, raises: [], tags: [], forbids: [].}
proc stripInternalFields(messages: JsonNode): JsonNode {....raises: [], tags: [RootEffect], forbids: [].}
- Return a wire-safe copy of messages with internal bookkeeping fields removed. usage is stored on assistant messages for local replay but rejected by strict validators (fireworks, glm-5p1, etc.).
proc verifyBody(p: Profile): string {....raises: [], tags: [], forbids: [].}
- JSON body for the provider-verification ping. Kept as a named proc so the test suite can assert it matches the streaming convention used by callModel (both must send "stream": true).
proc verifyProfile(p: Profile): (bool, string) {....raises: [Exception, Exception], tags: [ RootEffect, ReadDirEffect, ReadEnvEffect, ReadIOEffect, WriteIOEffect, TimeEffect], forbids: [].}