The tool-call protocol

This is the core of what makes Polyglot model-agnostic.

Free-text tool calls (the default)

Tools are described to the model in the system prompt, and it’s asked to emit calls as tagged blocks:

<tool_call name="read_file">
{"path": "src/app.ts"}
</tool_call>

Each tool’s parameters are written out in full, so the model sees every rule its call is checked against: the allowed values of a parameter that has a fixed list, what a list holds, the keys of a nested object and which are required, and patterns, formats and limits. This matters most for MCP tools, whose schemas are often richer than Polyglot’s own:

interface CreateTicketArgs {
  title: string;
  priority: "low" | "medium" | "high";
  labels?: string[]; // at most 5 items
  payload?: {
    queue: string;
    retries?: integer; // minimum 0
  };
}

The example call in the prompt fits the tool’s schema too.

A streaming parser reads the model’s output as it arrives and extracts those blocks. It’s fault-tolerant by design, because open-weight models get the details wrong constantly:

• trailing commas, single quotes, unquoted keys → repaired via a JSON-repair pass
• a near-miss tool name (read_files, readFile) → fuzzy-matched to the real one
• the model emitting OpenAI-style {"name": ..., "arguments": ...} fenced JSON instead of the taught envelope → recognized and accepted
• output chunked arbitrarily across network packets → the parser produces identical results regardless of where the chunk boundaries fall

The same parser and executor run under every provider, so an agent loop doesn’t silently behave differently on Claude vs. a flaky local model.

If a model produces several invalid calls in a row, Polyglot concludes it isn’t reliably usable for tool work and stops the turn rather than looping on garbage.

When a model stops short

Some local models end a turn without doing what they just said they would: “Let me start by reading the file.” and then nothing, or a reply with no text at all. Neither is a finished answer, so Polyglot asks the model to continue: to make the call it announced, or to summarise if the task is in fact done. It asks at most twice per message you send for each case, and a reply that finishes the task, asks you a question or says “let me know” ends the turn as usual.

A reply can also be cut off by the server’s output length limit: in the middle of an answer, in the middle of a tool call, or, with reasoning models, before anything visible is written. A tool call cut off this way is never run, since its arguments are incomplete and repairing them would, for example, write half a file; the model is told and makes the call again in smaller pieces. A complete call made before the limit runs as usual. An answer cut off before any call, or an empty reply, gets a request to continue, at most twice per message.

Seeing what was repaired

Repair is never silent. Every repaired call is marked ↺ repaired in the transcript; Ctrl+R (or /raw) shows the verbatim block the model actually emitted, and /reliability keeps a per-model tally for the session - tool calls made, repaired, failed to parse, given up on. The same counts are in the -p --output-format json envelope and, with audit.enabled, the raw pre-repair block is written to the audit log on every repair. The point is that a model quietly getting worse at tool syntax stays visible instead of being smoothed over.

polyglot replay <session> runs the other direction: it re-resolves a saved session’s model output through the current parser and reports what changed - a call that no longer resolves (a parser regression), a repair that’s newly needed, a fix that would now have handled a past failure.

Structured output

For openai-compatible servers that support response_format: {type: "json_schema"} (grammar / schema-constrained decoding - recent Ollama, llama.cpp server, vLLM, LM Studio), you can opt into constraining every completion to a strict JSON envelope:

{ "structuredOutput": true }

or POLYGLOT_STRUCTURED_OUTPUT=true. This makes malformed tool-call syntax structurally impossible for that turn. Trade-offs:

• Off by default - openai-compatible covers many backends with inconsistent schema support.
• Ignored when provider is anthropic (native tool use is already reliable).
• Replies are shown once the full response arrives rather than streamed token by token, since the whole completion is one JSON object.
• Not supported against the hosted OpenAI API - its strict mode requires every schema property to be required, which Polyglot’s tool schemas (with legitimately optional properties) don’t satisfy.