The tool-call protocol

This is the core of what makes Polyglot model-agnostic.

Free-text tool calls (the default)

Tools are described to the model in the system prompt, and it’s asked to emit calls as tagged blocks:

<tool_call name="read_file">
{"path": "src/app.ts"}
</tool_call>

A streaming parser reads the model’s output as it arrives and extracts those blocks. It’s fault-tolerant by design, because open-weight models get the details wrong constantly:

• trailing commas, single quotes, unquoted keys → repaired via a JSON-repair pass
• a near-miss tool name (read_files, readFile) → fuzzy-matched to the real one
• the model emitting OpenAI-style {"name": ..., "arguments": ...} fenced JSON instead of the taught envelope → recognized and accepted
• output chunked arbitrarily across network packets → the parser produces identical results regardless of where the chunk boundaries fall

The same parser and executor run under every provider, so an agent loop doesn’t silently behave differently on Claude vs. a flaky local model.

If a model produces several invalid calls in a row, Polyglot concludes it isn’t reliably usable for tool work and stops the turn rather than looping on garbage.

Structured output

For openai-compatible servers that support response_format: {type: "json_schema"} (grammar / schema-constrained decoding - recent Ollama, llama.cpp server, vLLM, LM Studio), you can opt into constraining every completion to a strict JSON envelope:

{ "structuredOutput": true }

or POLYGLOT_STRUCTURED_OUTPUT=true. This makes malformed tool-call syntax structurally impossible for that turn. Trade-offs:

• Off by default - openai-compatible covers many backends with inconsistent schema support.
• Ignored when provider is anthropic (native tool use is already reliable).
• Replies are shown once the full response arrives rather than streamed token by token, since the whole completion is one JSON object.
• Not supported against the hosted OpenAI API - its strict mode requires every schema property to be required, which Polyglot’s tool schemas (with legitimately optional properties) don’t satisfy.