First run with a local model

This walks through running Polyglot against a model on your own machine. No account, no API key.

1. Start a local server

Ollama - install it, then pull a coding model:

ollama pull qwen2.5-coder

Ollama serves an OpenAI-compatible API at http://localhost:11434/v1 automatically.

LM Studio - in the app, load a model, then start the local server from the Developer tab. It listens on http://localhost:1234/v1 by default.

Any other OpenAI-compatible server (vLLM, llama.cpp server, …) works the same way - see Local servers.

2. Tell Polyglot where to look

The fastest way is environment variables:

export POLYGLOT_PROVIDER=openai-compatible
export POLYGLOT_MODEL=qwen2.5-coder                 # a model your server has
export POLYGLOT_BASE_URL=http://localhost:11434/v1  # Ollama's default; adjust for LM Studio etc.

Or put it in a file so you don’t have to repeat it - see Configuration.

3. Run it

polyglot

You’ll get a TUI. Type a request and press Enter:

› list the TypeScript files in this directory and tell me which one is the entry point

Polyglot starts in manual mode: it reads files freely, but asks before running a command or writing anything. Approve with y, deny with n, or press c to type a redirection instead of a plain yes/no.

Press Shift+Tab to cycle to auto (runs without asking) or plan (read-only until it presents a plan). Ctrl+C exits.

If tool calls look unreliable

Small local models sometimes get the tool-call syntax wrong. If Polyglot reports the model “isn’t reliably producing valid tool calls”, try a stronger coding model, or - if your server supports it - turn on structured output:

export POLYGLOT_STRUCTURED_OUTPUT=true

Next

• Configuration - put your settings in a file
• Local servers - vLLM, llama.cpp, remote hosts
• Commands & shortcuts