Models & Providers — Choose Your Engine

Lesson 3: Models & Providers — Choose Your Engine

Pi is model-agnostic. The providers system connects it to Anthropic, OpenAI (including Codex), Google Gemini, xAI, DeepSeek, Mistral, Groq, OpenRouter, and many more — plus local servers like Ollama, LM Studio, vLLM, and llama.cpp. You switch with /model, and /scoped-models decides which models the Ctrl+P cycle offers. /login manages credentials (subscription OAuth or stored API keys).

Custom providers and models: models.json

The extension point here is ~/.pi/agent/models.json. You declare a provider with a baseUrl, an API dialect, and a list of models. Four API dialects are supported: openai-completions, openai-responses, anthropic-messages, and google-generative-ai.

{
  "providers": {
    "ollama": {
      "baseUrl": "http://localhost:11434/v1",
      "api": "openai-completions",
      "apiKey": "ollama",
      "models": [
        { "id": "llama3.1:8b" },
        { "id": "qwen2.5-coder:7b" }
      ]
    },
    "my-google": {
      "baseUrl": "https://generativelanguage.googleapis.com/v1beta",
      "api": "google-generative-ai",
      "apiKey": "$GEMINI_API_KEY",
      "models": [
        { "id": "gemma-4-31b-it", "reasoning": true, "contextWindow": 262144, "input": ["text", "image"] }
      ]
    }
  }
}

Per-model fields worth knowing: id, name, reasoning, input (text/image), contextWindow, maxTokens, and the four cost fields that drive the footer's cost estimate: cost.input, cost.output, cost.cacheRead, and cost.cacheWrite. Some OpenAI-compatible servers need compat.supportsDeveloperRole: false or compat.supportsReasoningEffort: false (common with Ollama, vLLM, and SGLang).

API keys support shell-command resolution ("apiKey": "!op read 'op://vault/item/credential'"), environment interpolation ($MY_API_KEY or ${KEY_PREFIX}_${SUFFIX}), and literal values. The file reloads every time you open /model — no restart needed.

Thinking levels

Pi exposes thinking as a dial, not a switch: off, minimal, low, medium, high, xhigh, max (set via /settings or defaultThinkingLevel). You can remap the token budget each level spends with thinkingBudgets in settings.json:

{
  "thinkingBudgets": { "minimal": 1024, "low": 4096, "medium": 10240, "high": 32768 }
}

Local inference

/llama downloads, loads, and unloads llama.cpp router models — a fully local option. Local models (Ollama, vLLM, LM Studio) have zero marginal cost per token and no data leaves your machine; the trade-off is capability and speed. Pi's docs also cover running pi inside containers for sandboxing.

Further Reading

• Providers — supported providers and setup
• Custom Models — models.json reference
• Custom Provider — full configuration guide
• llama.cpp — local inference via /llama