Providers and Models

Use this page when the first reply fails because the provider and the model do not match. Use it also when you want to adapt the concrete setup example to a different provider. If you already know which provider you want and only need a pasteable setup, use provider-cookbook.md.

For normal local setup, open Settings → Models in the WebUI to add provider credentials, create a model preset, and select the active model. Use the JSON below for manual deployments, local endpoints, provider-specific fields, or diagnosis.

For every setup, answer three questions:

Which provider owns the credential or endpoint?
What model name does that provider expect?
Does the provider need apiKey, apiBase, OAuth login, cloud credentials, or only a local server URL?

Prefer a named modelPresets entry for the model/provider pair, then select it with agents.defaults.modelPreset. Direct agents.defaults.provider and agents.defaults.model still work for existing configs, but presets make runtime /model switching and fallback chains clearer. Pin provider inside the preset while setting up. You can switch back to "auto" later.

Choose a Provider Without Guessing

The docs show concrete provider names so the JSON is copyable, not because nanoinfra ranks providers. Start from the service or endpoint you actually control:

If you have...	Configure...
An API key from a hosted provider or gateway	That provider's `providers.<name>.apiKey`, then a preset with that provider name and a model ID from that service.
An OpenCode Zen or Go key	`providers.opencodeZen.apiKey` or `providers.opencodeGo.apiKey`, then a preset with `provider: "opencode_zen"` or `provider: "opencode_go"`.
A company proxy or regional endpoint	The matching provider block plus `apiBase` if the proxy gives you a URL.
A local OpenAI-compatible server	A local provider block such as `ollama`, `vllm`, `lmStudio`, or `custom`, usually with `apiBase`.
An OAuth-based account	Run the matching `nanoinfra provider login ...` command, then select that provider explicitly in a preset.
No provider yet	Pick one outside nanoinfra based on account access, pricing, regional availability, privacy requirements, and the model IDs you need. Then come back with its key and model ID.

Minimal Shape

{
  "providers": {
    "openrouter": {
      "apiKey": "sk-or-v1-xxx"
    }
  },
  "modelPresets": {
    "primary": {
      "provider": "openrouter",
      "model": "anthropic/claude-opus-4.5",
      "maxTokens": 8192,
      "contextWindowTokens": 65536,
      "temperature": 0.1
    }
  },
  "agents": {
    "defaults": {
      "modelPreset": "primary"
    }
  }
}

The provider config gives nanoinfra credentials and endpoint details. The model preset names the provider/model pair. The agent defaults choose which named preset to use for normal turns. Replace the example provider and model together. Mixing an API key from one provider with a model ID from another is the most common first-run failure.

Provider, Model, API Key, and Base URL

These fields answer different questions:

Field	Where it lives	Meaning
`provider`	`modelPresets.<name>.provider`	Which nanoinfra provider adapter should send the request.
`model`	`modelPresets.<name>.model`	The model ID expected by that provider or gateway.
`apiKey`	`providers.<provider>.apiKey`	Credential for that provider. Use `${ENV_VAR}` for secrets.
`apiBase`	`providers.<provider>.apiBase`	HTTP base URL of the provider endpoint.
`proxy`	`providers.<provider>.proxy`	Optional HTTP proxy for this provider only. Supported for OpenAI-compatible providers, OpenAI Codex, and xAI OAuth.

You usually omit apiBase for hosted built-in providers such as OpenRouter, Anthropic direct, OpenAI direct, Groq, or Bedrock because nanoinfra knows their default endpoints. Set apiBase for custom, local OpenAI-compatible servers, provider proxies, regional endpoints, or subscription endpoints. Include the API version path when the endpoint requires it, for example https://api.example.com/v1 or http://localhost:11434/v1.

Use proxy when one provider must send HTTP traffic through a proxy without changing process-wide HTTP_PROXY / HTTPS_PROXY. This is supported for providers that use nanoinfra's OpenAI-compatible client, including openai, custom, named custom providers, OpenRouter-style gateways, local OpenAI-compatible servers, and similar registry entries. It is also supported for openai_codex and xai_grok, including OAuth token exchange/refresh and model requests. Native provider backends such as anthropic, bedrock, azure_openai, and github_copilot reject proxy. Use their endpoint-specific configuration instead.

Common Provider Patterns

Each of these has a runnable recipe in the Provider Cookbook — the credential, the config, the check, and what a failure means. This page used to restate them, and the two copies drifted. The same provider carried a different model ID and a different maxTokens on each page. So a reader following one got a different setup than a reader following the other.

Provider	Recipe
OpenRouter gateway	Recipe: OpenRouter Gateway
OpenCode Zen or Go	Recipe: OpenCode Zen or Go
OpenAI direct	Recipe: OpenAI Direct
Anthropic direct	Recipe: Anthropic Direct
Kimi coding plan	Recipe: Kimi Coding Plan
Any OpenAI-compatible endpoint	Recipe: Custom OpenAI-Compatible Provider
Ollama	Recipe: Ollama Local Model
vLLM or LM Studio	Recipe: vLLM or LM Studio

For every field and its default, see Provider and Model Configuration.

The four below stay here because no recipe covers them yet.

Eden AI Gateway

Eden AI exposes an OpenAI-compatible chat-completions endpoint at https://api.edenai.run/v3. Configure the built-in edenai provider and use the full provider/model identifier listed by Eden AI:

{
  "providers": {
    "edenai": {
      "apiKey": "${EDENAI_API_KEY}"
    }
  },
  "modelPresets": {
    "primary": {
      "provider": "edenai",
      "model": "anthropic/claude-sonnet-4-5",
      "maxTokens": 8192
    }
  },
  "agents": {
    "defaults": {
      "modelPreset": "primary"
    }
  }
}

nanoinfra sends the model ID unchanged, including its provider prefix. Use Eden AI's model listing to choose a currently available model. The WebUI can also load that catalog after the Eden AI API key is saved under Settings → Models.

ModelScope

ModelScope (魔搭社区) exposes an OpenAI-compatible LLM endpoint plus a separate async image generation API. Both are covered by the built-in modelscope provider.

Create a ModelScope access token, then choose a model whose page exposes API-Inference. The example below uses Qwen/Qwen3-32B. Hosted availability and quotas are controlled by ModelScope. See the official API-Inference guide for current service details.

{
  "providers": {
    "modelscope": {
      "apiKey": "${MODELSCOPE_API_KEY}"
    }
  },
  "modelPresets": {
    "primary": {
      "provider": "modelscope",
      "model": "Qwen/Qwen3-32B",
      "maxTokens": 8192,
      "contextWindowTokens": 65536
    }
  },
  "agents": {
    "defaults": {
      "modelPreset": "primary"
    }
  }
}

Use an inference-enabled model ID exactly as ModelScope publishes it (usually Namespace/model-name). The default base URL is https://api-inference.modelscope.cn/v1. Override providers.modelscope.apiBase only if your account routes through a different host. Chat model IDs may optionally be prefixed with modelscope/. nanoinfra strips that routing prefix before sending the request.

ModelScope image generation reuses the same provider key but is configured under tools.imageGeneration, not in a model preset:

{
  "tools": {
    "imageGeneration": {
      "enabled": true,
      "provider": "modelscope",
      "model": "Qwen/Qwen-Image-2512"
    }
  }
}

Use the image model's exact ModelScope ID without a leading modelscope/. The image client sends this value unchanged and handles ModelScope's async submit/poll flow. The example uses Qwen/Qwen-Image-2512. See Image Generation for supported sizes, aspect ratios, and the complete provider configuration.

AWS Bedrock

Bedrock can use the AWS credential chain, profile, region, or Bedrock bearer token depending on your AWS setup.

{
  "providers": {
    "bedrock": {
      "region": "us-east-1",
      "profile": "default"
    }
  },
  "modelPresets": {
    "primary": {
      "provider": "bedrock",
      "model": "bedrock/anthropic.claude-sonnet-4-5-20250929-v1:0",
      "maxTokens": 8192,
      "contextWindowTokens": 200000
    }
  },
  "agents": {
    "defaults": {
      "modelPreset": "primary"
    }
  }
}

See configuration-providers.md#providers for Bedrock-specific notes.

OAuth Providers

Some providers do not use API keys in config.json.

For OpenAI Codex:

nanoinfra provider login openai-codex --set-main

For an eligible X Premium / Grok subscription:

nanoinfra provider login xai-grok --set-main

This selects xai-grok/grok-4.5. The provider reads xAI's model catalog and exposes the hosted x_search tool only when the selected model advertises supportsBackendSearch. Otherwise the model runs without hosted X Search. When enabled, Grok can search current X posts and return inline source links without invoking a local nanoinfra tool. Credentials are stored under the active instance's auth/xai.json (normally ~/.nanoinfra/auth/xai.json), not in config.json and not in Grok Build's credential file.

The login is xAI subscription OAuth, not X Developer OAuth. It follows the public client contract documented and implemented by Grok Build. xAI may change that upstream contract independently of nanoinfra.

For GitHub Copilot:

nanoinfra provider login github-copilot --set-main

Each command authenticates the selected provider and makes its current default model active. OpenAI Codex and eligible GitHub Copilot models participate in Responses state retention, while native compaction remains provider-capability-specific. OAuth providers are not valid automatic fallbacks. See troubleshooting.md for proxy, headless-login, model-name, and config-key errors.

Provider Resolution

The recommended path is a named preset selected by agents.defaults.modelPreset. The effective model parameters come from:

the named modelPresets entry referenced by agents.defaults.modelPreset
otherwise the implicit default preset built from agents.defaults.model, provider, maxTokens, contextWindowTokens, temperature, and related fields.

Provider selection follows this practical rule:

Explicit provider in the active preset or implicit default config wins.
provider: "auto" tries model-name keywords, configured keys, local base URLs, and gateway providers.
Gateway providers such as OpenRouter and AiHubMix can route many model families, so the model name must be valid for that gateway.
Local providers should normally be explicit because generic local model names such as llama3.2 do not always contain provider keywords.

Model Name Prefixes

family/model-name does not always select provider family. Prefix-based provider inference only runs when the active provider is "auto".

Explicit provider wins: provider: "openrouter" with model: "anthropic/claude-sonnet-4.5" calls OpenRouter, not Anthropic.
With provider: "auto", a prefix matching a configured built-in or named custom provider can select that provider. Named custom prefixes are stripped before request, so companyProxy/gpt-4o-mini is sent upstream as gpt-4o-mini.
With an explicit named custom provider, the model is sent as written provider: "companyProxy" with model: "openai/gpt-4o-mini" sends openai/gpt-4o-mini to companyProxy.

Pin provider in presets when using gateway catalog IDs such as anthropic/claude-sonnet-4.5.

Model Presets

Model presets are the recommended model configuration surface. Use them when you want named model choices, runtime /model switching, or reusable fallback targets.

{
  "modelPresets": {
    "fast": {
      "label": "Fast",
      "provider": "openrouter",
      "model": "anthropic/claude-sonnet-4.5",
      "maxTokens": 4096,
      "contextWindowTokens": 65536,
      "temperature": 0.1
    },
    "deep": {
      "label": "Deep",
      "provider": "anthropic",
      "model": "claude-opus-4-5",
      "maxTokens": 8192,
      "contextWindowTokens": 200000,
      "temperature": 0.1
    }
  },
  "agents": {
    "defaults": {
      "modelPreset": "fast"
    }
  }
}

The preset name default is reserved for the implicit agents.defaults settings. Do not define modelPresets.default. Use /model default to return to the direct agents.defaults.* fields in older configs.

Fallback Models

Fallbacks are useful for transient provider failures, rate limits, or model availability issues. Keep fallbacks compatible with the task size and tool use. Prefer fallback presets so each candidate has a name and a complete provider, model, generation, and context-window configuration.

{
  "modelPresets": {
    "fast": {
      "label": "Fast",
      "provider": "openrouter",
      "model": "anthropic/claude-sonnet-4.5",
      "maxTokens": 4096,
      "contextWindowTokens": 65536,
      "temperature": 0.1
    },
    "deep": {
      "label": "Deep",
      "provider": "anthropic",
      "model": "claude-opus-4-5",
      "maxTokens": 8192,
      "contextWindowTokens": 200000,
      "temperature": 0.1
    },
    "localSmall": {
      "label": "Local Small",
      "provider": "ollama",
      "model": "llama3.2",
      "maxTokens": 4096,
      "contextWindowTokens": 32768,
      "temperature": 0.2
    }
  },
  "agents": {
    "defaults": {
      "modelPreset": "fast",
      "fallbackModels": ["deep", "localSmall"]
    }
  }
}

String entries in fallbackModels are preset names, not raw model names. nanoinfra tries them in order after the active preset. Each fallback preset uses its own provider, model, maxTokens, contextWindowTokens, temperature, and optional reasoningEffort.

Use inline fallback objects only when a model is not worth naming as a preset:

{
  "modelPresets": {
    "fast": {
      "provider": "openrouter",
      "model": "anthropic/claude-sonnet-4.5",
      "maxTokens": 4096,
      "contextWindowTokens": 65536
    }
  },
  "agents": {
    "defaults": {
      "modelPreset": "fast",
      "fallbackModels": [
        {
          "provider": "deepseek",
          "model": "deepseek-v4-pro",
          "maxTokens": 4096,
          "contextWindowTokens": 262144
        }
      ]
    }
  }
}

fallbackModels belongs under agents.defaults, not inside each preset. If fallback candidates use smaller context windows, nanoinfra builds context from the smallest window in the active chain. Every candidate can then receive the same prompt. See configuration-providers.md#model-fallbacks for failure conditions.

Quick Checks

Run these before debugging a chat app:

nanoinfra status
nanoinfra agent -m "Hello!"

If nanoinfra agent -m "Hello!" fails:

Symptom	Likely cause
401, unauthorized, invalid API key	Key is missing, expired, copied with whitespace, or stored under the wrong provider
model not found	Model ID does not exist for the selected provider or gateway
connection refused	Local provider server is not running or `apiBase` points to the wrong port
provider not found	The active preset uses a misspelled provider. Use registry names such as `openrouter`, `anthropic`, `ollama`, `vllm`, `lm_studio`
works in CLI but not chat app	Provider is fine. Debug gateway/channel setup in `channels.md` or `troubleshooting.md`

For the complete provider table and advanced provider-specific notes, see configuration-providers.md#providers.

Choose a Provider Without Guessing​

Minimal Shape​

Provider, Model, API Key, and Base URL​

Common Provider Patterns​

Eden AI Gateway​

ModelScope​

AWS Bedrock​

OAuth Providers​

Provider Resolution​

Model Name Prefixes​

Model Presets​

Fallback Models​

Quick Checks​