Skip to main content

How to Configure an OpenAI-Compatible Provider in nanoinfra

nanoinfra can call OpenAI-compatible model providers by configuring an apiBase, optional apiKey, and a model preset that references that provider name.

What you will build​

  • a custom provider entry
  • a model preset pointing at that provider
  • one successful nanoinfra agent run

When to use this​

Use this for a local or hosted service that exposes an OpenAI-compatible endpoint. That includes internal gateways, local model servers, and a provider proxy nanoinfra does not already name.

Install​

Quick Start ranks four install methods easiest first. This is the first of them. Use pip, Docker or a source checkout instead if you prefer, and come back here.

uv tool install nanoinfra
nanoinfra onboard --wizard

Verify the endpoint responds before debugging nanoinfra:

curl -sS https://api.example.com/v1/models

Minimal working example​

Merge this into ~/.nanoinfra/config.json:

{
  "providers": {
    "custom": {
      "apiKey": "${CUSTOM_API_KEY}",
      "apiBase": "https://api.example.com/v1"
    }
  },
  "modelPresets": {
    "primary": {
      "label": "Custom",
      "provider": "custom",
      "model": "provider-model-name",
      "maxTokens": 4096,
      "contextWindowTokens": 65536,
      "temperature": 0.1
    }
  },
  "agents": {
    "defaults": {
      "modelPreset": "primary"
    }
  }
}

Then run:

nanoinfra agent -m "Hello!"

Production notes​

  • Include the version path in apiBase when the service expects /v1.
  • Use separate provider names for separate endpoints.
  • Use a placeholder key such as EMPTY only when the endpoint requires a non-empty key but does not validate it.
  • Leave apiType unset for OpenAI-compatible custom endpoints.

Security notes​

  • Keep provider keys in environment variables.
  • Treat internal model gateways as sensitive network services.
  • Do not point nanoinfra at untrusted proxy endpoints for private workspaces.

Troubleshooting​

  • If curl /models fails, fix the provider endpoint before changing nanoinfra.
  • If nanoinfra says the model is unknown, check the model ID expected by the provider.
  • If auth fails, confirm whether the provider wants Bearer auth and whether the key is present in the environment that starts nanoinfra.