Skip to main content

How to Run an OpenAI-Compatible Agent API with nanoinfra

nanoinfra can expose a local OpenAI-compatible endpoint behind /v1/chat/completions. This lets existing OpenAI-style clients talk to a tool-using nanoinfra agent instead of a raw model.

What you will build​

  • a working nanoinfra agent
  • a local API server on 127.0.0.1:8900
  • a /v1/chat/completions request
  • optional session isolation with session_id

When to use this​

Use this when an existing client, another language, or a separate process already knows how to call an OpenAI-compatible API. Use the Python SDK when you want in-process access to sessions, memory, runtime helpers, and hooks.

Install​

Quick Start ranks four install methods easiest first. This is the first of them. Use pip, Docker or a source checkout instead if you prefer, and come back here.

uv tool install nanoinfra
nanoinfra plugins enable api
nanoinfra onboard --wizard
nanoinfra agent -m "Hello!"

Minimal working example​

Start the API server:

nanoinfra serve

Call the chat endpoint:

curl http://127.0.0.1:8900/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "messages": [{"role": "user", "content": "hi"}],
    "session_id": "demo"
  }'

Production notes​

  • Pass session_id to isolate users, jobs, or workflows.
  • Streaming uses Server-Sent Events when stream is true.
  • /v1/models reports the fixed model surface expected by compatible clients.
  • File uploads are supported through JSON base64 or multipart form data.

Security notes​

  • Local 127.0.0.1 usage does not require an API key.
  • If api.host is 0.0.0.0 or ::, configure api.apiKey before startup.
  • Treat the API as agent access, not just model access: tools and workspace permissions still matter.

Troubleshooting​

  • If /v1/chat/completions fails, test nanoinfra agent -m "Hello!" first.
  • If remote clients cannot connect, check api.host, api.port, firewall, and API key configuration.
  • If sessions mix together, pass unique session_id values.