Open WebUI

Open WebUI is a self-hosted, open-source chat interface for LLMs. It speaks the OpenAI wire format, so DirectInference drops in as a single connection — install Open WebUI, add the DI base URL and an llm_live_… key, and every message is served by the model best suited to it.

1 · Install Open WebUI

Two common ways to run it. Docker is the recommended path; pip is handy for a quick local try. Pick one:

Docker

docker run -d -p 3000:8080 \
  -v open-webui:/app/backend/data \
  --name open-webui \
  ghcr.io/open-webui/open-webui:main

Open http://localhost:3000 and create the first account — it becomes the admin (you need an admin to reach the connection settings below).

pip (Python)

pip install open-webui
open-webui serve

Open http://localhost:8080 and create the first account — it becomes the admin (you need an admin to reach the connection settings below).

Other install methods

Docker Compose, Kubernetes/Helm, and the bundled-Ollama image are in the Open WebUI docs. The two above are enough to reach a running UI.

2 · Connect DirectInference

Create an llm_live_… key on the API Keys page first — the same key works on every DI surface (Authentication). Then wire it in either at startup or from the Admin UI.

Option A — wire it at startup

Set the OpenAI connection variables when you launch the container, and Open WebUI starts already pointed at DirectInference. The two -e lines are the only addition to the install command above:

docker run -d -p 3000:8080 \
  -v open-webui:/app/backend/data \
  -e OPENAI_API_BASE_URL="https://api.directinference.com/di/v1" \
  -e OPENAI_API_KEY="llm_live_..." \
  --name open-webui \
  ghcr.io/open-webui/open-webui:main

Running from pip instead? Export the same two variables before open-webui serve.

These two variables are read once, on first launch

OPENAI_API_BASE_URL and OPENAI_API_KEY are persistent config: Open WebUI stores them in its database the first time the container starts and ignores later changes to the env vars. To change the connection afterwards, edit it in the Admin UI (Option B), or start the container with ENABLE_PERSISTENT_CONFIG=False to make the env vars authoritative on every restart.

Option B — add it in the Admin UI

Already running, or prefer to click through it? In ⚙️ Admin Settings → Connections → OpenAI → Manage, click ➕ Add New Connection and enter:

• URL — https://api.directinference.com/di/v1
• API Key — llm_live_...

Save. (Older builds put this under Connections → OpenAI API — the two fields are the same.)

3 · Pick a model and chat

Open a new chat and choose di-fusion from the model selector at the top. Open WebUI lists the catalog from /models automatically, so you’ll see three entries:

Model id	What it is
di-fusion	The DI model. Request handling is automatic — start here.
di-saver	The same model with effort pinned low — a cheaper default.
di-max	The same model with effort pinned high — a more thorough default.

That’s the whole catalog: one model, with di-saver/di-max pinning effort. There’s no model to choose — whichever id you pick is read as intent and echoed back unchanged.

4 · Confirm it’s live

• The model echo. The reply’s model is exactly the id you picked; nothing rewrites it.
• The dashboard. The chat shows up under Traces and Usage & analytics.
• Per-app attribution. If your build’s connection editor exposes a custom-headers field, set X-Title to a name for this install and it gets its own line in Applications; otherwise usage attributes to the API key’s name.

Troubleshooting

Symptom	What’s happening — and the fix
The model selector is empty	The connection didn’t save, or the URL/key is off. Re-open Connections → OpenAI → Manage and confirm the URL is https://api.directinference.com/di/v1 and the key is your llm_live_….
You changed the env var but the old connection is still used	OPENAI_API_BASE_URL / OPENAI_API_KEY are read once on first launch (see Option A). Edit the connection in the Admin UI, or set ENABLE_PERSISTENT_CONFIG=False.
The first message returns a 404	The base URL is one path segment off. It must end in /di/v1 — Open WebUI appends /chat/completions itself.
Per-model cost in the UI looks wrong	Open WebUI prices by the label it sees, which DI only echoes. Actual spend is in Usage & analytics and Spend & limits.

This page is the worked version of the general Custom providers guide; error envelope shapes and rate limits are in Errors & limits.