Agent frameworks

Standalone agent platforms — the self-hosted gateways and autonomous agent runtimes you install and point at a model — work with DirectInference the same way everything else does: give the framework DI’s base URL and an llm_live_… key, keep your model ids, and every request its agents make is served by DirectInference.

Three neighbouring pages — pick the one that fits

This page is for standalone agent platforms you run and configure (OpenClaw, and more below). Pointing an IDE or CLI coding assistant (Cursor, Claude Code) at DI is in AI coding agents. Wiring a code library (LangChain, LlamaIndex, …) is in Frameworks. Filling a generic settings-screen provider form is in Custom providers.

What every integration comes down to

However different two agent frameworks look, onboarding either one is the same four moves:

1. Pick the dialect the framework speaks — OpenAI, Anthropic, or Gemini — and use the matching base URL.
2. Set the base URL and key. A config file, an environment variable, or a provider form — whichever the framework offers.
3. Register the model ids. Use di-fusion (and di-saver / di-max for fast / smart slots). Frameworks that validate model names against a built-in list need the id added explicitly — di-fusion is on no vendor’s list.
4. Turn tools on. Enable the framework’s tool / function-calling switch so its agents can drive tool loops.

Base URL and key, per dialect

The framework speaks	Base URL	Credential
OpenAI-compatible	https://api.directinference.com/di/v1	Authorization: Bearer llm_live_…
Anthropic Messages	https://api.directinference.com/di — the client appends /v1/messages	x-api-key: llm_live_… + anthropic-version: 2023-06-01
Gemini	https://api.directinference.com/di — the client appends /v1beta/models/…	x-goog-api-key: / ?key= / Authorization: Bearer

The per-surface auth details are in Authentication; the full client-by-client base-URL table is in Migrate to DirectInference.

Model ids are labels, not a menu

There is one model. Send di-fusion, or keep whatever id the framework already uses — every id resolves, nothing 404s, and the id is echoed back unchanged. Map an agent’s fast / smart model slots to di-saver and di-max: the same model with effort pinned low and high. Never surface a model picker to the framework’s users — there is no model to choose, and request handling is automatic.

Two capability switches to set by hand

Tools and thinking won’t auto-detect from the id

Agent frameworks often decide whether a model supports tools or a thinking channel by matching its name against a built-in list. di-fusion isn’t on anyone’s list, so those switches can default off and the agent silently falls back to brittle text-parsing. Wherever a framework exposes function calling / tools or show thinking / reasoning toggles, turn them on — DirectInference serves all of them, because capability is a property of the request shape, never the model id.

The OpenAI Responses API is served too (statelessly, at /v1/responses), so agents built on it — including the OpenAI Agents SDK’s default surface — work unchanged. The one abstraction DirectInference does not serve is provider server-side tools (web search, code execution, URL context, and the Responses hosted tools): keep those call sites on the original provider. They fail with a clear error at request time, never a silent wrong answer. The full served / not-served list is in Capabilities.

Confirm an agent is running on DirectInference

The same three checks regardless of framework, strongest first:

1. Model echo — a response’s model is exactly the id the framework sent (modelVersion on the Gemini dialect).
2. The header — responses carry X-DI-Request-Type (pro, code, vision, document, long, json, reason, or flash); its presence is the proof DI served the request. See Response headers.
3. The dashboard — the call shows up in Traces and Usage. Set X-Title on the provider and the framework gets its own line in Applications.

Supported agent frameworks

Worked, end-to-end setups. More are added over time — any OpenAI-, Anthropic-, or Gemini-compatible agent platform already works with the four moves above even before it appears here.

OpenClaw

OpenClaw is a self-hosted gateway that connects chat surfaces (Slack, Telegram, Discord, WhatsApp, …) to a single embedded agent runtime. It is model-agnostic, so DirectInference plugs in as a custom openai-completions provider in ~/.openclaw/openclaw.json (JSON5).

{
  models: {
    providers: {
      directinference: {
        baseUrl: "https://api.directinference.com/di/v1",
        apiKey: "${DIRECTINFERENCE_API_KEY}",   // env substitution — never inline a key
        api: "openai-completions",
        models: [
          { id: "di-fusion", name: "DirectInference",       input: ["text", "image"] },
          { id: "di-saver",  name: "DirectInference Saver", input: ["text", "image"] },
          { id: "di-max",    name: "DirectInference Max",   input: ["text", "image"] },
        ],
      },
    },
  },
  agents: {
    defaults: {
      // string form sets the primary; object form adds fallbacks:
      // model: { primary: "directinference/di-fusion", fallbacks: ["directinference/di-saver"] }
      model: "directinference/di-fusion",
    },
  },
}

Then provide the key and point the agent at DI:

export DIRECTINFERENCE_API_KEY="llm_live_..."   # provider id → <PROVIDER>_API_KEY
openclaw models list          # confirms OpenClaw registered directinference/di-fusion from your config
openclaw models set directinference/di-fusion

The provider-scoped key env var is derived from the provider id (directinference → DIRECTINFERENCE_API_KEY, or the top-priority OPENCLAW_LIVE_DIRECTINFERENCE_KEY). Every channel wired to the gateway now runs its agent on DirectInference.

Things specific to OpenClaw

• Keep the /v1. The openai-completions adapter expects a base that already exposes /v1/chat/completions and /v1/models — use …/di/v1, do not strip it (unlike Anthropic-style clients).
• Register the ids under the provider’s models[] or they’re rejected. OpenClaw routes only to models it knows, and a custom id like di-fusion must be declared in the provider’s models[] (above). The separate agents.defaults.models map is an allowlist when set — it doesn’t register a runtime model on its own, so it’s not a substitute.
• Tools are automatic once a model is registered; the tools.profile (e.g. "coding") controls which built-in tools an agent may use.

Prefer to have an agent do it? The installable DirectInference skill includes an OpenClaw flow — ask your coding agent to “set up OpenClaw on DirectInference” and it writes the provider block and verifies the connection for you.

Hermes Agent

Hermes Agent (Nous Research) is a self-hosted, self-improving autonomous agent — persistent memory, learned skills, terminal / file / web tools, MCP, and scheduled work. It is model-agnostic and OpenAI-compatible, so DirectInference goes in as a custom provider in ~/.hermes/config.yaml. Use the named custom_providers form with key_env so the key stays out of the file:

custom_providers:
  - name: directinference
    base_url: https://api.directinference.com/di/v1   # keep the /v1
    key_env: DIRECTINFERENCE_API_KEY                  # env var that holds the key

model:
  default: di-fusion                 # any id; sent straight to DI, echoed back
  provider: custom:directinference   # use the named provider above
  supports_vision: true              # DI routes image requests to a vision model

Put the key in ~/.hermes/.env (or export it) and confirm with a scripted one-shot:

export DIRECTINFERENCE_API_KEY="llm_live_..."
hermes -z "Reply with exactly: PONG"   # one-shot: prompt in, final answer text out

Things specific to Hermes

• No server-side tool flags. The --jinja / --tool-call-parser flags Hermes documents are for raw self-hosted inference servers; DI is a managed endpoint that returns native OpenAI tool calls, so the agent’s tool loop works without them.
• The model id is passed through, not validated — provider: custom sends default (or model) straight to DI, so di-fusion / di-saver / di-max work unchanged.
• Isolate with HERMES_HOME (default ~/.hermes), not a --config flag — point it at a throwaway dir to relocate the whole profile for a clean test.

NemoClaw

NemoClaw is NVIDIA’s open-source secure runtime: it runs agents like OpenClaw and Hermes (both above) inside an OpenShell sandbox with policy-based egress control and managed inference. You wire DirectInference in at the inference-provider level — the gateway holds the key and forwards agent traffic to DI, so the credential never enters the sandbox.

openshell provider create \
  --name directinference \
  --type openai \
  --credential OPENAI_API_KEY="$LLM_API_KEY" \
  --config OPENAI_BASE_URL=https://api.directinference.com/di/v1
nemoclaw inference set   # select the directinference provider + a model id (e.g. di-fusion)

Things specific to NemoClaw

• Include the /v1 — OpenShell does not auto-append it; every documented OpenAI-compatible base URL ends in /v1, so use …/di/v1.
• Key via --credential, base URL via --config — the secret goes through --credential (stored host-side, never exposed to the sandbox); a same-named exported env var takes precedence over the stored file.
• Model id is passed through, not validated for a compatible endpoint, so di-fusion / di-saver / di-max work. NemoClaw validates a compatible endpoint by sending a real inference request, not by reading /v1/models.
• Egress: if outbound is locked down, allow api.directinference.com.

Odysseus

Odysseus is a self-hosted, privacy-first AI workspace (FastAPI) — chat, agents, deep research, documents, notes, and tools — that connects to any OpenAI-compatible API. Run it, then add DirectInference as a provider in the app:

git clone https://github.com/pewdiepie-archdaemon/odysseus.git
cd odysseus && cp .env.example .env && docker compose up -d --build
# http://localhost:7000 — first admin password: docker compose logs odysseus

1. Settings → Providers → add a custom OpenAI-compatible provider.
2. Base URL — https://api.directinference.com/di/v1 (keep the /v1).
3. API key — llm_live_....
4. Model — Odysseus probes /di/v1/models to fill the picker; choose di-fusion, or type any id (di-saver / di-max pin effort).

Things specific to Odysseus

• The custom base URL is set in the UI, not via env — .env carries an OPENAI_API_KEY but no generic OPENAI_BASE_URL.
• Model ids aren’t validated — di-fusion works whether probed or pinned, and is echoed back unchanged.
• Scripting: the POST /api/v1/chat webhook takes base_url + api_key inline (with a chat-scoped ody_ token from Settings → Integrations), so a single request can be routed through DI without stored provider config.