LibreChat

LibreChat is a self-hosted, multi-model chat UI. It speaks the OpenAI dialect through its custom endpoints, so adding DirectInference is two values in one config file: a base URL and an llm_live_… key. Your users then get DirectInference as a model in the endpoint selector — every message served by the model best suited to its shape, with nothing to choose.

This is the dedicated walkthrough. The one-paragraph version lives with the other settings-form tools in Custom providers; swapping an SDK in application code is Migrate to DirectInference.

The base URL is the whole integration

LibreChat custom endpoints are OpenAI-shaped. Point one at https://api.directinference.com/di/v1 with a DirectInference key and you are done — model ids, streaming, tools, vision, and titles all work unchanged.

What you need

Input	Required	Notes
A DirectInference API key (llm_live_…)	yes	Issue one on the API Keys page. No account? Signup is free at app.directinference.com; usage is pay-as-you-go.
A LibreChat instance	yes	Docker or from source — both read the same librechat.yaml.

Never paste the key into librechat.yaml directly. Put it in .env and reference it with ${...}, as below.

Add the custom endpoint

LibreChat reads custom endpoints from librechat.yaml. Add a DirectInference entry under endpoints.custom:

librechat.yaml

version: 1.3.13
cache: true

endpoints:
  custom:
    - name: "DirectInference"
      apiKey: "${DIRECTINFERENCE_API_KEY}"
      baseURL: "https://api.directinference.com/di/v1"
      models:
        default: ["di-fusion", "di-saver", "di-max"]
        fetch: true
      titleConvo: true
      titleModel: "di-saver"
      modelDisplayLabel: "DirectInference"

Then put the key in .env (same directory as librechat.yaml):

.env

DIRECTINFERENCE_API_KEY=llm_live_...

That is the entire integration. The rest of this page explains each line, the Docker mount, and how to verify it.

Field by field

Field	Value	Why
name	"DirectInference"	The label in LibreChat’s endpoint selector. Free text.
apiKey	"${DIRECTINFERENCE_API_KEY}"	Interpolated from .env at runtime, so the secret never lives in the config. Set it to "user_provided" instead to have each user paste their own key (see Multi-tenant).
baseURL	https://api.directinference.com/di/v1	The OpenAI-compatible DI surface. LibreChat appends /chat/completions and /models itself.
models.default	["di-fusion", "di-saver", "di-max"]	The fallback list shown if discovery is skipped or fails. These are the whole catalog — di-fusion, plus the same model with effort pinned low (di-saver) and high (di-max).
models.fetch	true	Populate the model list from GET /di/v1/models. With this on, the default list is just a fallback.
titleConvo	true	Auto-name conversations (a short extra completion per new chat).
titleModel	"di-saver"	Which catalog id to title with. di-saver keeps titling cheap; "current_model" reuses whatever the chat used.
modelDisplayLabel	"DirectInference"	The label shown next to the assistant’s messages.

Model ids are labels, not switches

Every id you send is accepted and echoed back — there is no model to pick. The id is read as intent, never as a selection, so use di-fusion for everything, or di-saver / di-max when you want a cheaper or stronger bias. See Request types for how DI decides, and Effort for the bias knob.

Run it with Docker

LibreChat’s Docker setup merges a docker-compose.override.yml over the shipped docker-compose.yml. Mount your librechat.yaml into the api service:

docker-compose.override.yml

services:
  api:
    volumes:
      - type: bind
        source: ./librechat.yaml
        target: /app/librechat.yaml

Keep librechat.yaml, docker-compose.override.yml, and .env together in the LibreChat project root, then restart so the config and env are re-read:

docker compose down && docker compose up -d

Running from source instead? librechat.yaml in the project root is picked up automatically (or set CONFIG_PATH=/path/to/librechat.yaml in .env); restart with npm run backend.

Capabilities

Everything LibreChat exposes for an OpenAI custom endpoint is served:

Feature	Status	Notes
Streaming	✓	Server-sent events, terminated with data: [DONE]. LibreChat streams every reply.
Tools / function calling	✓	Client-defined function tools on every request.
Vision / image input	✓	An image makes the request a vision request, served accordingly.
File / document input	✓	Served as the document request type.
Prompt caching	✓	See Prompt caching.
Title generation	✓	The extra titleConvo completion — covered above.

The only LibreChat features that don’t apply are the ones that aren’t chat completions — image generation, audio, and embeddings (for RAG / file search) — which DirectInference doesn’t serve. Point LibreChat’s fileConfig / RAG embeddings at a dedicated embeddings provider; DirectInference handles the chat endpoint. See Capabilities for the full served / not-served list.

Optional: headers, effort, and usage attribution

LibreChat lets a custom endpoint send static headers. Two DirectInference headers are worth setting:

librechat.yaml (endpoint excerpt)

      headers:
        X-Title: "LibreChat"
        X-DI-Effort: "medium"

• X-Title gives this LibreChat instance its own line in Usage & analytics (otherwise usage falls back to the API key’s name).
• X-DI-Effort (fast, minimal, low, medium, high, xhigh, max; default auto) biases everything LibreChat sends toward cost or quality. Details in Effort.

You don’t need dropParams

LibreChat’s dropParams exists for endpoints that reject the standard parameters it adds (temperature, top_p, presence_penalty, frequency_penalty, stop, user). DirectInference accepts the standard OpenAI parameters and silently strips anything it doesn’t recognise, so an extra field never becomes a 400 — leave dropParams unset unless you have another reason for it.

Multi-tenant: user-provided keys

For a shared instance where each person brings their own DirectInference account, set the key to user_provided instead of reading it from the environment — LibreChat then prompts each user for their own key and stores it per-user:

librechat.yaml (endpoint excerpt)

      apiKey: "user_provided"
      baseURL: "https://api.directinference.com/di/v1"

Per-user keys mean per-user spend caps and usage too — each key is billed and capped independently (see Spend & limits).

Confirm it’s live

After restarting, open LibreChat and pick DirectInference in the endpoint selector. Three checks, strongest first:

1. The model list is populated. The selector shows di-fusion, di-saver, and di-max — proof models.fetch reached GET /di/v1/models.

2. The header. Every served response carries X-DI-Request-Type (pro, code, vision, document, long, json, reason, or flash). It is the only customer-readable response header, and its presence proves the request was served by DirectInference. You can prove the base URL and key from a terminal before touching LibreChat:

   curl

   # -D - dumps response headers; the body is discarded
   curl -s -o /dev/null -D - https://api.directinference.com/di/v1/chat/completions \
     -H "Authorization: Bearer $DIRECTINFERENCE_API_KEY" \
     -H "Content-Type: application/json" \
     -d '{ "model": "di-fusion", "messages": [{ "role": "user", "content": "ping" }] }' \
     | grep -i x-di-request-type
   
   # x-di-request-type: flash

3. The dashboard. The chat shows up under Traces and Usage; with X-Title set, LibreChat is its own line in Applications.

If something looks wrong

Symptom	What’s happening — and the fix
The endpoint doesn’t appear in LibreChat	librechat.yaml wasn’t loaded. With Docker, confirm the docker-compose.override.yml bind mount and that you re-ran docker compose up -d; from source, check CONFIG_PATH and restart.
The model dropdown is empty	models.fetch couldn’t reach /models — the key or base URL is wrong. Run the curl above; it is ground truth that the pair works. The default list still backs the selector meanwhile.
A vendor-branded “incorrect API key” error	The base URL wasn’t applied — LibreChat tested against a default vendor. Save the config, restart, and re-test; the curl above proves the key itself.
404 on the first message	The base URL is one path segment off. It must be exactly https://api.directinference.com/di/v1 (LibreChat appends /chat/completions).
402	Balance exhausted or a spend cap reached — deliberate, won’t clear on retry. Top up or raise the cap at billing. See Spend & limits.
Per-model cost estimates look wrong	LibreChat prices by the label it sees, which DI only echoes. Real spend is in Usage & analytics.
Replies arrive but nothing shows in Traces	LibreChat is still on its previous endpoint, or the key is user_provided and unset for that user. Check a reply for X-DI-Request-Type; if absent, re-save the config and restart.

Error envelope shapes and limits are in Errors & limits.

Automate it

The whole integration is packaged as an agent skill — point a coding agent at your LibreChat checkout and ask it to “add DirectInference to LibreChat,” and it writes the librechat.yaml block, wires the .env var and Docker mount, and verifies the result with a bundled script. See AI coding agents for installing skills.