# LibreChat

[LibreChat](https://www.librechat.ai/) is a self-hosted, multi-model chat UI. It speaks the OpenAI dialect through its **custom endpoints**, so adding DirectInference is two values in one config file: a base URL and an `llm_live_…` key. Your users then get DirectInference as a model in the endpoint selector — every message served by the model best suited to its shape, with nothing to choose.

This is the dedicated walkthrough. The one-paragraph version lives with the other settings-form tools in [Custom providers](https://docs.directinference.com/custom-providers/); swapping an SDK in application code is [Migrate to DirectInference](https://docs.directinference.com/migrate/).

:::note[The base URL is the whole integration]
LibreChat custom endpoints are OpenAI-shaped. Point one at `https://api.directinference.com/di/v1` with a DirectInference key and you are done — model ids, streaming, tools, vision, and titles all work unchanged.
:::

## What you need

| Input | Required | Notes |
| --- | --- | --- |
| A DirectInference API key (`llm_live_…`) | yes | Issue one on the [API Keys](https://app.directinference.com/api-keys) page. No account? Signup is free at [app.directinference.com](https://app.directinference.com); usage is pay-as-you-go. |
| A LibreChat instance | yes | Docker or from source — both read the same `librechat.yaml`. |

Never paste the key into `librechat.yaml` directly. Put it in `.env` and reference it with `${...}`, as below.

## Add the custom endpoint

LibreChat reads custom endpoints from `librechat.yaml`. Add a `DirectInference` entry under `endpoints.custom`:

```yaml title="librechat.yaml"
version: 1.3.13
cache: true

endpoints:
  custom:
    - name: "DirectInference"
      apiKey: "${DIRECTINFERENCE_API_KEY}"
      baseURL: "https://api.directinference.com/di/v1"
      models:
        default: ["di-fusion", "di-saver", "di-max"]
        fetch: true
      titleConvo: true
      titleModel: "di-saver"
      modelDisplayLabel: "DirectInference"
```

Then put the key in `.env` (same directory as `librechat.yaml`):

```bash title=".env"
DIRECTINFERENCE_API_KEY=llm_live_...
```

That is the entire integration. The rest of this page explains each line, the Docker mount, and how to verify it.

### Field by field

| Field | Value | Why |
| --- | --- | --- |
| `name` | `"DirectInference"` | The label in LibreChat's endpoint selector. Free text. |
| `apiKey` | `"${DIRECTINFERENCE_API_KEY}"` | Interpolated from `.env` at runtime, so the secret never lives in the config. Set it to `"user_provided"` instead to have each user paste their own key (see [Multi-tenant](#multi-tenant-user-provided-keys)). |
| `baseURL` | `https://api.directinference.com/di/v1` | The OpenAI-compatible DI surface. LibreChat appends `/chat/completions` and `/models` itself. |
| `models.default` | `["di-fusion", "di-saver", "di-max"]` | The fallback list shown if discovery is skipped or fails. These are the whole catalog — `di-fusion`, plus the same model with effort pinned low (`di-saver`) and high (`di-max`). |
| `models.fetch` | `true` | Populate the model list from `GET /di/v1/models`. With this on, the `default` list is just a fallback. |
| `titleConvo` | `true` | Auto-name conversations (a short extra completion per new chat). |
| `titleModel` | `"di-saver"` | Which catalog id to title with. `di-saver` keeps titling cheap; `"current_model"` reuses whatever the chat used. |
| `modelDisplayLabel` | `"DirectInference"` | The label shown next to the assistant's messages. |

:::tip[Model ids are labels, not switches]
Every id you send is accepted and echoed back — there is no model to pick. The id is read as intent, never as a selection, so use `di-fusion` for everything, or `di-saver` / `di-max` when you want a cheaper or stronger bias. See [Request types](https://docs.directinference.com/request-types/) for how DI decides, and [Effort](https://docs.directinference.com/effort/) for the bias knob.
:::

## Run it with Docker

LibreChat's Docker setup merges a `docker-compose.override.yml` over the shipped `docker-compose.yml`. Mount your `librechat.yaml` into the `api` service:

```yaml title="docker-compose.override.yml"
services:
  api:
    volumes:
      - type: bind
        source: ./librechat.yaml
        target: /app/librechat.yaml
```

Keep `librechat.yaml`, `docker-compose.override.yml`, and `.env` together in the LibreChat project root, then restart so the config and env are re-read:

```bash
docker compose down && docker compose up -d
```

Running from source instead? `librechat.yaml` in the project root is picked up automatically (or set `CONFIG_PATH=/path/to/librechat.yaml` in `.env`); restart with `npm run backend`.

## Capabilities

Everything LibreChat exposes for an OpenAI custom endpoint is served:

| Feature | Status | Notes |
| --- | --- | --- |
| Streaming | ✓ | Server-sent events, terminated with `data: [DONE]`. LibreChat streams every reply. |
| Tools / function calling | ✓ | Client-defined function tools on every request. |
| Vision / image input | ✓ | An image makes the request a `vision` request, served accordingly. |
| File / document input | ✓ | Served as the `document` request type. |
| Prompt caching | ✓ | See [Prompt caching](https://docs.directinference.com/caching/). |
| Title generation | ✓ | The extra `titleConvo` completion — covered above. |

The only LibreChat features that don't apply are the ones that aren't chat completions — image generation, audio, and embeddings (for RAG / file search) — which DirectInference doesn't serve. Point LibreChat's `fileConfig` / RAG embeddings at a dedicated embeddings provider; DirectInference handles the chat endpoint. See [Capabilities](https://docs.directinference.com/capabilities/) for the full served / not-served list.

## Optional: headers, effort, and usage attribution

LibreChat lets a custom endpoint send static `headers`. Two DirectInference headers are worth setting:

```yaml title="librechat.yaml (endpoint excerpt)"
      headers:
        X-Title: "LibreChat"
        X-DI-Effort: "medium"
```

- `X-Title` gives this LibreChat instance its own line in [Usage & analytics](https://docs.directinference.com/usage/) (otherwise usage falls back to the API key's name).
- `X-DI-Effort` (`fast`, `minimal`, `low`, `medium`, `high`, `xhigh`, `max`; default `auto`) biases everything LibreChat sends toward cost or quality. Details in [Effort](https://docs.directinference.com/effort/).

:::note[You don't need `dropParams`]
LibreChat's `dropParams` exists for endpoints that reject the standard parameters it adds (`temperature`, `top_p`, `presence_penalty`, `frequency_penalty`, `stop`, `user`). DirectInference accepts the standard OpenAI parameters and silently strips anything it doesn't recognise, so an extra field never becomes a `400` — leave `dropParams` unset unless you have another reason for it.
:::

## Multi-tenant: user-provided keys

For a shared instance where each person brings their own DirectInference account, set the key to `user_provided` instead of reading it from the environment — LibreChat then prompts each user for their own key and stores it per-user:

```yaml title="librechat.yaml (endpoint excerpt)"
      apiKey: "user_provided"
      baseURL: "https://api.directinference.com/di/v1"
```

Per-user keys mean per-user spend caps and usage too — each key is billed and capped independently (see [Spend & limits](https://docs.directinference.com/spend/)).

## Confirm it's live

After restarting, open LibreChat and pick **DirectInference** in the endpoint selector. Three checks, strongest first:

1. **The model list is populated.** The selector shows `di-fusion`, `di-saver`, and `di-max` — proof `models.fetch` reached `GET /di/v1/models`.
2. **The header.** Every served response carries `X-DI-Request-Type` (`pro`, `code`, `vision`, `document`, `long`, `json`, `reason`, or `flash`). It is the only customer-readable response header, and its presence proves the request was served by DirectInference. You can prove the base URL and key from a terminal before touching LibreChat:

   ```bash
   # -D - dumps response headers; the body is discarded
   curl -s -o /dev/null -D - https://api.directinference.com/di/v1/chat/completions \
     -H "Authorization: Bearer $DIRECTINFERENCE_API_KEY" \
     -H "Content-Type: application/json" \
     -d '{ "model": "di-fusion", "messages": [{ "role": "user", "content": "ping" }] }' \
     | grep -i x-di-request-type

   # x-di-request-type: flash
   ```

3. **The dashboard.** The chat shows up under [Traces](https://app.directinference.com/traces) and [Usage](https://docs.directinference.com/usage/); with `X-Title` set, LibreChat is its own line in [Applications](https://app.directinference.com/applications).

## If something looks wrong

| Symptom | What's happening — and the fix |
| --- | --- |
| The endpoint doesn't appear in LibreChat | `librechat.yaml` wasn't loaded. With Docker, confirm the `docker-compose.override.yml` bind mount and that you re-ran `docker compose up -d`; from source, check `CONFIG_PATH` and restart. |
| The model dropdown is empty | `models.fetch` couldn't reach `/models` — the key or base URL is wrong. Run the curl above; it is ground truth that the pair works. The `default` list still backs the selector meanwhile. |
| A vendor-branded "incorrect API key" error | The base URL wasn't applied — LibreChat tested against a default vendor. Save the config, restart, and re-test; the curl above proves the key itself. |
| `404` on the first message | The base URL is one path segment off. It must be exactly `https://api.directinference.com/di/v1` (LibreChat appends `/chat/completions`). |
| `402` | Balance exhausted or a spend cap reached — deliberate, won't clear on retry. Top up or raise the cap at [billing](https://app.directinference.com/billing). See [Spend & limits](https://docs.directinference.com/spend/). |
| Per-model cost estimates look wrong | LibreChat prices by the label it sees, which DI only echoes. Real spend is in [Usage & analytics](https://docs.directinference.com/usage/). |
| Replies arrive but nothing shows in Traces | LibreChat is still on its previous endpoint, or the key is `user_provided` and unset for that user. Check a reply for `X-DI-Request-Type`; if absent, re-save the config and restart. |

Error envelope shapes and limits are in [Errors & limits](https://docs.directinference.com/errors/).

## Automate it

The whole integration is packaged as an agent skill — point a coding agent at your LibreChat checkout and ask it to "add DirectInference to LibreChat," and it writes the `librechat.yaml` block, wires the `.env` var and Docker mount, and verifies the result with a bundled script. See [AI coding agents](https://docs.directinference.com/agents/) for installing skills.