# Frameworks

Most frameworks wrap one of the OpenAI, Anthropic, or Gemini SDKs. Point that underlying client at DirectInference — base URL plus an `llm_live_…` key — and the framework works unchanged. Model ids stay as they are; every id is accepted and echoed back.

This page has the drop-in for the common Python frameworks, then a map of how each framework's higher-level abstractions land on the wire — so you can predict what is served before you run it.

:::note[The base URL is the whole change]
The per-SDK base URLs and the auth header each surface expects are in [Migrate to DirectInference](https://docs.directinference.com/migrate/). Pointing a CLI or coding agent (Cursor, Claude Code) at DI is in [AI coding agents](https://docs.directinference.com/agents/); filling a settings-screen provider form is in [Custom providers](https://docs.directinference.com/custom-providers/).
:::

## LangChain

Set `base_url` and `api_key` on the chat model — the OpenAI or Anthropic class, whichever your code already uses. The `model` string is unchanged.

```python
from langchain_openai import ChatOpenAI

llm = ChatOpenAI(
    model="gpt-5.5-mini",
    base_url="https://api.directinference.com/di/v1",
    api_key="llm_live_...",
)
```

```python
from langchain_anthropic import ChatAnthropic

llm = ChatAnthropic(
    model="claude-sonnet-4-6",
    base_url="https://api.directinference.com/di",   # the client appends /v1/messages
    api_key="llm_live_...",
)
```

In JavaScript: `new ChatOpenAI({ apiKey, configuration: { baseURL: "https://api.directinference.com/di/v1" } })`, or `new ChatAnthropic({ apiKey, anthropicApiUrl: "https://api.directinference.com/di" })`.

## LlamaIndex

Use `OpenAILike` for the OpenAI-compatible surface — unlike the stock `OpenAI` class it does not validate the model name against a fixed list, so any id (including `di-fusion`) is accepted.

```python
from llama_index.llms.openai_like import OpenAILike

llm = OpenAILike(
    model="di-fusion",
    api_base="https://api.directinference.com/di/v1",
    api_key="llm_live_...",
    is_chat_model=True,
)
```

## How framework abstractions map to the wire

A framework's high-level feature compiles down to a request shape. DirectInference serves the shapes that are part of the OpenAI / Anthropic / Gemini APIs — including the OpenAI **Responses API** (statelessly, at `/v1/responses`). The one line it does not serve is provider **server-side tools** (web search, code execution, URL context, and the Responses hosted tools). The mapping below lets you tell, per feature, which side of that line you are on — without reading framework source.

| Framework | Abstraction | Compiles to | Served? |
| --- | --- | --- | --- |
| PydanticAI | `output_type=` structured output | Forced tool call (`final_result` tool + forced `tool_choice`) | ✓ Served |
| PydanticAI | `builtin_tools=[WebSearchTool()]` | Provider server-side tool (e.g. Gemini `googleSearch`) | ✗ Keep on the original provider |
| PydanticAI | `provider:model` string | The matching DI surface (`openai:` / `anthropic:` / `google-gla:`) | ✓ Served |
| LangChain | `ChatOpenAI` / `ChatAnthropic` | Chat Completions / Messages | ✓ Served |
| LangChain | grounding / web-search tools | Provider server-side tool | ✗ Keep on the original provider |
| LlamaIndex | `OpenAILike`, `Anthropic` | Chat Completions / Messages | ✓ Served |
| Vercel AI SDK (≥5) | `provider("id")` (default) | OpenAI **Responses API** | ✓ Served (stateless) — or `provider.chat("id")` for Chat Completions |
| LiteLLM | `openai/…`, `anthropic/…` | Chat Completions / Messages | ✓ Served |
| Instructor | `response_model=` | Forced tool call / JSON mode | ✓ Served |

:::caution[One thing to watch]
The common abstraction that lands on a shape DirectInference doesn't serve is **provider server-side tools** (framework builtins like PydanticAI `WebSearchTool` or LangChain grounding — keep those call sites on the original provider). The **Vercel AI SDK ≥5 default** `provider("id")` uses the Responses API, which is now served statelessly — it works as a drop-in; `provider.chat("id")` selects Chat Completions if you prefer. Unsupported shapes fail at request time with a descriptive error, never a silent wrong answer. The full list of served and not-served shapes is in [Capabilities](https://docs.directinference.com/capabilities/).
:::

## Structured output works everywhere

Every framework above reaches structured output the same way underneath — a forced tool call or a response schema — and DirectInference guarantees the result is usable: a tool-bearing request never returns a `200` with unparseable arguments. That is what makes the `output_type` / `response_model` patterns reliable on DI inside an agent loop; the guarantee is spelled out in [Request types](https://docs.directinference.com/request-types/).