Frameworks

Most frameworks wrap one of the OpenAI, Anthropic, or Gemini SDKs. Point that underlying client at DirectInference — base URL plus an llm_live_… key — and the framework works unchanged. Model ids stay as they are; every id is accepted and echoed back.

This page has the drop-in for the common Python frameworks, then a map of how each framework’s higher-level abstractions land on the wire — so you can predict what is served before you run it.

The base URL is the whole change

The per-SDK base URLs and the auth header each surface expects are in Migrate to DirectInference. Pointing a CLI or coding agent (Cursor, Claude Code) at DI is in AI coding agents; filling a settings-screen provider form is in Custom providers.

LangChain

Set base_url and api_key on the chat model — the OpenAI or Anthropic class, whichever your code already uses. The model string is unchanged.

OpenAI

from langchain_openai import ChatOpenAI

llm = ChatOpenAI(
    model="gpt-5.5-mini",
    base_url="https://api.directinference.com/di/v1",
    api_key="llm_live_...",
)

Anthropic

from langchain_anthropic import ChatAnthropic

llm = ChatAnthropic(
    model="claude-sonnet-4-6",
    base_url="https://api.directinference.com/di",   # the client appends /v1/messages
    api_key="llm_live_...",
)

In JavaScript: new ChatOpenAI({ apiKey, configuration: { baseURL: "https://api.directinference.com/di/v1" } }), or new ChatAnthropic({ apiKey, anthropicApiUrl: "https://api.directinference.com/di" }).

LlamaIndex

Use OpenAILike for the OpenAI-compatible surface — unlike the stock OpenAI class it does not validate the model name against a fixed list, so any id (including di-fusion) is accepted.

from llama_index.llms.openai_like import OpenAILike

llm = OpenAILike(
    model="di-fusion",
    api_base="https://api.directinference.com/di/v1",
    api_key="llm_live_...",
    is_chat_model=True,
)

How framework abstractions map to the wire

A framework’s high-level feature compiles down to a request shape. DirectInference serves the shapes that are part of the OpenAI / Anthropic / Gemini APIs — including the OpenAI Responses API (statelessly, at /v1/responses). The one line it does not serve is provider server-side tools (web search, code execution, URL context, and the Responses hosted tools). The mapping below lets you tell, per feature, which side of that line you are on — without reading framework source.

Framework	Abstraction	Compiles to	Served?
PydanticAI	output_type= structured output	Forced tool call (final_result tool + forced tool_choice)	✓ Served
PydanticAI	builtin_tools=[WebSearchTool()]	Provider server-side tool (e.g. Gemini googleSearch)	✗ Keep on the original provider
PydanticAI	provider:model string	The matching DI surface (openai: / anthropic: / google-gla:)	✓ Served
LangChain	ChatOpenAI / ChatAnthropic	Chat Completions / Messages	✓ Served
LangChain	grounding / web-search tools	Provider server-side tool	✗ Keep on the original provider
LlamaIndex	OpenAILike, Anthropic	Chat Completions / Messages	✓ Served
Vercel AI SDK (≥5)	provider("id") (default)	OpenAI Responses API	✓ Served (stateless) — or provider.chat("id") for Chat Completions
LiteLLM	openai/…, anthropic/…	Chat Completions / Messages	✓ Served
Instructor	response_model=	Forced tool call / JSON mode	✓ Served

One thing to watch

The common abstraction that lands on a shape DirectInference doesn’t serve is provider server-side tools (framework builtins like PydanticAI WebSearchTool or LangChain grounding — keep those call sites on the original provider). The Vercel AI SDK ≥5 default provider("id") uses the Responses API, which is now served statelessly — it works as a drop-in; provider.chat("id") selects Chat Completions if you prefer. Unsupported shapes fail at request time with a descriptive error, never a silent wrong answer. The full list of served and not-served shapes is in Capabilities.

Structured output works everywhere

Every framework above reaches structured output the same way underneath — a forced tool call or a response schema — and DirectInference guarantees the result is usable: a tool-bearing request never returns a 200 with unparseable arguments. That is what makes the output_type / response_model patterns reliable on DI inside an agent loop; the guarantee is spelled out in Request types.