OpenAI-compatible API

The flagship drop-in surface. Anything written against the OpenAI Chat Completions API works by changing only the base URL.

Base URL

https://api.directinference.com/di/v1 — authenticate with Authorization: Bearer llm_live_…. See Authentication for keys and headers.

Chat completions

Standard messages, system prompts, and sampling parameters. The model id is treated as intent and echoed back unchanged.

Python

from openai import OpenAI

client = OpenAI(api_key="llm_live_...", base_url="https://api.directinference.com/di/v1")

resp = client.chat.completions.create(
    model="gpt-5.5-mini",
    messages=[
        {"role": "system", "content": "You are a concise assistant."},
        {"role": "user", "content": "Name three uses for a paperclip."},
    ],
    temperature=0.7,
    max_tokens=300,
)

print(resp.choices[0].message.content)

TypeScript

import OpenAI from "openai";

const client = new OpenAI({
  apiKey: "llm_live_...",
  baseURL: "https://api.directinference.com/di/v1",
});

const resp = await client.chat.completions.create({
  model: "gpt-5.5-mini",
  messages: [
    { role: "system", content: "You are a concise assistant." },
    { role: "user", content: "Name three uses for a paperclip." },
  ],
  temperature: 0.7,
  max_tokens: 300,
});

console.log(resp.choices[0].message.content);

curl

curl https://api.directinference.com/di/v1/chat/completions \
  -H "Authorization: Bearer llm_live_..." \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-5.5-mini",
    "messages": [
      { "role": "system", "content": "You are a concise assistant." },
      { "role": "user", "content": "Name three uses for a paperclip." }
    ],
    "temperature": 0.7,
    "max_tokens": 300
  }'

HTTPie

echo '{ "model": "gpt-5.5-mini", "messages": [{ "role": "system", "content": "You are a concise assistant." }, { "role": "user", "content": "Name three uses for a paperclip." }], "temperature": 0.7, "max_tokens": 300 }' \
  | https POST api.directinference.com/di/v1/chat/completions \
      Authorization:'Bearer llm_live_...' \
      Content-Type:application/json

Go

client := openai.NewClient(
  option.WithAPIKey("llm_live_..."),
  option.WithBaseURL("https://api.directinference.com/di/v1"),
)

resp, err := client.Chat.Completions.New(context.TODO(), openai.ChatCompletionNewParams{
  Model: "gpt-5.5-mini",
  Messages: []openai.ChatCompletionMessageParamUnion{
    openai.SystemMessage("You are a concise assistant."),
    openai.UserMessage("Name three uses for a paperclip."),
  },
  Temperature: openai.Float(0.7),
  MaxTokens:   openai.Int(300),
})
if err != nil {
  panic(err)
}

fmt.Println(resp.Choices[0].Message.Content)

Streaming

Set stream: true for token-by-token Server-Sent Events — data: frames terminated by a final data: [DONE] sentinel. Add stream_options: { include_usage: true } to receive a final chunk carrying token usage.

Python

stream = client.chat.completions.create(
    model="gpt-5.5-mini",
    messages=[{"role": "user", "content": "Stream a haiku about latency."}],
    stream=True,
)

for chunk in stream:
    delta = chunk.choices[0].delta.content
    if delta:
        print(delta, end="", flush=True)

TypeScript

const stream = await client.chat.completions.create({
  model: "gpt-5.5-mini",
  messages: [{ role: "user", content: "Stream a haiku about latency." }],
  stream: true,
});

for await (const chunk of stream) {
  process.stdout.write(chunk.choices[0]?.delta?.content ?? "");
}

curl

curl https://api.directinference.com/di/v1/chat/completions \
  -H "Authorization: Bearer llm_live_..." \
  -H "Content-Type: application/json" \
  -N \
  -d '{
    "model": "gpt-5.5-mini",
    "messages": [{ "role": "user", "content": "Stream a haiku about latency." }],
    "stream": true
  }'

HTTPie

# --stream keeps SSE frames flowing as they arrive
echo '{ "model": "gpt-5.5-mini", "messages": [{ "role": "user", "content": "Stream a haiku about latency." }], "stream": true }' \
  | https --stream POST api.directinference.com/di/v1/chat/completions \
      Authorization:'Bearer llm_live_...' \
      Content-Type:application/json

Go

stream := client.Chat.Completions.NewStreaming(context.TODO(), openai.ChatCompletionNewParams{
  Model: "gpt-5.5-mini",
  Messages: []openai.ChatCompletionMessageParamUnion{
    openai.UserMessage("Stream a haiku about latency."),
  },
})

for stream.Next() {
  chunk := stream.Current()
  if len(chunk.Choices) > 0 {
    fmt.Print(chunk.Choices[0].Delta.Content)
  }
}
if err := stream.Err(); err != nil {
  panic(err)
}

Reasoning output

Responses can include the reasoning produced while serving the request, in one canonical field. Non-streaming replies carry message.reasoning — a string, or null when there is none. Streams deliver it as delta.reasoning chunks that arrive before the first delta.content. Provider spellings such as reasoning_content are normalized away — you only ever read reasoning.

Python

stream = client.chat.completions.create(
    model="gpt-5.5-mini",
    messages=[{"role": "user", "content": "Why do mirrors flip left-right but not up-down?"}],
    stream=True,
)

for chunk in stream:
    delta = chunk.choices[0].delta
    if getattr(delta, "reasoning", None):
        print(delta.reasoning, end="", flush=True)   # thinking — arrives first
    if delta.content:
        print(delta.content, end="", flush=True)     # the answer

TypeScript

const stream = await client.chat.completions.create({
  model: "gpt-5.5-mini",
  messages: [{ role: "user", content: "Why do mirrors flip left-right but not up-down?" }],
  stream: true,
});

for await (const chunk of stream) {
  const delta = chunk.choices[0]?.delta as { reasoning?: string; content?: string };
  if (delta?.reasoning) process.stdout.write(delta.reasoning); // thinking — arrives first
  if (delta?.content) process.stdout.write(delta.content);     // the answer
}

curl

curl -N https://api.directinference.com/di/v1/chat/completions \
  -H "Authorization: Bearer llm_live_..." \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-5.5-mini",
    "messages": [{ "role": "user", "content": "Why do mirrors flip left-right but not up-down?" }],
    "stream": true
  }'

# reasoning deltas stream first, then the answer:
# data: {"choices":[{"delta":{"reasoning":"Light reflects ..."},"index":0}],...}
# data: {"choices":[{"delta":{"content":"A mirror swaps ..."},"index":0}],...}

HTTPie

echo '{ "model": "gpt-5.5-mini", "messages": [{ "role": "user", "content": "Why do mirrors flip left-right but not up-down?" }], "stream": true }' \
  | https --stream POST api.directinference.com/di/v1/chat/completions \
      Authorization:'Bearer llm_live_...' \
      Content-Type:application/json

# reasoning deltas stream first, then the answer:
# data: {"choices":[{"delta":{"reasoning":"Light reflects ..."},"index":0}],...}
# data: {"choices":[{"delta":{"content":"A mirror swaps ..."},"index":0}],...}

A pause is not a hang

A client that reads only delta.content sees silence while reasoning streams, then the whole answer at once — it looks stalled but isn’t. Either render delta.reasoning as a thinking indicator or ignore it; the answer is complete without it. Reasoning length follows effort — lower it to spend less time thinking.

Tools & function calling

Pass tools with JSON-Schema parameters; the response carries tool_calls to execute and feed back. Tool-shaped requests map to the code request type.

Python

tools = [{
    "type": "function",
    "function": {
        "name": "get_weather",
        "description": "Get the current weather for a city.",
        "parameters": {
            "type": "object",
            "properties": {"city": {"type": "string"}},
            "required": ["city"],
        },
    },
}]

resp = client.chat.completions.create(
    model="gpt-5.5-mini",
    messages=[{"role": "user", "content": "What is the weather in Paris?"}],
    tools=tools,
)

for call in resp.choices[0].message.tool_calls or []:
    print(call.function.name, call.function.arguments)

TypeScript

const tools = [{
  type: "function",
  function: {
    name: "get_weather",
    description: "Get the current weather for a city.",
    parameters: {
      type: "object",
      properties: { city: { type: "string" } },
      required: ["city"],
    },
  },
}] as const;

const resp = await client.chat.completions.create({
  model: "gpt-5.5-mini",
  messages: [{ role: "user", content: "What is the weather in Paris?" }],
  tools,
});

for (const call of resp.choices[0].message.tool_calls ?? []) {
  console.log(call.function.name, call.function.arguments);
}

curl

curl https://api.directinference.com/di/v1/chat/completions \
  -H "Authorization: Bearer llm_live_..." \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-5.5-mini",
    "messages": [{ "role": "user", "content": "What is the weather in Paris?" }],
    "tools": [{
      "type": "function",
      "function": {
        "name": "get_weather",
        "description": "Get the current weather for a city.",
        "parameters": {
          "type": "object",
          "properties": { "city": { "type": "string" } },
          "required": ["city"]
        }
      }
    }]
  }'

HTTPie

echo '{ "model": "gpt-5.5-mini", "messages": [{ "role": "user", "content": "What is the weather in Paris?" }], "tools": [{ "type": "function", "function": { "name": "get_weather", "description": "Get the current weather for a city.", "parameters": { "type": "object", "properties": { "city": { "type": "string" } }, "required": ["city"] } } }] }' \
  | https POST api.directinference.com/di/v1/chat/completions \
      Authorization:'Bearer llm_live_...' \
      Content-Type:application/json

Go

tools := []openai.ChatCompletionToolParam{{
  Function: openai.FunctionDefinitionParam{
    Name:        "get_weather",
    Description: openai.String("Get the current weather for a city."),
    Parameters: openai.FunctionParameters{
      "type": "object",
      "properties": map[string]any{
        "city": map[string]string{"type": "string"},
      },
      "required": []string{"city"},
    },
  },
}}

resp, err := client.Chat.Completions.New(context.TODO(), openai.ChatCompletionNewParams{
  Model:    "gpt-5.5-mini",
  Messages: []openai.ChatCompletionMessageParamUnion{
    openai.UserMessage("What is the weather in Paris?"),
  },
  Tools: tools,
})
if err != nil {
  panic(err)
}

for _, call := range resp.Choices[0].Message.ToolCalls {
  fmt.Println(call.Function.Name, call.Function.Arguments)
}

Vision

Send image content parts alongside text. Image input always uses the vision request type, regardless of the model id you send.

Python

resp = client.chat.completions.create(
    model="gpt-5.5-mini",
    messages=[{
        "role": "user",
        "content": [
            {"type": "text", "text": "What is in this image?"},
            {"type": "image_url", "image_url": {"url": "https://example.com/photo.jpg"}},
        ],
    }],
)

print(resp.choices[0].message.content)

TypeScript

const resp = await client.chat.completions.create({
  model: "gpt-5.5-mini",
  messages: [{
    role: "user",
    content: [
      { type: "text", text: "What is in this image?" },
      { type: "image_url", image_url: { url: "https://example.com/photo.jpg" } },
    ],
  }],
});

console.log(resp.choices[0].message.content);

curl

curl https://api.directinference.com/di/v1/chat/completions \
  -H "Authorization: Bearer llm_live_..." \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-5.5-mini",
    "messages": [{
      "role": "user",
      "content": [
        { "type": "text", "text": "What is in this image?" },
        { "type": "image_url", "image_url": { "url": "https://example.com/photo.jpg" } }
      ]
    }]
  }'

HTTPie

echo '{ "model": "gpt-5.5-mini", "messages": [{ "role": "user", "content": [{ "type": "text", "text": "What is in this image?" }, { "type": "image_url", "image_url": { "url": "https://example.com/photo.jpg" } }] }] }' \
  | https POST api.directinference.com/di/v1/chat/completions \
      Authorization:'Bearer llm_live_...' \
      Content-Type:application/json

Structured output

Use response_format with a JSON schema to constrain the reply. A response schema maps the call to the json request type.

Python

resp = client.chat.completions.create(
    model="gpt-5.5-mini",
    messages=[{"role": "user", "content": "Extract the name and age from: Ada is 36."}],
    response_format={
        "type": "json_schema",
        "json_schema": {
            "name": "person",
            "schema": {
                "type": "object",
                "properties": {"name": {"type": "string"}, "age": {"type": "integer"}},
                "required": ["name", "age"],
            },
        },
    },
)

print(resp.choices[0].message.content)   # strict JSON

TypeScript

const resp = await client.chat.completions.create({
  model: "gpt-5.5-mini",
  messages: [{ role: "user", content: "Extract the name and age from: Ada is 36." }],
  response_format: {
    type: "json_schema",
    json_schema: {
      name: "person",
      schema: {
        type: "object",
        properties: { name: { type: "string" }, age: { type: "integer" } },
        required: ["name", "age"],
      },
    },
  },
});

console.log(resp.choices[0].message.content); // strict JSON

curl

curl https://api.directinference.com/di/v1/chat/completions \
  -H "Authorization: Bearer llm_live_..." \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-5.5-mini",
    "messages": [{ "role": "user", "content": "Extract the name and age from: Ada is 36." }],
    "response_format": {
      "type": "json_schema",
      "json_schema": {
        "name": "person",
        "schema": {
          "type": "object",
          "properties": { "name": { "type": "string" }, "age": { "type": "integer" } },
          "required": ["name", "age"]
        }
      }
    }
  }'

HTTPie

echo '{ "model": "gpt-5.5-mini", "messages": [{ "role": "user", "content": "Extract the name and age from: Ada is 36." }], "response_format": { "type": "json_schema", "json_schema": { "name": "person", "schema": { "type": "object", "properties": { "name": { "type": "string" }, "age": { "type": "integer" } }, "required": ["name", "age"] } } } }' \
  | https POST api.directinference.com/di/v1/chat/completions \
      Authorization:'Bearer llm_live_...' \
      Content-Type:application/json

Handling follows the request shape

You never select a request type directly — it is inferred from the call. See Request types for the full classification order.

Supported parameters

Standard Chat Completions sampling and control fields pass straight through: temperature, top_p, max_tokens / max_completion_tokens, n, stop, presence_penalty, frequency_penalty, logit_bias, logprobs / top_logprobs, seed, user, response_format, tools / tool_choice / parallel_tool_calls, and stream / stream_options.

Unknown fields are dropped, not rejected

Vendor-specific or newer-SDK fields the surface doesn’t recognize are silently stripped before the request is served — they never cause a 400. Your existing client keeps working even as it adds parameters.

Parameter parity

Three points worth confirming, because integrations depend on them:

• max_completion_tokens — the newer OpenAI spelling — is recognized and forwarded, not dropped. Send either it or max_tokens.
• temperature is accepted across the full 0–2 range and is never rejected by DirectInference, including on requests routed to a reasoning model.
• stream_options: { include_usage: true } is honored: the final SSE chunk carries token usage, on every request type.

Listing models

GET /v1/models lists the DI Model three ways: di-fusion (the default), plus di-saver and di-max — the same model with the effort knob pinned, not separate models (every entry carries root: "di-fusion"). The pinned ids give model pickers, fast/smart model slots, and compare features real ids to hold onto: put di-saver in a background/fast slot and di-max in a quality slot. There is still no model to choose — and any other model id works on each request; it is treated as intent and echoed back unchanged.

di-fast was renamed to di-saver

“Saver” matches the cost-optimizing behavior (“fast” read as latency-only). di-fast is no longer listed but still resolves to di-saver with identical behavior, so existing integrations keep working — switch to di-saver when convenient.

Caching & response headers

Reuse a stable prompt prefix to cut cost and time-to-first-token: add a cache_control breakpoint to the cacheable content — see Prompt caching. Every response also reports the classified request type in the X-DI-Request-Type header (Response headers).