OpenAI Agents SDK with a custom endpoint

The Agents SDK takes a plain AsyncOpenAI client, so the base URL is one argument. Two defaults are worth knowing before you ship: tracing is on and exports to the OpenAI platform, and the SDK speaks the Responses API unless you tell it otherwise.

curl https://aicomp.ai/v1/models \
  -H "Authorization: Bearer sk-your-gateway-key"

What you need before you start

• A gateway API key and base URL (https://aicomp.ai/v1)
• pip install openai-agents
• A decision about tracing — see the next section, it is not optional in practice

The working configuration

import os
from openai import AsyncOpenAI
from agents import (
    Agent, Runner, OpenAIChatCompletionsModel, set_tracing_disabled,
)

# 1. Tracing is ON by default and uploads to the OpenAI platform.
#    With no platform key in the environment, every run logs a tracing
#    failure -- your model traffic goes to the gateway, your traces do not.
set_tracing_disabled(True)

client = AsyncOpenAI(
    base_url="https://aicomp.ai/v1",
    api_key=os.environ["GATEWAY_API_KEY"],
)

# 2. The SDK defaults to the Responses API. Pick the transport explicitly.
model = OpenAIChatCompletionsModel(model="gpt-5.6-luna", openai_client=client)

agent = Agent(name="Assistant", instructions="Answer precisely.", model=model)
result = Runner.run_sync(agent, "What is a rate limit?")
print(result.final_output)

Three things are deliberate in that block. The base URL sits on the client rather than on the agent, because the agent takes a model and the model takes the client. The transport is explicit, because the default is the Responses API. And tracing is switched off, because leaving it on sends traces somewhere other than where your requests go.

Default one: tracing exports to the OpenAI platform

Tracing is enabled by default and uses the same key as your model requests unless you say otherwise. Changing the base URL moves the model traffic and leaves the exporter where it was. With no platform key present, every run logs a tracing failure — a log line that looks like a routing problem and has nothing to do with routing.

import os
from openai import AsyncOpenAI
from agents import set_default_openai_client, set_tracing_export_api_key

client = AsyncOpenAI(base_url="https://aicomp.ai/v1", api_key=os.environ["GATEWAY_API_KEY"])

# Keep the gateway key for model traffic; do NOT reuse it as the tracing key.
# Without use_for_tracing=False the SDK would send the gateway key to the
# OpenAI platform's tracing endpoint.
set_default_openai_client(client, use_for_tracing=False)
set_tracing_export_api_key(os.environ["OPENAI_PLATFORM_KEY"])

# Or keep tracing on but keep payloads out of it:
#   Runner.run(agent, "hi", run_config=RunConfig(trace_include_sensitive_data=False))
#   export OPENAI_AGENTS_TRACE_INCLUDE_SENSITIVE_DATA=0

The use_for_tracing=False argument is the one to notice. Without it the SDK may send your gateway credential to the platform's tracing endpoint, which is a credential-handling problem rather than a cost one — and it costs two lines to avoid.

Default two: the Responses API

A plain agent targets POST /v1/responses. Endpoints that serve only chat completions return 404, and a 404 on a URL you copied correctly is a genuinely confusing thing to debug.

from agents import set_default_openai_api

# Applies when the address and key come from OPENAI_BASE_URL / OPENAI_API_KEY.
set_default_openai_api("chat_completions")

# Without this, or without an explicit OpenAIChatCompletionsModel, the SDK
# targets POST /v1/responses. An endpoint that only serves chat completions
# returns 404 -- which reads like a broken base URL and is not one.

What it costs

Rates checked 2026-09-26. Gateway rates move with upstream promotions — verify the current number in your dashboard before committing to a budget.

Rates checked 2026-09-26. Agent runs resend the accumulated history on every step, so input tokens dominate — the model column is worth more than any client-side change on this page.

How the bill is actually computed

Two numbers are metered separately and added: input tokens (everything you send — system prompt, conversation history, retrieved documents, tool results) and output tokens (what the model generates). Per-token, output is the expensive one — across the 216 models we track, the median output-to-input ratio is 4.0×, and the widest is 12.0×.

And yet on agent-shaped workloads the input side dominates the invoice. That is not a contradiction, it is arithmetic: an agent re-sends its entire context on every turn, so a turn that produces 800 output tokens may carry 60,000 input tokens. At a 4× unit price ratio, the input still accounts for roughly 95% of that turn's cost. Optimising output length on this workload is optimising the wrong end.

The practical reading: unit price decides, but token shape decides harder. A model that costs twice as much per token can still be cheaper end to end if it needs half as many turns to finish the task, and a prompt that carries 40k tokens of irrelevant history costs the same as one that carries 40k tokens of useful context. Before switching models, measure which side of the sum your workload actually lives on.

Agents SDK failures and what each one is actually telling you

When this is the wrong move

Cases where the Agents SDK is the wrong tool

Where the money actually goes, ranked

Cost advice tends to be a list of tricks with no ordering. These five are ordered by how much they typically move a real bill, and each one names the number that tells you whether it worked.

Concretely: one day at 1200k input / 200k output costs $0.240 on gpt-5.6-luna and $0.396 on deepseek-v4-flash — a difference of $0.156 per day, or $3.12 over a 20-day month.

The ordering is not universal — it depends on your token shape. On agent loops, context size outranks model choice; on bulk generation, model choice wins because there is no context to trim. Work out which half of the sum your workload sits on before spending effort on the wrong lever. Our output vs input pricing breakdown covers how to tell.

FAQ

Get API access