Pydantic AI with a custom OpenAI-compatible endpoint
Pydantic AI reaches any OpenAI-compatible endpoint through its provider, but the default model prefix does not target the route you expect — it posts to the Responses API, which most compatible endpoints do not serve. Pick the chat model class and the same gateway works everywhere else in this cluster.
/v1 and a key from that endpoint. The setting below is the only thing that changes — request and response handling stay identical. Rates checked 2026-09-20.The route is the whole story
Two model classes exist, and they land on different paths. OpenAIChatModel posts to /v1/chat/completions — the route every OpenAI-compatible provider implements. OpenAIResponsesModel posts to /v1/responses, which is newer and far less widely supported. In current Pydantic AI the bare openai: prefix resolves to the Responses model, so the short, obvious way to write an agent is also the one that fails against almost every gateway.
from pydantic_ai import Agent
from pydantic_ai.models.openai import OpenAIChatModel
from pydantic_ai.providers.openai import OpenAIProvider
model = OpenAIChatModel(
"claude-sonnet-5",
provider=OpenAIProvider(
base_url="https://aicomp.ai/v1",
api_key="sk-...",
),
)
agent = Agent(model)Written that way, the endpoint is visible in the source, the route is unambiguous, and the same provider object can be reused across several agents. The string-prefix equivalent is openai-chat:claude-sonnet-5, which is shorter but hides both the route and the endpoint.
Installing and configuring
Provider support is an optional group, so the client is not there by default:
pip install "pydantic-ai-slim[openai]"Environment variables are honoured if you prefer them — OPENAI_BASE_URL and OPENAI_API_KEY — and they are a reasonable choice for a deployed service where the endpoint differs per environment. The trade-off is visibility: a base URL named in code is reviewable, one inherited from the environment is not, and "which endpoint did that job use" is exactly the question you ask when a bill surprises you.
Retries compound, and you pay for all of them
This is the part that surprises people on a metered endpoint. The underlying client retries on its own — a couple of attempts by default, on 408, 429, 5xx, timeouts and connection errors — and the agent layer has its own retry budget above that. The two multiply, so one logical call can hit the network several times over, and every one of those attempts is billable.
from openai import AsyncOpenAI
from pydantic_ai import Agent
from pydantic_ai.models.openai import OpenAIChatModel
from pydantic_ai.providers.openai import OpenAIProvider
# Keep the retry policy in one place: the client stops retrying,
# so retries are decided by the layer you control.
client = AsyncOpenAI(base_url="https://aicomp.ai/v1", api_key="sk-...", max_retries=0)
model = OpenAIChatModel("claude-sonnet-5", provider=OpenAIProvider(openai_client=client))
agent = Agent(model)Setting the client's retry count to zero does not disable retries — it moves the policy to one place you control, which is what you want when each attempt costs money. The other lever is validation: a model that struggles with a complex schema triggers the retry path on every run, so simplifying the output schema can cut requests more than switching model does.
What it actually costs
Pydantic AI agents sit in the middle of the range: a run carries a system prompt, tool schemas and whatever history you pass, then produces a structured response that is usually short. The distinctive cost is not the shape but the multiplier — retries mean the number of requests is higher than the number of runs.
| Model | Vendor | Rate in / out per 1M | Per run | Per month |
|---|---|---|---|---|
| gpt-5.6-luna | OpenAI | $0.1 / $0.6 | $0.0055 | $20 |
| MiniMax-M3 | MiniMax | $0.15 / $0.6 | $0.0074 | $27 |
| deepseek-v4-flash | DeepSeek | $0.22 / $0.66 | $0.0103 | $37 |
| gemini-3.7-flash | $0.375 / $1.875 | $0.0195 | $70 | |
| claude-haiku-4-5-20251001 | Anthropic | $0.5 / $2.5 | $0.0260 | $94 |
| deepseek-v4-pro | DeepSeek | $0.66 / $1.98 | $0.0309 | $111 |
| glm-5.3 | Zhipu | $0.7 / $2.2 | $0.0330 | $119 |
| qwen3.8-max | Alibaba | $1 / $3 | $0.0468 | $168 |
| claude-sonnet-5 | Anthropic | $1 / $5 | $0.0520 | $187 |
| gpt-5.6-terra | OpenAI | $1 / $6 | $0.0546 | $197 |
| kimi-k3 | Moonshot | $1.5 / $7.5 | $0.0780 | $281 |
| claude-opus-5 | Anthropic | $2.5 / $12.5 | $0.1300 | $468 |
How this fails in practice
| What you see | What it usually is | Fix |
|---|---|---|
| 404 or method not allowed | Default prefix targets /v1/responses | Use OpenAIChatModel or the openai-chat: prefix |
| ModuleNotFoundError on import | OpenAI extra not installed | Install pydantic-ai-slim[openai] |
| Requests exceed the number of runs | Client retries multiply with agent retries | Set the client's retry count explicitly |
| Validation never succeeds | Output schema too complex for the model | Simplify the schema; retries are billable |
| Wrong endpoint being used | Base URL inherited from the environment | Pass base_url to the provider in code |
| Timeouts under concurrency | Client timeout defaults and retry policy | Supply a pre-built client with explicit timeouts |
https://aicomp.ai/v1).
Create one free →
Confirming it took effect
Run the agent once and check the gateway usage log for both the request and its path. A successful result tells you a model answered; only the log tells you which route it answered on — and with this library the route is the thing most likely to be wrong.
FAQ
Why does my gateway reject requests from Pydantic AI?
Because the default model prefix targets a different route. In current Pydantic AI the bare openai: prefix resolves to a Responses model, which posts to /v1/responses; most OpenAI-compatible endpoints implement only /v1/chat/completions. Use openai-chat:, or instantiate OpenAIChatModel directly, and requests land on the route the endpoint actually serves.
How do I set the base URL?
Pass it to the provider: OpenAIProvider(base_url="…", api_key="…"), then hand the provider to the model class. The environment equivalents OPENAI_BASE_URL and OPENAI_API_KEY work too, but naming it in code makes the endpoint part of the reviewable source rather than the ambient environment.
Can I supply my own HTTP client?
Yes — the provider accepts a pre-built async client through its openai_client parameter, which is how you control retries, timeouts and organisation headers. It is also the only way to point the same model at an Azure-style client.
Why do I see more requests than I made?
Retries compound. The underlying client retries on its own — a couple of attempts by default, on 408, 429, 5xx and timeouts — and the agent has its own retry budget on top. A single logical call can therefore reach the network several times, and on a metered endpoint you pay for all of them. Set the client's retry count explicitly if you want a single policy.
Which model class should I pick for a compatible endpoint?
The chat one. OpenAIChatModel — reachable as the openai-chat: prefix — is what speaks to every OpenAI-compatible provider, because chat completions is the route they all implement. Reach for the Responses model only when the endpoint is known to serve that route.
Does the endpoint affect structured output?
No. Validation happens client-side against the schema you declared, so the endpoint sees an ordinary request. What it does affect is retries: a model that returns malformed output when a schema is complex will trigger the retry path, and on a metered endpoint those extra attempts are what shows up in the bill.