OpenAI-compatible API endpoints: what compatibility really covers

By the aitokensai editors · Verified 2026-11-01 Sources: OpenAI API docs

"OpenAI-compatible" has become the de facto standard for LLM APIs. Understanding exactly which parts are covered (and which three features are not) is what lets you switch providers without rewriting your integration.

In short: An OpenAI-compatible endpoint accepts the same request and response shapes as the OpenAI API, so switching is a base URL change. Chat completions is nearly universal; streaming is common; tool calling and structured output are where compatibility usually breaks.

What the contract includes

Layer	Usually compatible	Notes
`/v1/chat/completions`	Yes	The core of the standard; nearly universal
`/v1/models`	Yes	Good first check that a key works
`/v1/embeddings`	Often	Dimensions may differ from OpenAI's
Streaming (SSE)	Often	Chunk shape is usually, but not always, identical
Tool / function calling	Sometimes	The most common source of subtle breakage
Structured output / JSON mode	Sometimes	Frequently a prompt-level rather than API-level feature

Test it in one minute

Don't have an account yet? You need an API key from the gateway before the steps below will work — how the discount works and what you take on. Create one free → (no prepay, $1 minimum top-up)

curl https://aicomp.ai/v1/chat/completions \
  -H "Authorization: Bearer sk-your-gateway-key" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-5.6-luna",
    "messages": [{"role": "user", "content": "say OK"}],
    "stream": false
  }'

You want a JSON object with a choices array. Repeat with "stream": true, if you get incremental chunks, streaming works too.

Then point your client at it

from openai import OpenAI

client = OpenAI(
    base_url="https://aicomp.ai/v1",        # the only line that changes
    api_key="sk-your-gateway-key",
)

resp = client.chat.completions.create(
    model="gpt-5.6-luna",
    messages=[{"role": "user", "content": "say OK"}],
)

In Node it is the same idea with baseURL. Keep model IDs in configuration They are the one thing that genuinely differs between providers.

Why this matters commercially

Compatibility turns provider choice into a configuration decision. When two endpoints speak the same contract, you can move traffic for price or quality reasons without a rewrite, which is the difference between being locked in and having leverage.

Same contract, different rates — gateway rate vs official list
Model	Gateway rate in / out per 1M tokens	Official list in / out per 1M tokens	Diff
gpt-5.6-luna	$0.1 / $0.6	$0.2 / $1.2	50%
gpt-5.6-terra	$1 / $6	$2 / $12	50%
claude-sonnet-5	$1 / $5	$2 / $10	50%
deepseek-v4-pro	$0.66 / $1.98	— / —	—
kimi-k3	$1.5 / $7.5	— / —	—

Rates checked 2026-11-01. Gateway rates move with upstream promotions (verify the current number in your dashboard before committing to a budget).

Before you switch

List the features you actually use, do not assume, grep your code for stream, tools and response_format
Run a shadow test on real prompts before routing production traffic
Keep a fallback endpoint configured; compatibility bugs surface under load
Log model IDs and token usage per request so you can compare after the switch

Check current rates → Free to sign up · $1 minimum top-up · No prepayment

FAQ

What does OpenAI-compatible actually mean?

The endpoint accepts requests shaped like the OpenAI API (same paths, same field names, same response envelope) so an existing OpenAI client works against it by changing only the base URL and key.

Is every feature compatible?

No. Chat completions is nearly universal; streaming is common; tool calling and structured output vary the most. Always test the specific features your app depends on.

How do I test compatibility quickly?

Send one non-streaming request and one streaming request to /v1/chat/completions, then one tool call. If all three return the expected shapes, your client will almost certainly work.

Why does compatibility matter for cost?

Because it removes the switching cost. When several providers speak the same contract, you can move traffic based on price and quality instead of rewriting your integration each time.

Does using a compatible endpoint change my code?

Usually one line: the base_url you pass to the client. Model IDs differ between providers, so keep them in configuration rather than hard-coding them.

Which providers offer an OpenAI-compatible API?

Most of the infrastructure layer does: gateways that resell several vendors through one endpoint, routers such as LiteLLM and OpenRouter, and local servers like Ollama, vLLM and LM Studio. What they share is the /v1/chat/completions request and response shape, not the catalogue. Each one exposes a different set of models behind the same wire format, so switching between them changes which model IDs resolve.

Is an OpenAI-compatible API the same as the OpenAI API?

No. Compatible describes the interface, not the behaviour: the endpoint paths and JSON shapes match, so an existing OpenAI client works without modification. It does not mean the model performs identically, that every parameter is honoured, or that rate limits and error semantics line up. Verify behaviour separately from connectivity. A call that succeeds is not proof the model did the same thing.

What actually goes over the wire

Every OpenAI-compatible call is an HTTP POST to a base URL plus a resource path, with two things that decide everything else: the path, and the bearer token.

POST /v1/chat/completions HTTP/1.1
Host: aicomp.ai
Authorization: Bearer sk-...
Content-Type: application/json

{"model": "gpt-5.6-luna",
 "messages": [{"role": "user", "content": "..."}],
 "stream": true}

Read that literally, because three separate failure modes hide in it. The Host comes from your base URL, if the request arrives somewhere unexpected, this is the line that was wrong. The Authorization header is the only thing identifying you, so a shared key means indistinguishable spend. And the model value inside the body is not validated against anything client-side: send an ID the server does not serve and you get a server-side error, not a client-side warning.

Nothing else in the request is provider-specific. The message array, the tool schemas, the stream flag and the response shape are the wire format, and they are what makes one base URL substitutable for another. This is also the reason the change is reversible: you are not recompiling anything, you are editing one string.

When this is the wrong move

Compatibility is narrower than the label suggests. These are the cases where relying on it will hurt.

Situation	Why it breaks	Do this instead
You need embeddings, audio, files or fine-tuning	Compatibility claims usually cover chat completions and model listing only.	List the routes the endpoint actually exposes before designing around it.
You depend on provider-specific parameters	Extras outside the shared surface are dropped or rejected, often silently.	Keep to the common parameter set, or target the provider directly for those calls.
You parse error message strings	Error bodies are not standardised, and status codes alone do not carry enough detail.	Branch on status codes and log the body; do not match on prose.
You need guaranteed latency or a signed agreement	A compatibility layer is an extra hop with its own behaviour under load.	Measure it under your real concurrency, and keep the direct route for the critical path.
You need per-seat or per-project attribution	The endpoint sees one key, so spend is indistinguishable by caller.	Issue a key per environment or per service before you need to tell them apart.

Get API access