OpenAI-compatible API endpoints: what compatibility really covers
"OpenAI-compatible" has become the de facto standard for LLM APIs. Understanding exactly which parts are covered — and which three features are not — is what lets you switch providers without rewriting your integration.
What the contract includes
| Layer | Usually compatible | Notes |
|---|---|---|
/v1/chat/completions | Yes | The core of the standard; nearly universal |
/v1/models | Yes | Good first check that a key works |
/v1/embeddings | Often | Dimensions may differ from OpenAI's |
| Streaming (SSE) | Often | Chunk shape is usually, but not always, identical |
| Tool / function calling | Sometimes | The most common source of subtle breakage |
| Structured output / JSON mode | Sometimes | Frequently a prompt-level rather than API-level feature |
Test it in one minute
curl https://aicomp.ai/v1/chat/completions \
-H "Authorization: Bearer sk-your-gateway-key" \
-H "Content-Type: application/json" \
-d '{
"model": "gpt-5.6-luna",
"messages": [{"role": "user", "content": "say OK"}],
"stream": false
}'You want a JSON object with a choices array. Repeat with "stream": true — if you get incremental chunks, streaming works too.
Then point your client at it
from openai import OpenAI
client = OpenAI(
base_url="https://aicomp.ai/v1", # the only line that changes
api_key="sk-your-gateway-key",
)
resp = client.chat.completions.create(
model="gpt-5.6-luna",
messages=[{"role": "user", "content": "say OK"}],
)In Node it is the same idea with baseURL. Keep model IDs in configuration — they are the one thing that genuinely differs between providers.
Why this matters commercially
Compatibility turns provider choice into a configuration decision. When two endpoints speak the same contract, you can move traffic for price or quality reasons without a rewrite — which is the difference between being locked in and having leverage.
| Model | Gateway rate in / out per 1M tokens | Official list in / out per 1M tokens | Diff |
|---|---|---|---|
| gpt-5.6-luna | $0.1 / $0.6 | $0.2 / $1.2 | 50% |
| gpt-5.6-terra | $1 / $6 | $2 / $12 | 50% |
| claude-sonnet-5 | $1 / $5 | $2 / $10 | 50% |
| deepseek-v4-pro | $0.66 / $1.98 | — / — | — |
| kimi-k3 | $1.5 / $7.5 | — / — | — |
Rates checked 2026-09-16. Gateway rates move with upstream promotions — verify the current number in your dashboard before committing to a budget.
Before you switch
- List the features you actually use — do not assume, grep your code for
stream,toolsandresponse_format - Run a shadow test on real prompts before routing production traffic
- Keep a fallback endpoint configured; compatibility bugs surface under load
- Log model IDs and token usage per request so you can compare after the switch
FAQ
What does OpenAI-compatible actually mean?
The endpoint accepts requests shaped like the OpenAI API — same paths, same field names, same response envelope — so an existing OpenAI client works against it by changing only the base URL and key.
Is every feature compatible?
No. Chat completions is nearly universal; streaming is common; tool calling and structured output vary the most. Always test the specific features your app depends on.
How do I test compatibility quickly?
Send one non-streaming request and one streaming request to /v1/chat/completions, then one tool call. If all three return the expected shapes, your client will almost certainly work.
Why does compatibility matter for cost?
Because it removes the switching cost. When several providers speak the same contract, you can move traffic based on price and quality instead of rewriting your integration each time.
Does using a compatible endpoint change my code?
Usually one line: the base_url you pass to the client. Model IDs differ between providers, so keep them in configuration rather than hard-coding them.