LiteLLM model names: the prefix is a routing instruction

LiteLLM reads your model string as two things, not one. It splits at the first slash: the segment before names the provider, and everything after is the model id handed to that provider untouched. Get that split wrong and you get one of two failures that look alike but are opposites — one raised before any request is sent, one returned by a server that received a perfectly routed call.

curl https://aicomp.ai/v1/models \
  -H "Authorization: Bearer sk-your-gateway-key"

What the split actually does

LiteLLM's own documentation for OpenAI-compatible endpoints states the rule directly: put openai/ in front of your model name so LiteLLM knows you are calling a /chat/completions endpoint. For the older /completions route the documented prefix is text-completion-openai/ instead. The part after the first slash is passed to the provider unchanged.

"Unchanged" is the half people miss. It means additional slashes are not separators — they are part of the model id. So openai/glm/glm-5.2 resolves to provider openai and sends glm/glm-5.2 upstream. That is not a hack; it is the only way to reach an endpoint that registers its ids with a vendor segment already in them.

def split_model(model: str):
    """Mirror LiteLLM's documented rule: the first segment names the provider,
    everything after the first slash is the model id and is passed through
    to that provider unchanged -- including any further slashes."""
    provider, sep, rest = model.partition("/")
    if not sep:
        return None, model      # no provider in the string: routing must be inferred
    return provider, rest


for m in ["gpt-5.6-luna",
          "openai/gpt-5.6-luna",
          "openai/glm/glm-5.2",
          "anthropic/claude-sonnet-5",
          "support-bot"]:
    provider, sent = split_model(m)
    if provider is None:
        print(f"{m:30} -> no provider given, LiteLLM must guess or refuse")
    else:
        print(f"{m:30} -> provider={provider:10} sent upstream={sent}")

Run that against the ids you actually use before you debug anything else. It needs no LiteLLM installed, and it tells you in one line what the endpoint is going to receive.

The error that fires before the request does

Pass a string with no provider and no match in LiteLLM's internal map, and you get: litellm.exceptions.BadRequestError: LLM Provider NOT provided. Pass in the LLM provider you are trying to call. You passed model=support-bot.

Read the last clause literally — it is telling you the request never happened. No socket opened, no token counted, nothing billed. This is the single most useful fact on the page, because it splits the failure space cleanly:

• The routing error is free. It is raised inside your process. Retrying it, or rotating a key over it, cannot work — the request is identical and so is the answer.
• A 404 is not free and is not the same thing. It means routing succeeded, a request went out, and the endpoint rejected the id. Now the key is proven valid and the id is the suspect.
• A 401 is a third thing again. Authentication precedes routing on the server, so the id may be fine and the credential is not. Our error-code walkthrough covers that ordering.

Three ways to set the provider, and where each belongs

The prefix in the string is the most explicit and the easiest to read in a log. It is also the only one that survives being handed to a framework that re-serialises config.

import os
import litellm

# The prefix is a routing instruction, not part of the model name.
# LiteLLM strips the first segment and sends the rest to the endpoint as-is.
resp = litellm.completion(
    model="openai/gpt-5.6-luna",
    api_base="https://aicomp.ai/v1",
    api_key=os.environ["GATEWAY_API_KEY"],
    messages=[{"role": "user", "content": "Reply with the single word: ok"}],
    max_tokens=8,
)

print("answered by:", resp.model)      # identity, as the endpoint reports it
print("usage      :", resp.usage)      # None here means usage was never returned

# When you cannot change the model string (it comes from a database, a UI,
# a framework config), pass the provider as its own argument instead:
same = litellm.completion(
    model="gpt-5.6-luna",
    custom_llm_provider="openai",      # same routing, no prefix in the string
    api_base="https://aicomp.ai/v1",
    api_key=os.environ["GATEWAY_API_KEY"],
    messages=[{"role": "user", "content": "Reply with the single word: ok"}],
    max_tokens=8,
)

But the right place for the mapping, once you have more than one caller, is the proxy. There, model_name is what callers ask for and litellm_params.model is what is sent — so a friendly alias is decoupled from the qualified id behind it.

# litellm_config.yaml
# Callers ask for a friendly name; the proxy owns the provider-qualified one.
# Swapping the backend later is a one-line change here, not a client change.
model_list:
  - model_name: fast
    litellm_params:
      model: openai/deepseek-v4-flash-0731
      api_base: https://aicomp.ai/v1
      api_key: os.environ/GATEWAY_API_KEY
  - model_name: strong
    litellm_params:
      model: openai/claude-sonnet-5
      api_base: https://aicomp.ai/v1
      api_key: os.environ/GATEWAY_API_KEY

That indirection is what makes a backend swap a one-line change instead of a coordinated release across every service that hardcodes a model name. It is also the natural place to put a fallback, and therefore the place to think about how many upstream calls one request can turn into — which is why the retry and timeout budget is the next page to read once names resolve.

How many ids actually need the stacked form

In the 839-model catalogue we track, 12 ids contain a slash. For those, a single openai/ prefix is not enough — the vendor segment has to be preserved, which is exactly what stacking does.

Showing 8 of 12. The remaining 827 ids carry no slash, so a single prefix is sufficient for them.

The general rule, and the only one worth memorising: never derive the id from anything except the endpoint's own list. Not from a docs page, not from a URL slug, not from what worked on a different endpoint.

# The endpoint's own list is the only authoritative source of model ids.
curl -s https://aicomp.ai/v1/models -H "Authorization: Bearer $GATEWAY_API_KEY" > /tmp/models.json

head -c 200 /tmp/models.json
#   -> starts with {"object":"list" ...   good
#   -> starts with <!DOCTYPE or <html     something else answered; fix the URL first

python3 - <<'PY'
import json
d = json.load(open("/tmp/models.json"))
ids = [m.get("id") for m in d.get("data", [])]
print(len(ids), "models returned")
print("\n".join(str(i) for i in ids[:10]))
PY

One more thing that command catches: if the first 200 bytes are HTML rather than JSON, something between you and the endpoint answered — a captive portal, a corporate proxy — and no amount of config change will fix it from inside your app. That is the same first check as in base URL not working, and it is worth running before the model-name work rather than after.

What resolving to the wrong model costs

A name that resolves silently is worse than one that errors. If a fallback or an alias points somewhere you did not intend, every call succeeds, nothing raises, and the invoice reflects a model you never chose.

Rates checked 2026-09-26. Gateway rates move with upstream promotions — verify the current number in your dashboard before committing to a budget.

Rates checked 2026-09-26. Gateway rates move with upstream promotions — verify the current number in your dashboard before committing to a budget. Across the 216 models in our cleaned catalogue the median output-to-input ratio is 4.0×.

Related guides

• One endpoint for every tool — the hub for tool-specific setup
• LiteLLM with a custom endpoint — getting the base URL right first
• LiteLLM retries and timeouts — the budget after names resolve
• Endpoint error codes: 401 vs 404 — routing vs credential vs path
• Reading /v1/models — where the authoritative id comes from
• Base URL not working — when the URL is the problem, not the name

Name-resolution failures and what each one is actually telling you

When this is the wrong move

Four shortcuts that turn a two-minute fix into an afternoon

Rolling it out without finding out the hard way

The failure mode you want to avoid is discovering a problem through a production bill or a customer-visible error. Four steps, in order:

1. Prove it on one request per alias, with the resolved model id printed back. One call, from a script, with an explicit timeout and the model ID echoed back. You are testing reachability, authentication and model availability — three things that can fail independently.
2. Measure before you switch. Record tokens per task and cost per task on the current path first. Without that baseline, "it got cheaper" is an impression, not a result.
3. Move one workload, not everything. Pick the workload with the most predictable shape — batch jobs over interactive traffic. Leave the interactive path on the old configuration until the batch numbers are in.
4. Decide the rollback condition in advance. Write down what makes you revert (error rate above X, cost per task above Y, latency above Z) before you start, so the decision is not made under pressure.

Keep the base URL in configuration, never inline. That single choice is what makes step four take a minute instead of an afternoon.

FAQ

Get API access