Smolagents with a Custom OpenAI-Compatible Endpoint

Smolagents is Hugging Face's agent library: you build an agent that writes Python to solve the task, and the agent calls a model through a model class you pick up front. Connecting a gateway is two arguments, but which class you construct decides what the model id has to look like — and getting that wrong produces an error that points at the key rather than the id.

LiteLLMModel needs the provider prefix in the model id; OpenAIServerModel must not have one. Both take api_base, both accept an api_key, and the two look interchangeable in the signature. They are not: one routes through LiteLLM, which parses the prefix to decide how to build the request, and the other hands the id straight to an OpenAI SDK client.

This page covers which class to pick, what each one does with the id you pass, and why a code-writing agent grows the input side faster than a chat loop does.

Two classes, two id rules

from smolagents import CodeAgent, LiteLLMModel

model = LiteLLMModel(
    model_id="openai/gpt-5.6-terra",      # prefix required
    api_base="https://gw.example.com/v1",
    api_key="sk-your-gateway-key",
)
agent = CodeAgent(tools=[], model=model)
agent.run("Summarise the last 30 days of signups from signups.csv")

from smolagents import CodeAgent, OpenAIServerModel

model = OpenAIServerModel(
    model_id="gpt-5.6-terra",             # no prefix
    api_base="https://gw.example.com/v1",
    api_key="sk-your-gateway-key",
)
agent = CodeAgent(tools=[], model=model)

The difference is entirely in model_id. In LiteLLMModel, the text before the slash is what LiteLLM reads to choose a request format: openai/... selects the Chat Completions shape, anthropic/... selects Messages, ollama_chat/... selects Ollama's. Drop the prefix and LiteLLM is left guessing from the model name alone, which is how you get "LLM Provider NOT provided" — an error that names the provider, not the id, so the natural fix is to go and check your key.

In OpenAIServerModel there is no router. The id is sent exactly as written in the model field of the request body. Add openai/ there and your gateway receives openai/gpt-5.6-terra as the model name and returns model not found, while the model is plainly listed on your dashboard.

So the two failure modes read oppositely: a missing prefix looks like a credentials problem, and an unwanted prefix looks like an inventory problem.

api_base behaves identically in both — it is the root the client builds on, so it carries the version path your gateway expects and never the operation path. Neither class appends /chat/completions for you to remove; both add it.

Which one to pick

LiteLLMModel also carries num_ctx, and it is a trap when you are not on Ollama: it sets the context window for Ollama models specifically. Passing it for a gateway model does nothing — no error, no effect — so a request you expected to be trimmed to 8k goes out at full length.

One internal detail worth knowing if you are debugging message shapes: LiteLLMModel sets flatten_messages_as_text from whether the id starts with ollama. On an OpenAI-compatible gateway it is off, which is what you want.

Why the input side grows faster here

A CodeAgent does not answer once. It writes Python, runs it, reads the output or the traceback, writes again, and repeats — up to max_steps. Every step re-sends the whole message history, and each failed step appends an execution result that can be longer than the answer would have been.

That last part is what makes the shape different from a chat loop. A traceback plus the offending code plus a variable dump gets appended to the conversation and re-sent on the next step, so a run that fails three times before succeeding can cost more input than a run that got it right immediately. Capping max_steps is a cost control, not just a safety rail.

A day of work on this page's workload is 300k input and 45k output tokens — 6M input and 0.9M output across 20 working days. The input side dominates because the agent keeps re-reading its own history; see output vs input pricing for why that ratio decides which model is cheapest here.

One day = 300k input + 45k output on this page's workload. 20 days = 6M input and 0.9M output. Rates checked 2026-10-11 — gateway rates move with upstream promotions, so verify the current number in your dashboard before committing to a budget.

Verifying the route

1. Pick the class first, then write the id to match. OpenAIServerModel takes a bare id; LiteLLMModel takes openai/<id>.
2. Point api_base at the version root, not at /chat/completions.
3. Ignore num_ctx unless the id starts with ollama.
4. Set max_steps on the agent. Each retry re-sends the history and appends the failure.
5. Read the error's target. Provider NOT provided is the id in LiteLLMModel. model not found is an unwanted prefix in OpenAIServerModel.

FAQ

FAQ

Related

• OpenAI SDK with a custom endpoint — what OpenAIServerModel builds on
• LiteLLM with a custom endpoint — the router behind LiteLLMModel, and its model-name mapping
• Agno with a custom endpoint — another Python agent framework, with a different default-for-invalid-value problem
• Tools that take one endpoint — the full matrix

Get API access