Point OpenHands at an OpenAI-compatible endpoint

Q: Where does OpenHands read its LLM settings from?

From config.toml in the working directory, under an [llm] section with model , api_key and base_url . Every one of those has an environment-variable equivalent with an LLM_ prefix — LLM_MODEL , LLM_API_KEY , LLM_BASE_URL — which is the form to use in containers and CI. Where both are present, the values in the file win, which is worth knowing before you spend an afternoon debugging an environment variable that is being ignored.

Q: Why is the retry count a cost setting?

Because a retry resends the whole accumulated prompt. With num_retries at its default of 8, a model that intermittently fails on a long context can turn one failed task into eight full-history charges. On a workload that is roughly 150 parts input to one part output, retries are the single most expensive thing that can go wrong, and lowering the count is a one-line change.

Q: Does the base URL need /v1 on the end?

Yes — give the root that ends in /v1 and let the client append the rest of the path. Adding a path beyond that, or adding a trailing slash, produces requests to a doubled path and a 404 that looks like an endpoint outage rather than a typo. If every request 404s, check the URL first; it is the most common cause and the quickest to rule out.

OpenHands is an agent loop, not a chat window. That single fact decides everything about its cost: each turn resends the entire accumulated history, so input — not output — is the rate that sets the bill, and the guardrails that ship switched off are the ones that matter most.

In short: OpenHands reads its model configuration from an [llm] block in config.toml or from LLM_-prefixed environment variables. A 25-turn task sends about 3M input against 20k output — roughly 150:1 — and both max_budget_per_task (default 0.0, no limit) and num_retries (default 8, each resending full history) work against you.

Before you start: you need an endpoint root ending in /v1 and a key from that endpoint. The setting below is the only thing that changes — request and response handling stay identical. Rates checked 2026-09-26.

The setting

OpenHands reads its model configuration from config.toml in the working directory. Three keys under [llm] are enough to move it off the default provider:

[llm]
model = "openai/claude-sonnet-5"
api_key = "sk-..."
base_url = "https://aicomp.ai/v1"

[llm.caching]
# Prompt caching is on by default. If the endpoint does not support it,
# OpenHands pays full price for the history it resends every turn.
enabled = true

[core]
max_iterations = 25
max_budget_per_task = 2.00

Every one of those has an environment-variable equivalent, which is the form to use when you are running in a container or in CI:

# Env vars override config.toml — useful in CI and containers
export LLM_MODEL="openai/claude-sonnet-5"
export LLM_BASE_URL="https://aicomp.ai/v1"
export LLM_API_KEY="sk-..."

# The two guardrails that are NOT set for you:
export MAX_BUDGET_PER_TASK="2.00"   # default 0.0 = no limit
export MAX_ITERATIONS="25"          # default 100
export LLM_NUM_RETRIES="2"          # default 8 — each retry resends history

Two of those variables are not what most people expect. MAX_BUDGET_PER_TASK defaults to 0.0, which means no limit. LLM_NUM_RETRIES defaults to 8, and every retry resends the full history. On a workload this input-heavy, both defaults point the wrong way.

What an OpenHands task actually costs

An agent loop does not send one prompt and get one answer. It sends a prompt, executes, observes the result, and sends the whole thing again with the observation appended. Over a 25-turn task the prompt grows from roughly 40k tokens to roughly 200k, averaging about 120k — so one task is around 3M input tokens and 20k output tokens. That is a ratio near 150:1, and it is the most extreme shape of any tool covered on this site.

Forty agent tasks at 25 turns each: 3M input and 20k output per task. Rates checked 2026-09-26.
Model	Vendor	Rate in / out per 1M	40 tasks	Input share of the bill	Same, monthly
gpt-5.6-luna	OpenAI	$0.1 / $0.6	$12.48	96%	$12
MiniMax-M3	MiniMax	$0.15 / $0.6	$18.48	97%	$18
deepseek-v4-flash	DeepSeek	$0.22 / $0.66	$26.93	98%	$27
gemini-3.7-flash	Google	$0.375 / $1.875	$46.50	97%	$46
claude-haiku-4-5-20251001	Anthropic	$0.5 / $2.5	$62.00	97%	$62
deepseek-v4-pro	DeepSeek	$0.66 / $1.98	$80.78	98%	$81
glm-5.3	Zhipu	$0.7 / $2.2	$85.76	98%	$86
qwen3.8-max	Alibaba	$1 / $3	$122.40	98%	$122
claude-sonnet-5	Anthropic	$1 / $5	$124.00	97%	$124
gpt-5.6-terra	OpenAI	$1 / $6	$124.80	96%	$125
kimi-k3	Moonshot	$1.5 / $7.5	$186.00	97%	$186
claude-opus-5	Anthropic	$2.5 / $12.5	$310.00	97%	$310

Across the twelve models in that table, input is between 96% and 98% of the total. The ranking itself is the same one you get on a chat workload, and that is the honest answer: once a workload is this far towards input, no output rate on offer is extreme enough to overturn it. What actually differs is how exposed you are — at 96% input, a 10% move in the input rate moves the bill by nearly 10%, while the same move in the output rate is invisible.

Cost note. At 150:1 the output rate is almost irrelevant — a model with a cheap input rate and an expensive output rate wins here, while the same model loses on a chat workload. The useful consequence is that you do not need to re-rank models for every agent tool; you need the input rate, and you need the endpoint to honour the cache.

The practical consequence is that prompt caching matters more than model choice. OpenHands turns it on by default, so on an endpoint that honours the cached rate, the resent history is billed far below the list input rate. On an endpoint that does not, you pay the full input rate for every token of history on every turn, and no amount of model shopping will close that gap.

How this fails in practice

The failures that account for most setup problems, and what each one actually means.
What you see	What it usually is	Fix
404 on every request	An extra path segment or trailing slash on base_url	Give the root ending in /v1 and nothing after it
model not found	Model ID missing the provider prefix the resolver expects	Copy the ID from /v1/models and add the prefix
A stuck task keeps spending	max_budget_per_task defaults to 0.0, which is no limit	Set an explicit per-task budget
One task costs eight times what it should	num_retries defaults to 8, and each retry resends history	Lower retries to 2 and let the failure surface
Cost far higher than the table predicted	Prompt caching not honoured by the endpoint	Confirm the endpoint supports cached input before comparing
Config changes seem to be ignored	config.toml takes precedence over environment variables	Change the file, not the variable

You need a key before the code below runs. Create an account, generate a key, and copy the base URL (https://aicomp.ai/v1). Create one free →

Check current rates → Free to sign up · $1 minimum top-up · No prepayment

Confirming it took effect

Run one deliberately small task, then read the usage log on the endpoint. A request recorded there is the only proof that matters — a successful response can still come from a provider OpenHands has configured from an earlier run, and the UI does not tell you which one answered. If the log is empty, the request never left the application: check the model ID, then the URL, then the key, in that order.

FAQ

Where does OpenHands read its LLM settings from?

From config.toml in the working directory, under an [llm] section with model, api_key and base_url. Every one of those has an environment-variable equivalent with an LLM_ prefix — LLM_MODEL, LLM_API_KEY, LLM_BASE_URL — which is the form to use in containers and CI. Where both are present, the values in the file win, which is worth knowing before you spend an afternoon debugging an environment variable that is being ignored.

Why does my model name come back as not found?

OpenHands resolves model identifiers LiteLLM-style, so a model that is not natively OpenAI usually needs a provider prefix such as openai/ in front of the ID. A bare identifier that works fine against the endpoint with curl often fails here for exactly that reason. Copy the ID from the endpoint's own /v1/models listing and add the prefix the resolver expects, rather than reconstructing the name from memory.

What does MAX_BUDGET_PER_TASK actually do?

It stops a single task once the spend crosses the figure you set. The default is 0.0, which means no limit rather than a zero budget — that distinction is easy to get backwards. On an agent loop that resends its full history every turn, an unlimited budget is not a theoretical risk: a task that goes in circles will keep billing until something external stops it.

How much does prompt caching change the bill here?

A lot, because this workload is almost entirely input. OpenHands enables prompt caching by default, so on an endpoint that supports it the resent history is billed at the cached rate instead of the full input rate. On an endpoint that does not, you pay full price for every token of history on every turn. Before comparing models on an agent workload, check whether the endpoint actually honours the cache — otherwise the comparison is between two different billing models, not two prices.

Why is the retry count a cost setting?

Because a retry resends the whole accumulated prompt. With num_retries at its default of 8, a model that intermittently fails on a long context can turn one failed task into eight full-history charges. On a workload that is roughly 150 parts input to one part output, retries are the single most expensive thing that can go wrong, and lowering the count is a one-line change.

Does the base URL need /v1 on the end?

Yes — give the root that ends in /v1 and let the client append the rest of the path. Adding a path beyond that, or adding a trailing slash, produces requests to a doubled path and a 404 that looks like an endpoint outage rather than a typo. If every request 404s, check the URL first; it is the most common cause and the quickest to rule out.

How do I know the change actually took effect?

Run one small task and then read the usage log on the endpoint. A request recorded there is the only real proof, because a plausible response can also be produced by a provider OpenHands still has configured from earlier, and nothing in the UI tells you which provider answered. If the log is empty, the request never left the application — check the model ID first, then the URL, then the key.

Get API access

Point OpenHands at an OpenAI-compatible endpoint

The setting

What an OpenHands task actually costs

How this fails in practice

Confirming it took effect

FAQ

Related

839 models. One key. $1 to start.