Cody with a Custom OpenAI-Compatible Endpoint
Cody is Sourcegraph's coding assistant, and its endpoint configuration lives in your editor's user settings rather than in an environment variable or a provider menu. You declare a list of models, and each entry carries its own connection details.
Every model entry declares its own context window, and the default is small. inputTokens defaults to 7000 and outputTokens to 4000 — numbers chosen for early local models, not for the 200k-context models a gateway will serve you. Cody trims what it sends to fit those numbers, so a correct connection can still behave like a much smaller model. apiEndpoint sits alongside apiKey and model inside each entry, and the entry only appears in the picker after a restart.
https://aicomp.ai/v1).
Create one free →
This page covers the settings shape, why the token numbers matter more than the URL, and what a month of Cody sessions costs once you stop throwing away context.
The settings entry
Open your VS Code user settings JSON and add a cody.dev.models array:
{
"cody.dev.models": [
{
"provider": "openai",
"model": "gpt-5.6-terra",
"inputTokens": 128000,
"outputTokens": 8192,
"apiKey": "sk-your-gateway-key",
"apiEndpoint": "https://gw.example.com/v1",
"options": { "temperature": 0.0 }
}
]
}The provider value is one of openai, google, groq or ollama. openai is the one that speaks the OpenAI wire format, which is what an OpenAI-compatible gateway expects. apiEndpoint overrides the provider's default endpoint — leave it out and the entry points at the provider's own host, which is the most common reason a configured model still does not reach your gateway.
apiKey is required for all four provider types. There is no keyless path.
inputTokens is a budget, not a fact
This is the field that costs the most to get wrong, because nothing errors.
inputTokens and outputTokens are optional, and their documented defaults are 7000 and 4000. Cody does not query your endpoint to find out how large the real window is — it uses the number you wrote to decide how much repository context to assemble and how much to trim. Set nothing, and a model that accepts 200k tokens receives a fraction of that:
{
"provider": "openai",
"model": "claude-opus-5",
"inputTokens": 200000,
"outputTokens": 16384,
"apiKey": "sk-your-gateway-key",
"apiEndpoint": "https://gw.example.com/v1"
}What you see when the number is too low is not an error. Cody silently drops files from the context it would otherwise have included, and the answers get vaguer as the repo gets bigger. That reads as "the model is worse than advertised" rather than "my configuration is truncating the prompt", which is why it survives so long undiagnosed.
Setting it too high is the opposite failure: the assembled prompt exceeds what the endpoint accepts, and you get a context length error on every request. See context length exceeded for how to read that error against what you configured. The right value is the smaller of your model's window and what your gateway will accept — not the largest number you can find on a pricing page.
Restart, then pick the model
The entry does not appear immediately. Once configured, VS Code has to be restarted before the model shows up in the dropdown for chat and for edits. If the model name in the status bar still reads a default Cody model after you save the file, the setting was not picked up, or the JSON is invalid, or the entry is missing a required field.
To see what actually went out, open the Cody sidebar, click the ... next to Settings and Support, and choose Open Output Channel. That panel is the only place the request details surface, and it is the fastest way to tell a wrong host from a wrong key from a wrong model id.
Enterprise instances configure it server-side
If you are on a Sourcegraph Enterprise instance, the endpoint is set on the instance rather than in each developer's editor:
{
"cody.enabled": true,
"completions": {
"provider": "openai",
"accessToken": "sk-your-gateway-key",
"endpoint": "https://gw.example.com/v1",
"chatModel": "gpt-5.6-terra",
"completionModel": "gpt-5.6-terra"
}
}Note that this side uses endpoint and accessToken, not apiEndpoint and apiKey. The two surfaces are separate: the site config decides what the instance reaches, and the VS Code setting decides what an individual developer's editor declares. A developer-side entry does not override the instance, which is another way a correct-looking configuration appears to do nothing.
Why the input side is most of the bill
Cody assembles a prompt from your repository — the files it judges relevant, plus your question. That assembly is billed as input on every single turn, including autocomplete suggestions, which fire far more often than chat messages. Meanwhile the output is one completion.
A day of work on this page's workload is 320k input and 45k output tokens — 6.4M input and 0.9M output across 20 working days. The ratio matters more than the total: at the volumes above, cutting the input token price in half moves the monthly number further than any output-side change. See output vs input pricing for how to work out which side your own workload sits on.
| Model | Gateway rate in / out per 1M tokens | One day | 20 days |
|---|---|---|---|
| claude-opus-5 | $2.5 / $12.5 | $1.36 | $27.25 |
| gpt-5.6-terra | $1.00 / $6.00 | $0.59 | $11.80 |
| deepseek-v4-pro | $0.66 / $1.98 | $0.30 | $6.01 |
| kimi-k3 | $1.5 / $7.5 | $0.82 | $16.35 |
| gemini-3.7-flash | $0.375 / $1.875 | $0.20 | $4.09 |
One day = 320k input + 45k output on this page's workload. 20 days = 6.4M input and 0.9M output. Rates checked 2026-10-11 — gateway rates move with upstream promotions, so verify the current number in your dashboard before committing to a budget.
Verifying the route
- Set
apiEndpointinside the entry. Omitting it sends traffic to the provider's own host even though the entry looks complete. - Set
inputTokensandoutputTokensexplicitly. The defaults are 7000 and 4000, and they are never inferred from the endpoint. - Restart VS Code before checking. The model does not appear in the picker until the editor reloads.
- Open the Output Channel and read the request it actually built — host, path and model id.
- Ask about a file you did not open. If the answer is vague, context is being trimmed; if it errors, you overshot the window.
FAQ
FAQ
Why does my model answer like a smaller one?
inputTokens is still at its 7000 default, so Cody is trimming the repository context it sends. Raise it to what your model actually accepts.
The model does not appear in the dropdown
Restart VS Code. The picker is populated at startup, and a saved settings file is not enough on its own.
apiKey is required even for a local server with no auth?
Yes. All four provider types require the field. Put any non-empty value there if the server does not check it.
I set apiEndpoint but traffic still goes to the default host
Check whether you are on a Sourcegraph Enterprise instance — the site config takes precedence, and the developer-side entry does not override it.
Is this the same as configuring Continue or Zed?
The shape is different: Cody declares an array of model objects each carrying its own endpoint and context budget, where Continue and Zed use a flatter provider configuration. Cody is also the only one of the three with a server-side configuration surface that outranks the editor.
Does a bigger inputTokens make it more expensive?
Yes, and linearly — the assembled context is billed as input on every turn, including autocomplete. The number is both a quality setting and a cost setting. See monitoring API spend for keeping an eye on it.
Related
- Continue with a custom endpoint — another VS Code extension, configured more flatly
- Zed with a custom endpoint — editor with its own provider settings
- Context length exceeded — what the error says when you overshoot the window
- Output vs input pricing — which side your workload actually sits on
- Tools that take one endpoint — the full matrix