Twinny with a custom OpenAI-compatible endpoint
Twinny is a free VS Code extension with two independent features: a sidebar chat and inline fill-in-the-middle completion. Both can point at an OpenAI-compatible endpoint, and the configuration form presents them side by side, which makes it look like one setting expressed twice.
It is not. Chat and completion call different routes. A gateway that serves the chat surface and nothing else will give you a working chat panel and suggestions that never appear, with no error anywhere to explain the difference.
https://aicomp.ai/v1).
Create one free →
Two slots, not one setting
The settings split along job type:
| Job | Field group | Default route |
|---|---|---|
| Chat | twinny.provider, twinny.apiUrl, twinny.apiKey, twinny.model | /v1/chat/completions |
| Completion (FIM) | twinny.fimProvider, twinny.fimApiUrl, twinny.fimApiKey, twinny.fimModel | /v1/completions |
The two groups are independent. You can run chat against a hosted gateway and completion against a local server, or both against one endpoint But you have to configure each, and each is validated by a different symptom when it is wrong.
Completion is the one that fails quietly. A broken chat slot produces a visible error in the sidebar. A broken completion slot produces nothing at all. No inline suggestion, no message, no log line the user is likely to see. Silence is the only signal.
Why a chat-only gateway breaks completion
The completion route is not a smaller version of the chat route. It is a raw prompt call: Twinny sends a fill-in-the-middle prompt and expects raw continuation tokens back, in the format the model family was trained on.
That produces two independent requirements:
- The endpoint must expose a completions-style route (
/v1/completions), not only/v1/chat/completions. - The model behind it must be one that accepts a raw prompt. A chat-tuned, instruction-following model will answer a FIM prompt as though it were a question, or refuse it.
Many hosted models are chat-only by design. That is not a defect — it simply means the completion slot needs either a base model or a route that re-frames the FIM prompt server-side. A gateway that offers a pure chat surface satisfies requirement one for chat and neither requirement for completion.
Practical consequence: if chat works and completion is silent, the extension is not misconfigured. The endpoint is.
The field name that misleads
Twinny ships presets for local runtimes, and one of the settings that appears in older community write-ups is named after a specific runtime while being a general endpoint field. The name describes a preset, not a restriction (a URL placed there is used as a plain OpenAI-compatible endpoint).
This is worth knowing for a specific reason: the fix people reach for when completion fails is to change that field, because its name suggests it governs the local path. It governs nothing of the sort. What governs completion is the FIM slot (fimProvider / fimApiUrl / fimModel), which is a separate group a few lines further down.
Editing the runtime-named field and watching completion stay broken is the expected outcome, not a puzzle.
The preset table, read as a routing table
Twinny's documented presets are useful less as a list of supported runtimes and more as evidence that the paths differ per preset. The docs give per-preset job paths, chat, autocomplete and embeddings each on their own route, with defaults that differ between presets.
The defaults worth internalising:
- The generic OpenAI-compatible preset assumes
localhost:8080with/v1,/v1/completionsand/v1/embeddings. - Presets differ on whether the version segment is part of the field or appended.
That last point is the same class of trap as in every other editor in this cluster: the version segment is either yours or the tool's, and the form tells you which by the default it ships. Read the default before typing.
If a server needs a path none of the presets cover, the form takes an arbitrary hostname, port, protocol and path And there is a test control that reports whether the server answered and what it said. Use it before assuming the key is wrong.
Prompt templates are part of completion, not an extra
Because completion is a raw-prompt call, the template matters as much as the endpoint. Twinny keeps FIM templates as files, and the custom template is editable.
A wrong template does not produce a connection error. It produces output that ignores the code below the cursor, or tokens containing your own prompt. Both look like model quality problems and are template problems.
The rule: name the model after its family so automatic template detection works, or select the template by hand. A model named arbitrarily will fall back to a generic template, and a generic template is wrong for most families.
Wiring it to one endpoint
For a gateway, the working configuration is chat and completion on the same base URL and key, with the model chosen per slot — the model IDs differ because the completion slot should use something that accepts a raw prompt.
| Model | Gateway rate in / out per 1M tokens | One day | 20 days |
|---|---|---|---|
| deepseek-v4-pro | $0.66 / $1.98 | $0.21 | $4.28 |
| claude-sonnet-5 | $1.00 / $5.00 | $0.38 | $7.60 |
| qwen3.8-max | $1.00 / $3.00 | $0.32 | $6.48 |
| gpt-5.6-terra | $1.00 / $6.00 | $0.41 | $8.16 |
One day = 240k input + 28k output on this page's workload. 20 days = 4.8M input and 0.56M output. Rates checked 2026-10-15, gateway rates move with upstream promotions, so verify the current number in your dashboard before committing to a budget.
A working month of that pattern is about 240k input and 28k output tokens a day, so 4.8M input and 0.56M output over 20 working days. The input figure is the one to check a candidate model against, because a completion call that looks cheap per request is cheap only if its input rate is.
The economics here are unusual, and worth stating plainly. Completion fires constantly but each call is small; chat is occasional but resends its conversation. That means the two slots reward different choices, and optimising a single model for both is how a bill ends up worse than either slot needed to be.
Troubleshooting, in the order that saves time
- Chat broken, completion broken, the base URL or key is wrong. Fix that first; a shared cause explains both.
- Chat works, completion silent. Assume the endpoint does not serve a completions route or the model is chat-only. Test the completions route directly, outside the extension.
- Completion returns junk, the FIM template does not match the model family. Set the template explicitly.
- Neither updates after editing settings (reload the VS Code window). The extension reads its configuration on load.
FAQ
FAQ
Does Twinny work with any OpenAI-compatible endpoint?
For chat, effectively yes, any server implementing the chat completions API works. For completion, only endpoints that expose a completions-style route, and only models that accept a raw prompt.
Why do inline suggestions never appear while chat is fine?
The two slots use different routes and different fields. A chat-only endpoint leaves the completion route unserved, and the extension reports that as silence rather than an error.
Can chat and completion use different providers?
Yes, and that is a supported configuration rather than a workaround (the two field groups are independent).
Is the runtime-named setting the one that controls local completion?
No. It is an endpoint field that accepts any OpenAI-compatible base URL. Completion is governed by the FIM slot, which is separate.
Do I need a specific model for completion?
One that accepts a raw prompt. Chat-tuned instruction models may answer the prompt instead of continuing it. Pair the model with a matching FIM template.
Related
- Tools that take one endpoint, the full matrix, including the clients that need a completions route rather than a chat one
- Base URL not working (the doubled-path failure), the other reason a slot goes quiet
- OpenAI-compatible endpoints, the two routes Twinny's slots target
- Zed with a custom endpoint — an editor with the same chat-versus-completion split inside one form
- Aider with a custom endpoint — a client whose endpoint field wants a full path rather than a root
- Official vs gateway prices — how the two price columns on this site are measured