Langflow with a Custom OpenAI-Compatible Endpoint
Langflow is a visual builder for agents and RAG flows, and until recently it had no general-purpose slot for a custom endpoint. The community worked around it by rewriting component code or misusing the local-model components. Version 1.11 added the real thing: an OpenAI Compatible provider that takes a base URL and a key, then discovers the model list from the endpoint itself.
That is the good news. The two things that go wrong are both version-dependent, and neither one produces an error message that names the actual cause.
If you are on 1.11 or later, use Settings → Model Providers → OpenAI Compatible. On anything older there is no such provider, and you have to pick between two older routes. Choosing the wrong route for your version is the first failure; a doubled /v1 is the second.
https://aicomp.ai/v1).
Create one free →
First, find out which route your version has
| Your version | What to use | Why |
|---|---|---|
| 1.11+ | Model Providers → OpenAI Compatible | Native. Validates and discovers models. |
| older | the vLLM component | Despite the name it is a general OpenAI-compatible client with api_base, api_key, model_name |
| older | OPENAI_API_BASE environment variable | Rewrites every LangChain-based OpenAI component at once |
The OpenAI Compatible provider ships with the standard install. There is no separate bundle to add.
The provider takes one endpoint, and that is a hard limit
An OpenAI Compatible provider holds exactly one base URL. If you need two custom endpoints at the same time, this provider cannot express it, and no amount of adding fields changes that.
The documented way out is to leave one of them on the built-in OpenAI provider and override its base URL, or to use the vLLM bundle for the second. Both work. What does not work is adding a second OpenAI Compatible provider and expecting the two to coexist.
For a single gateway this is a non-issue. It only bites when you are comparing two endpoints inside one flow, which is the situation this site's whole subject matter sits in.
The doubled version segment
Langflow appends /chat/completions when it calls the model. Whether it also appends /v1 depends on the version, and that is the trap.
Look at the component's default value before you type. If the shipped default already ends in /v1, your value must not repeat it. Writing https://gw.example.com/v1 into a field whose default was https://api.openai.com/v1 is correct; writing it into a field that was blank and gets /v1 appended produces https://gw.example.com/v1/v1.
The result is a 404 from your gateway with a path in it that no route matches. Langflow surfaces this as a connection or model error rather than as a bad URL, so the doubled segment is rarely the first thing you check.
The safe reading: your base URL is a root, and everything after the version segment is Langflow's business. If you pasted a URL that ended in /chat/completions, trim back to the version segment.
SSRF protection blocks localhost by default
This one catches almost everyone running a local server, and the error does not mention SSRF.
Langflow ships with server-side request forgery protection enabled. Loopback and private addresses are refused: localhost, 127.0.0.1, ::1, and private ranges. Saving http://localhost:8000/v1 fails even though the server is running and reachable from your shell.
Allowlist the host before saving:
export LANGFLOW_SSRF_ALLOWED_HOSTS=localhost
# or an address / range
export LANGFLOW_SSRF_ALLOWED_HOSTS=127.0.0.1
export LANGFLOW_SSRF_ALLOWED_HOSTS=10.0.0.0/8A hostname and an IP are different entries. Allowlisting localhost does not allowlist 127.0.0.1, so a value that still fails after the first export is usually addressed by the other form.
What saving actually does
Saving the provider is not a config write. Langflow calls the endpoint's /v1/models route to validate the connection, and the discovered models populate the Language Models and Embedding Models groups.
Two consequences follow. First, an endpoint that does not serve /v1/models fails at save time even though completions would have worked. Second, models appear as a list you have to enable; a discovered model that is not enabled is not selectable, and the symptom is an empty model picker in a node that otherwise looks configured.
Environment variables, if you prefer them to the UI:
export OPENAI_COMPATIBLE_BASE_URL=https://gw.example.com/v1
export OPENAI_COMPATIBLE_API_KEY=sk-your-gateway-keyThe key is genuinely optional. Local servers that authenticate nothing can leave it empty.
Where the tokens go
A Langflow flow is not one request per user message, and the two shapes differ enough to matter.
A RAG flow embeds the query, retrieves chunks, and sends the question plus every retrieved chunk plus the chat history. Raising the retrieved-chunk count from 4 to 8 roughly doubles a payload that was already bigger than the question.
An agent flow re-sends the entire conversation on every tool call, with the tool result appended. One user message can become several full-context requests.
So input scales with what you retrieve and how many tools fire, not with what the user typed. Output is one answer or one structured result per turn, which is small in comparison.
A day on this page's workload is 220k input and 55k output tokens, or 4.4M input and 1.1M output across 20 working days. Retrieval depth is the cheapest lever on that number.
See output vs input pricing for the other half of the arithmetic, and prompt caching savings for why a stable system prompt plus a stable retrieval order changes the bill at this shape.
| Model | Gateway rate in / out per 1M tokens | One day | 20 days |
|---|---|---|---|
| gpt-5.6-terra | $1.00 / $6.00 | $0.55 | $11.00 |
| claude-sonnet-5 | $1.00 / $5.00 | $0.49 | $9.90 |
| deepseek-v4-pro | $0.66 / $1.98 | $0.25 | $5.08 |
| qwen3.8-max | $1.00 / $3.00 | $0.39 | $7.70 |
| glm-5.3 | $0.70 / $2.2 | $0.28 | $5.50 |
One day = 220k input + 55k output on this page's workload. 20 days = 4.4M input and 1.1M output. Rates checked 2026-10-14, gateway rates move with upstream promotions, so verify the current number in your dashboard before committing to a budget.
Verifying the route
- Check your version first. 1.11+ has the OpenAI Compatible provider; older versions do not, and no menu search will find it.
- Look at the default in the Base URL field. If it already ends in
/v1, do not add another one. - Allowlist localhost before saving if the endpoint is local, in both the hostname and IP forms.
- Confirm the endpoint serves
/v1/models. Validation calls it, so an endpoint without it fails at save. - Enable the discovered models. Discovery is not selection.
- Run one component alone before the whole flow. Langflow's playground runs a single component, which separates an endpoint problem from a wiring problem.
FAQ
FAQ
Why does Langflow reject http://localhost:8000/v1 when the server is running?
SSRF protection blocks loopback addresses by default. Set LANGFLOW_SSRF_ALLOWED_HOSTS to the hostname or the IP before saving the provider.
Why do I get a 404 with /v1/v1 in the path?
Your value contains the version segment that Langflow appends itself. Trim the base URL back to the version root and let Langflow add the operation path.
Can I add two custom endpoints?
Not through this provider, which holds one endpoint. Put the second on the built-in OpenAI provider with an overridden base URL, or use the vLLM bundle component.
Do I need an API key for a local endpoint?
No. The key field is optional, and leaving it empty is correct for a server that authenticates nothing.
Why is my discovered model missing from the node dropdown?
Discovery and enablement are separate steps. Open the provider, find the model under Language Models or Embedding Models, and enable it.
Does a Langflow flow cost more than the same conversation in a chat client?
Yes, on the input side. RAG flows send every retrieved chunk each turn and agent flows resend the full conversation per tool call, so the bill follows retrieval depth and tool count rather than message length.
Related
- Flowise with a custom endpoint — a low-code canvas where the node you pick decides whether a Base URL field exists
- Dify with a custom endpoint, the third of the canvas tools, with a provider list instead of a free-text field
- Base URL not working (the doubled path and trailing slash failures), in general form
- RAGFlow with a custom endpoint, the same version-drift problem, with a different default
- OpenAI-compatible error codes (what 401), 404 and 429 mean with a gateway in the middle
- Tools that take one endpoint — the full matrix