Use Claude in LangChain with a custom endpoint

LangChain talks to models through a chat model object, so changing where requests go is a constructor argument — not a rewrite of your chains, agents or parsers.

In short: In LangChain, set the endpoint through the chat model's URL parameter rather than monkey-patching the SDK — ChatAnthropic takes anthropic_api_url, ChatOpenAI takes base_url. Everything downstream (chains, agents, memory) keeps working unchanged.

You need a key before the code below runs. Create an account, generate a key, and copy the base URL (https://aicomp.ai/v1). Create one free →

Check current rates → Free to sign up · $1 minimum top-up · No prepayment

Confirm the key works first. One command, no SDK, costs nothing:

curl https://aicomp.ai/v1/models \
  -H "Authorization: Bearer sk-your-gateway-key"

A JSON list of model IDs means the key is good. Invalid token means it was copied wrong.

Anthropic model

from langchain_anthropic import ChatAnthropic

llm = ChatAnthropic(
    model="claude-sonnet-5",
    anthropic_api_url="https://aicomp.ai/v1",   # older versions: base_url
    anthropic_api_key="sk-your-gateway-key",
    max_tokens=1024,
    temperature=0,
)

print(llm.invoke("Say OK").content)

OpenAI-compatible path

from langchain_openai import ChatOpenAI

llm = ChatOpenAI(
    model="gpt-5.6-luna",
    base_url="https://aicomp.ai/v1",
    api_key="sk-your-gateway-key",
)

print(llm.invoke("Say OK").content)

Pick one path, not both. The Anthropic path is the safer choice for Claude models because tool calling and streaming follow the native contract; use the OpenAI path when you want one client across many vendors.

Everything above still works

Once the model object is constructed, nothing downstream changes — chains, agents, memory, retrievers and output parsers all read from it as before:

from langchain_core.prompts import ChatPromptTemplate

prompt = ChatPromptTemplate.from_messages([("human", "{question}")])
chain = prompt | llm
print(chain.invoke({"question": "Summarise rate limiting in one line."}).content)

Streaming and callbacks

for chunk in llm.stream("Explain prompt caching briefly."):
    print(chunk.content, end="", flush=True)

Streaming requires the endpoint to support SSE. If chunks arrive as one block, the endpoint is buffering rather than streaming — functionally fine, but you lose the time-to-first-token benefit.

Cost note. Agent loops are where LangChain bills surprise people. Every step resends the conversation, so a fifteen-step agent run can cost more than a hundred plain calls. Log usage per step — see how to monitor API spend.

Rates

Model rates reachable from LangChain — gateway rate vs official list
Model	Gateway rate in / out per 1M tokens	Official list in / out per 1M tokens	Diff
claude-haiku-4-5	$0.5 / $2.5	$1 / $5	50%
claude-sonnet-5	$1 / $5	$2 / $10	50%
claude-opus-5	$2.5 / $12.5	$5 / $25	50%

Rates checked 2026-09-16. Gateway rates move with upstream promotions — verify the current number in your dashboard before committing to a budget.

Troubleshooting

Symptom	Fix
Unknown field anthropic_api_url	Version mismatch — use base_url on older releases.
404 on messages	Doubled `/v1`. Check what the base URL already ends with.
401 authentication_error	Key not passed to the model object, or overridden by an env var. Pass it explicitly.
Tools not called	Bind tools with `llm.bind_tools(...)` and confirm the endpoint supports tool use.

FAQ

How do I point LangChain at a custom Anthropic endpoint?

Set the URL parameter on the chat model constructor instead of patching the SDK globally. In current LangChain that is anthropic_api_url on ChatAnthropic; some versions call it base_url. Everything downstream keeps working because it talks to the model object, not the client.

Does streaming still work?

Yes, if the endpoint supports server-sent events. Pass streaming=True or use the .stream() method as usual, and test it once before relying on it in production.

Can I use the OpenAI-compatible path instead?

Yes, when the endpoint exposes one. ChatOpenAI with base_url and a non-OpenAI model ID works for many gateways, but tool calling and structured output are the features most likely to diverge — test them explicitly.

Where do costs get out of hand in LangChain?

Agent loops. Each step resends the accumulated context, so a ten-step agent run costs far more than ten individual calls. Log token usage per step or you will not see it coming.

Do I need to change my prompts or parsers?

No. The endpoint change is transparent to everything above the model layer — prompts, output parsers, memory and retrieval behave identically.

Get API access