Vercel AI SDK with a custom endpoint

By the aitokensai editors · Verified 2026-11-09 Sources: AI SDK docs

The AI SDK ships a default OpenAI provider that is hard-wired to OpenAI's URL. It also ships the factory that builds the same provider around any OpenAI-compatible base URL (one extra import), and every model your gateway exposes becomes usable from generateText and streamText.

Step 1 — build your own provider

// lib/gateway.ts
import { createOpenAI } from '@ai-sdk/openai';

// createOpenAI(), not openai(). The latter is bound to OpenAI's own URL.
export const gateway = createOpenAI({
  baseURL: process.env.GATEWAY_BASE_URL ?? 'https://aicomp.ai/v1',
  apiKey: process.env.GATEWAY_API_KEY ?? '',
  // 'compatible' relaxes strict-mode checks some OpenAI-compatible
  // routes trip over. Drop it if your endpoint is strictly OpenAI-shaped.
  compatibility: 'compatible',
});

// Model IDs are strings, not enums (anything your endpoint accepts works).
export const cheap = gateway('deepseek-v4-flash');
export const strong = gateway('deepseek-v4-pro');

This is the whole trick. Everything after this point is ordinary AI SDK code.

Step 2 — a plain generation

import { generateText } from 'ai';
import { gateway } from './lib/gateway';

const { text, usage } = await generateText({
  model: gateway('deepseek-v4-pro'),
  prompt: 'Explain what this regex does, in one paragraph.',
  maxOutputTokens: 400,
});

console.log(text);
// usage.promptTokens / usage.completionTokens are your own cost basis —
// log them, then multiply by your rates rather than trusting a library table.
console.log(usage.promptTokens, usage.completionTokens);

Step 3 — streaming from a route handler

// app/api/chat/route.ts
import { streamText } from 'ai';
import { gateway } from '@/lib/gateway';

export async function POST(req: Request) {
  const { messages } = await req.json();

  const result = streamText({
    model: gateway('deepseek-v4-flash'),
    messages,
    system: 'Answer concisely. No preamble.',
  });

  return result.toDataStreamResponse();
}

Step 4 — tool calling

Tool calling rides the same wire format, so it works through the custom provider. The thing people forget is maxSteps: without it the model produces a tool call and stops, because acting on the result is a second step.

import { generateText, tool } from 'ai';
import { z } from 'zod';
import { gateway } from './lib/gateway';

const { text, toolResults } = await generateText({
  model: gateway('deepseek-v4-pro'),
  prompt: 'What is the weather in Taipei?',
  tools: {
    weather: tool({
      description: 'Get current weather for a city',
      parameters: z.object({ city: z.string() }),
      execute: async ({ city }) => ({ city, celsius: 28 }),
    }),
  },
  maxSteps: 3,   // allow the model to act on the tool result
});

console.log(text, toolResults);

curl https://aicomp.ai/v1/models \
  -H "Authorization: Bearer sk-your-gateway-key"

How this fails in practice

Why changing one URL is enough

An OpenAI-compatible client does not have a provider hardcoded into it. It holds a base URL, and at request time it concatenates that base URL with a resource path — /chat/completions, /models, then sends the same JSON body it would have sent anywhere else, with your key in an Authorization: Bearer header. The base URL is therefore the only thing that decides where the request physically goes.

That is why this change is one line and not a rewrite. Your message array, your tool definitions, your streaming handling and your response parsing are all unchanged, because the wire format is unchanged. What changes is who answers, and what they charge per token. The two values that travel with the new destination are the base URL itself and the API key; the model ID is the third, and it has to be one the new endpoint actually serves, which you can list with a single call to /v1/models rather than guess at.

The practical consequence is that this is reversible. Keep the base URL in configuration rather than inline, and moving between providers is a config change you can undo in a minute — which is also the right way to test it: point a single non-critical call at the new endpoint first, confirm it in the usage log, then move the rest.

What it costs at production volume

A single-user demo costs nothing. The number that matters is a day of real agent traffic — here priced at 200k input and 60k output tokens:

One month at 20 working days, gateway rates checked 2026-11-09. These are the token volumes, not a quote Your own shape may be larger or smaller.

The table above prices one shape, a heavy coding day. This one prices the range, and the point of putting them side by side is that the ordering is not stable across them. A model that wins on output-heavy generation can lose on a workload that is mostly input, because the two directions are billed separately. Pick the model on the row that matches what you actually ship, not on the row that happens to be first.

FAQ

Related

• All tools with one endpoint — the matrix
• LiteLLM: same idea in Python, plus fallbacks
• Node SDK: baseURL vs baseUrl
• Python SDK: the same three values
• Log usage yourself instead of trusting a price table
• Full price index

Get API access