Vercel AI SDK with a custom endpoint

The AI SDK ships a default OpenAI provider that is hard-wired to OpenAI's URL. It also ships the factory that builds the same provider around any OpenAI-compatible base URL — one extra import, and every model your gateway exposes becomes usable from generateText and streamText.

Step 1 — build your own provider

// lib/gateway.ts
import { createOpenAI } from '@ai-sdk/openai';

// createOpenAI() — not openai(). The latter is bound to OpenAI's own URL.
export const gateway = createOpenAI({
  baseURL: process.env.GATEWAY_BASE_URL ?? 'https://aicomp.ai/v1',
  apiKey: process.env.GATEWAY_API_KEY ?? '',
  // 'compatible' relaxes strict-mode checks some OpenAI-compatible
  // routes trip over. Drop it if your endpoint is strictly OpenAI-shaped.
  compatibility: 'compatible',
});

// Model IDs are strings, not enums — anything your endpoint accepts works.
export const cheap = gateway('deepseek-v4-flash');
export const strong = gateway('deepseek-v4-pro');

This is the whole trick. Everything after this point is ordinary AI SDK code.

Step 2 — a plain generation

import { generateText } from 'ai';
import { gateway } from './lib/gateway';

const { text, usage } = await generateText({
  model: gateway('deepseek-v4-pro'),
  prompt: 'Explain what this regex does, in one paragraph.',
  maxOutputTokens: 400,
});

console.log(text);
// usage.promptTokens / usage.completionTokens are your own cost basis —
// log them, then multiply by your rates rather than trusting a library table.
console.log(usage.promptTokens, usage.completionTokens);

Step 3 — streaming from a route handler

// app/api/chat/route.ts
import { streamText } from 'ai';
import { gateway } from '@/lib/gateway';

export async function POST(req: Request) {
  const { messages } = await req.json();

  const result = streamText({
    model: gateway('deepseek-v4-flash'),
    messages,
    system: 'Answer concisely. No preamble.',
  });

  return result.toDataStreamResponse();
}

Step 4 — tool calling

Tool calling rides the same wire format, so it works through the custom provider. The thing people forget is maxSteps: without it the model produces a tool call and stops, because acting on the result is a second step.

import { generateText, tool } from 'ai';
import { z } from 'zod';
import { gateway } from './lib/gateway';

const { text, toolResults } = await generateText({
  model: gateway('deepseek-v4-pro'),
  prompt: 'What is the weather in Taipei?',
  tools: {
    weather: tool({
      description: 'Get current weather for a city',
      parameters: z.object({ city: z.string() }),
      execute: async ({ city }) => ({ city, celsius: 28 }),
    }),
  },
  maxSteps: 3,   // allow the model to act on the tool result
});

console.log(text, toolResults);

curl https://aicomp.ai/v1/models \
  -H "Authorization: Bearer sk-your-gateway-key"

How this fails in practice

What it costs at production volume

A single-user demo costs nothing. The number that matters is a day of real agent traffic — here priced at 200k input and 60k output tokens:

FAQ

Related

• All tools with one endpoint — the matrix
• LiteLLM: same idea in Python, plus fallbacks
• Node SDK: baseURL vs baseUrl
• Python SDK: the same three values
• Log usage yourself instead of trusting a price table
• Full price index

Get API access