Batch API pricing vs realtime gateway rates
Batch halves the list price in exchange for a 24-hour delivery window. That is a real saving — but before you give up sub-second responses for it, it is worth checking whether your gateway's realtime rate already sits at the batch level.
The two discounts that exist
Most vendors publish the same two discounts, and they stack. They are worth knowing because between them they can take the effective input rate down by an order of magnitude:
- Batch — roughly half the list rate. Both input and output are quoted at about 50% of list. You submit a file of requests and collect results later, within a 24-hour window. No streaming, and each request is independent.
- Cached input — about 10% of the input rate. Prompt tokens the model has recently seen bill at roughly a tenth. No flag to set, but it only applies to a prefix that is byte-for-byte identical.
- Together — about 5% of the input rate on the cached portion of a batch job, because the two multiply.
| Model | Standard list in / out | Batch (50%) in / out | Cached input 10% of input | Batch + cached 5% of input |
|---|---|---|---|---|
| gpt-5.6-sol | $4 / $20 | $2 / $10 | $0.4 | $0.2 |
| gpt-5.6-terra | $2 / $12 | $1 / $6 | $0.2 | $0.1 |
| gpt-5.6-luna | $0.2 / $1.2 | $0.1 / $0.6 | $0.02 | $0.01 |
Discount multipliers are the published list-price policies of the relevant vendors, checked 2026-09. List prices above are the official figures and move without notice — check the vendor's own page before budgeting.
The question nobody asks: is batch even cheaper than what you already pay?
The 50% figure always gets quoted against list price. But if you buy through a gateway, your realtime rate is already below list — so the honest comparison is gateway realtime vs official batch, not list vs batch. That is the number that decides whether waiting 24 hours actually saves you anything.
| Model | Gateway realtime in / out | Official batch (50%) in / out | Gateway vs batch in / out — negative means the gateway costs more | Worth waiting 24h? |
|---|---|---|---|---|
| gpt-5.6-sol | $2.5 / $15 | $2 / $10 | -25% / -50% | Official batch is cheaper |
| gpt-5.6-terra | $1 / $6 | $1 / $6 | +0% / +0% | Identical — no reason to wait 24h |
| gpt-5.6-luna | $0.1 / $0.6 | $0.1 / $0.6 | +0% / +0% | Identical — no reason to wait 24h |
What batch cannot do
The discount has hard limits, and they rule out most interactive work:
| Constraint | What it means in practice |
|---|---|
| No streaming | You get the complete response or nothing. Useless for chat UIs and anything typing to a user. |
| 24-hour window | Typically much faster, but you cannot rely on it being fast. |
| Requests are independent | No multi-turn conversation inside one job. Agent loops — where each step depends on the previous result — cannot be batched. |
| Immutable once submitted | You can cancel a job, not edit it. |
The workloads that fit are the ones nobody is waiting on: nightly classification, evaluation suites, embedding backfills, document summarisation in bulk, report generation. In practice teams find a surprising share of their spend sitting in work that could have been submitted overnight and never had to be real-time.
If you also want caching to apply
Caching is prefix-dependent: the prompt must be identical from position zero up to the cached length. That single constraint determines whether you get the discount or not.
- Stable content first — system prompt, tool definitions, retrieved documents.
- Variable content last — user message, current question.
- No timestamp, request ID or random ordering inside the prefix. A single differing character kills the match.
For workloads with a long fixed system prompt and a short variable tail — RAG with a shared knowledge base, agents with a stable tool schema — this alone cuts input cost substantially with no code change.
What this means if you buy through a gateway
One thing to be explicit about: the gateway's price list exposes a single input rate and a single output rate per model. There is no separate batch tier and no cached-input tier visible in its catalogue. Whatever batch and caching discounts exist upstream, you should budget against the two rates you can actually see.
That is not necessarily a bad outcome — the table above shows why. Budget against real numbers rather than assumed discounts, and re-check both sides when either moves.
FAQ
FAQ
How much does the Batch API actually save?
It replaces the list rate with half of it on both input and output. The trade is delivery: results come back within a 24-hour window and there is no streaming, so anything user-facing cannot use it.
Do batch and prompt caching stack?
Yes. Cached input bills at roughly 10% of the standard input rate, and batch halves rates again, so a batch job that reuses a cached prefix can pay about 5% of list on the cached portion. It only applies to the part of the prompt that is byte-identical and already seen.
Is the gateway cheaper than batch?
It depends on the model, and it is worth checking per model — the table above compares them at current rates. On some models the gateway's realtime rate is effectively the same as the official batch rate, which means paying nothing in latency to get the batch-level unit price. Check the current number before deciding, because both sides move.
Does the gateway offer batch or cached-input pricing?
No. Its price list exposes a single input and a single output rate per model, with no separate batch or cached tier. Budget against those two numbers rather than assuming a further discount applies.
When is batch the wrong choice?
Real-time chat, coding assistants, agent loops where each step depends on the previous one, and anything with a user waiting. Batch requests are also independent — there is no multi-turn within a single batch job.