Prompt caching: what a cache hit actually costs

Every article on prompt caching stops at “cached input is about 90% cheaper.” That is true and it is not the number you need. The number you need is: a cache hit costs roughly one fifth of the uncached rate we charge — close to 5× on every model we can check, and 6.25× on gpt-5.6-sol. Whether that is worth restructuring your prompts depends entirely on your hit rate.

curl https://aicomp.ai/v1/models \
  -H "Authorization: Bearer sk-your-gateway-key"

The three rates you are actually billed at

Caching replaces one input rate with three. Vendors have converged on the same shape — a read multiplier of 0.1× base input — but they differ on what you pay to create the cache:

Sources: Anthropic prompt caching, OpenAI prompt caching, Gemini caching.

What a hit costs next to what you already pay

This is the comparison nobody publishes, because it needs both rates at once. The last column is the gateway's uncached realtime input divided by the official cache-hit rate — how many times cheaper a hit is than the call you are making today:

Cache-hit rates are derived from each vendor's published multiplier, not from gateway billing data — see the note on data sources below.

Two things fall out of it. First, the ratio is remarkably stable: 5.00× on six of the seven models. Second, the one model that breaks the pattern, gpt-5.6-sol, does so because its list rate is promotional rather than standard — see why the same model has two prices for the full table.

Where caching stops paying

If your provider charges a write premium, there is a reuse count below which caching loses money. With a write multiplier w and a read multiplier r, the break-even number of reads per write is:

N = (w - r) / (1 - r)

• Anthropic 5-minute tier (w = 1.25, r = 0.1): N = 1.28 — the second read already saves.
• Anthropic 1-hour tier (w = 2.0, r = 0.1): N = 2.11 — you need three reads before the premium pays back.
• No write charge (DeepSeek, OpenAI's automatic mode before GPT-5.6): the formula has no threshold; caching cannot lose money.

The practical risk is not the arithmetic, it is expiry. A session that goes quiet longer than the cache window and then wakes up pays a full write on the entire accumulated context, at a rate above plain input. On a 100K-token session that single turn can cost 12–20× a cache hit. If your traffic is bursty rather than continuous, model that cold start before you commit.

What actually breaks the cache

Matching is exact from the first token, and it is all-or-nothing — there is no partial hit that degrades gracefully. The failures are almost always in the prefix, not the model:

• A date, timestamp or request ID near the front of the system prompt.
• A tool list whose order changes between requests, or which is re-serialised differently.
• Template whitespace injected into an otherwise static block.
• History appended before the static content instead of after it.

The fix is ordering: everything immutable first, a clear breakpoint after the last static block, and conversation history append-only after it.

Marking the breakpoint

import anthropic

client = anthropic.Anthropic()

# 静态前缀（系统提示 / 工具定义 / 参考资料）放前面，
# 动态内容（用户消息、时间戳）一律放后面 —— 顺序错了缓存直接失效。
resp = client.messages.create(
    model="claude-sonnet-5",
    max_tokens=1024,
    system=[
        {
            "type": "text",
            "text": SYSTEM_PROMPT,          # 长且不变的那一段
            "cache_control": {"type": "ephemeral"},
        }
    ],
    messages=[{"role": "user", "content": user_input}],
)

# usage 里能直接读出命中量 —— 这是唯一可靠的验证方式
print(resp.usage.cache_read_input_tokens, resp.usage.cache_creation_input_tokens)

Read cache_read_input_tokens back rather than assuming a hit. It is the only measurement that survives routing differences across providers.

Where these numbers come from

The gateway catalogue publishes input and output rates only — there is no cached or batch rate in it. So the 0.1×, 1.25× and 2× multipliers above are taken from each vendor's published pricing documentation and checked in September 2026; they are not gateway billing data. We do not know what a cached token bills at through the gateway, because that is not published anywhere. Budget on the two rates you can actually see, and if cache economics are material, measure a direct call and compare it with your gateway invoice for the same workload.

FAQ

Related

• Batch pricing vs realtime: whether the 24-hour wait pays
• Why the same model has two prices — all 24 models we can check
• Eight levers for a smaller Claude bill
• How to attribute spend per request
• All Anthropic models compared

Get API access