How to monitor AI API spend before it becomes a surprise
Token spend accrues invisibly inside loops. By the time the invoice arrives you cannot tell which feature caused it Which is why the fix is not a fancier dashboard, it is logging usage per request.
Three levels of monitoring
| Level | Tells you | Limitation |
|---|---|---|
| Provider dashboard | Accurate totals, per model | No attribution to features or customers |
| Per-request logging | Which feature and model drove it | Requires one wrapper in your code |
| Threshold alerts | That something changed today | Only useful if the baseline is right |
Most teams have the first and assume it is enough. It is not, it tells you the bill went up, not why.
The one wrapper that does most of the work
import time, json
def tracked_call(client, model, feature, **kwargs):
t0 = time.time()
resp = client.messages.create(model=model, **kwargs)
usage = resp.usage
log({
"ts": time.time(),
"feature": feature,
"model": model,
"input_tokens": usage.input_tokens,
"output_tokens": usage.output_tokens,
"latency_ms": int((time.time() - t0) * 1000),
})
return respLog input_tokens and output_tokens separately. Output costs several times more
per token, so a blended figure hides exactly the signal you need.
Turn tokens into money
RATES = { # USD per 1M tokens (keep in config, not in code
"claude-sonnet-5": (1).0, 5.0),
"claude-haiku-4-5": (0.5, 2.5),
}
def cost_of(model, in_tok, out_tok):
i, o = RATES.get(model, (0, 0))
return in_tok / 1_000_000 * i + out_tok / 1_000_000 * oCheck the current numbers on the price index, rates move, so hard-coding them in your billing code is how cost reports silently drift out of reality.
Alert on change, not on totals
A fixed threshold ("alert above $500/month") only fires when it is already expensive. Alert on day-over-day change instead (a 3x jump on a Tuesday is the shape of a bug), and it is worth knowing about even when the absolute number is small.
What to review weekly
- Cost per completed task, not cost per token — the only number that reflects value
- Which model drove the most spend, and whether that matches where the value is
- Any feature whose tokens per task are trending up — usually creeping context
- Retry volume: retries are spend that produced nothing
How the bill is actually computed
Two numbers are metered separately and added: input tokens (everything you send, system prompt, conversation history, retrieved documents, tool results) and output tokens (what the model generates). Per-token, output is the expensive one (across the 226 models we track), the median output-to-input ratio is 4.0×, and the widest is 12.0×.
And yet on agent-shaped workloads the input side dominates the invoice. That is not a contradiction, it is arithmetic: an agent re-sends its entire context on every turn, so a turn that produces 800 output tokens may carry 60,000 input tokens. At a 4× unit price ratio, the input still accounts for roughly 95% of that turn's cost. Optimising output length on this workload is optimising the wrong end.
| Model | Input / 1M | Output / 1M | Output ÷ input | One agent turn | Input share of that turn |
|---|---|---|---|---|---|
claude-haiku-4-5-20251001 | $0.500 | $2.50 | 5.0× | $0.032 | 94% |
claude-sonnet-5 | $1.00 | $5.00 | 5.0× | $0.064 | 94% |
claude-opus-5 | $2.50 | $12.50 | 5.0× | $0.160 | 94% |
The practical reading: unit price decides, but token shape decides harder. A model that costs twice as much per token can still be cheaper end to end if it needs half as many turns to finish the task, and a prompt that carries 40k tokens of irrelevant history costs the same as one that carries 40k tokens of useful context. Before switching models, measure which side of the sum your workload actually lives on.
Monitoring blind spots that cost real money
| Symptom | What is actually happening | How to confirm | Fix |
|---|---|---|---|
| Daily totals look fine but the month is over budget | Averaging hides spikes; one runaway session can dominate a month while every daily number looks acceptable. | Group spend by hour, then by session (spikes stop hiding once the bucket is smaller). | Alert on the session level, not only on the daily total. |
| Attribution by service is impossible | Several services share one key, so the usage log shows one undifferentiated caller. | Try to attribute last week's spend to a specific service, if you cannot, this is the cause. | Issue a key per service and per environment before you need the answer. |
| Recorded tokens differ from what the client reported | Retries, cancelled streams and partial responses all bill differently from a clean call. | Compare client-side counts against server-side counts for one known workload. | Treat the server-side number as the source of truth and reconcile the difference. |
| Streaming calls show zero usage | Usage is often delivered in a final chunk that a broken handler never reads. | Log every chunk including the last, and check for a usage field. | Enable usage-in-streaming where it is offered, and read the final chunk explicitly. |
| Cost per task drifts upward with no code change | Context grew (more files in scope, longer history, larger retrieved documents). | Track input tokens per task over time as its own metric. | Alert on tokens per task, not just on total spend. |
| An alert fires after the money is already spent | Billing data lags real usage, sometimes by hours. | Compare the timestamp of the alert against the timestamp of the spike. | Meter tokens at the call site and alert there, using billing only for reconciliation. |
FAQ
Why is API spend so hard to control?
Because it is invisible by default. Unlike a subscription, cost accrues per token inside a loop, and by the time the invoice arrives you cannot tell which feature caused it. Logging usage per request is the only fix.
What should I log?
Input tokens and output tokens separately, plus model, feature or endpoint name, and a request ID. Separate is important: output costs several times more, so a blended number hides the thing you most need to see.
How do I catch a runaway bill early?
Alert on day-over-day change rather than a fixed threshold. A 3x jump on a Tuesday matters even if the absolute number is still small, that is the shape of a bug, not of growth.
Is the provider dashboard enough?
It gives accurate totals but weak attribution. You will see the bill went up, not that it was the new extraction job on model X. Per-request logging is what turns a total into an answer.
What is the cheapest way to start?
Wrap your client in one function that logs usage, and aggregate into a daily table. That single change covers most of what teams need before they reach for a dedicated observability tool.
How do I set up token usage alerts?
Alert on a rate, not a balance. A spend threshold tells you the money is already gone; a threshold on tokens-per-hour or cost-per-request catches the loop while it is still small. Emit both from the same log line you already write, and make the alert fire on the second consecutive breach so a single unusual batch job does not page anyone.
How do I attribute API spend to a user or a feature?
Attach an identifier at the call site and log it alongside usage.prompt_tokens and usage.completion_tokens. Multiplying those by your own rates at log time (rather than trusting a provider's cost field) is the only way the number stays correct when you route through a gateway, because the gateway's rate is not what the upstream cost map assumes.
Related
- Reduce cost — eight levers
- Batch discounts vs gateway rates
- Prompt caching — per-model hit rates
- Why one model carries two prices
- Rate limits explained
- n8n workflows — where unattended runs turn a cheap call into a real bill
- Cost calculator
- Current price index
- What it costs when a session runs out of context