API rate limits explained — and what to do when you hit one
Rate limits are not a single number. They are enforced on three axes at once, per model, and most "we are being throttled" incidents are actually unbounded parallelism or missing backoff rather than a ceiling that is too low.
The three axes
| Axis | What it counts | What hits it first |
|---|---|---|
| Requests per minute | Number of API calls | Chatbots, autocomplete, high-frequency polling |
| Input tokens per minute | Tokens you send | Agent loops, long-context applications |
| Output tokens per minute | Tokens generated | Long-form generation, batch summarisation |
Your real ceiling is whichever axis you reach first. An application that sends few but very large requests will hit the token limit long before the request limit, which is why "but we only make 20 calls a minute" is not a defence.
What the response tells you
Rate limit headers come back on responses and tell you how much quota remains on each axis. Log them — it turns a mysterious throttling incident into a number you can act on:
import requests, time, random
def call_with_backoff(payload, max_retries=6):
delay = 1.0
for attempt in range(max_retries):
r = requests.post("https://aicomp.ai/v1/messages", headers=HEADERS, json=payload, timeout=60)
if r.status_code != 429:
return r
retry_after = float(r.headers.get("retry-after", delay))
time.sleep(retry_after + random.uniform(0, 0.3)) # jitter
delay *= 2
raise RuntimeError("rate limited after retries")Respect retry-after when it is present; fall back to exponential backoff when it is not.
Add jitter so parallel workers do not retry in lockstep.
The fix most people actually need
Before requesting a higher limit, check concurrency on your side. Fanning out hundreds of parallel requests against a per-minute token budget will trip the limit no matter how high the ceiling is. A small semaphore usually resolves it:
from concurrent.futures import ThreadPoolExecutor
with ThreadPoolExecutor(max_workers=8) as ex: # bound the parallelism
results = list(ex.map(process_one, items))Reducing tokens also raises throughput
Under a token-per-minute cap, sending less per request means getting through more requests. Compacting conversation history and trimming irrelevant context is therefore a throughput fix as well as a cost fix — see how to reduce Claude API cost.
Errors you will see
| Status | Meaning | Action |
|---|---|---|
| 429 | Rate limit reached | Back off and retry with jitter |
| 529 | Upstream overloaded | Same retry logic; not caused by your quota |
| 500 | Server error | Retry with backoff; alert if persistent |
| 400 | Invalid request | Do not retry — fix the payload |
Rate-limit failures and what each one is actually telling you
| Symptom | What is actually happening | How to confirm | Fix |
|---|---|---|---|
| 429s arrive in bursts, not under steady load | Limits are usually per-minute and per-token; a batch job can exhaust a minute's allowance instantly. | Plot requests per minute alongside the errors, bursts align with job starts. | Spread batch work with a rate limiter instead of firing it concurrently. |
| Retries make it worse instead of better | Immediate retries consume the same window that just refused you, so every retry deepens the hole. | Watch whether error rate rises after you add retries. | Back off exponentially with jitter, and honour any retry-after value returned. |
| One service starves another | Several services share one key, so one noisy caller consumes the shared allowance. | Split usage by model and by time; a single caller's spike will be visible. | Issue a separate key per service, which also makes spend attributable. |
| Limits hit sooner than the request count suggests | Long-context requests consume a token budget that is separate from the request count. | Compare input tokens per request against the errors, not just the request rate. | Trim context, or throttle on tokens rather than on requests. |
| Streaming connections appear to leak slots | A stream holds a connection for its whole duration, occupying concurrency that request-count limits do not model. | Count concurrent open streams at the moment errors begin. | Cap concurrent streams and close them explicitly on error paths. |
| The documented limit does not match what you observe | Limits vary by model and by account tier, and are not uniform across a catalogue. | Test one model at a time to isolate which limit you are actually hitting. | Treat published numbers as a starting point and measure your own ceiling. |
When this is the wrong move
Three fixes that make rate limiting worse, and one that does not address the cause
| Situation | Why it breaks | Do this instead |
|---|---|---|
| You are about to raise concurrency to fix a 429 | Adding callers to a saturated window converts throttling into a queue that hides the real limit. | Reduce concurrency, then add back slowly while watching the error rate. |
| You plan to retry indefinitely | Unbounded retries convert a rate limit into an unbounded bill. | Cap attempts, back off, and surface the failure to a queue you can replay later. |
| You are sharing one key across environments | Production and development compete for the same allowance, and neither is attributable. | Separate keys per environment before you tune anything else. |
| You are about to switch models to escape a limit | Limits are per model and per tier; a different model has a different limit, not a larger one. | Identify which axis you are hitting first (switching models does not change your request rate). |
FAQ
What are the Claude API rate limits?
They are enforced on three axes at once, requests per minute, input tokens per minute and output tokens per minute And they are set per model rather than per account. Your effective ceiling is whichever axis you hit first, and it changes as your usage tier changes.
How do I know which limit I hit?
The error response names the limit, and the response headers carry the remaining quota for each axis. Log those headers on every request; guessing is what makes rate limit incidents drag on.
What is the correct way to handle a 429?
Exponential backoff with jitter. Retrying immediately in a tight loop makes the problem worse and can extend the throttling window. Cap the total retry time and surface a clear error after that.
Does a rate limit mean I should switch model?
No. Switching changes the per-token cost, not the limit you are hitting, and different models have different ceilings. Fix the retry logic first, then reduce tokens per request if you are still constrained.
How do I get a higher limit?
Request a tier increase from the provider with real projected volume. In the meantime, concurrency control on your side is the practical lever, most self-inflicted 429s come from unbounded parallelism, not from the ceiling being too low.