API rate limits explained — and what to do when you hit one
Rate limits are not a single number. They are enforced on three axes at once, per model, and most "we are being throttled" incidents are actually unbounded parallelism or missing backoff rather than a ceiling that is too low.
The three axes
| Axis | What it counts | What hits it first |
|---|---|---|
| Requests per minute | Number of API calls | Chatbots, autocomplete, high-frequency polling |
| Input tokens per minute | Tokens you send | Agent loops, long-context applications |
| Output tokens per minute | Tokens generated | Long-form generation, batch summarisation |
Your real ceiling is whichever axis you reach first. An application that sends few but very large requests will hit the token limit long before the request limit, which is why "but we only make 20 calls a minute" is not a defence.
What the response tells you
Rate limit headers come back on responses and tell you how much quota remains on each axis. Log them — it turns a mysterious throttling incident into a number you can act on:
import requests, time, random
def call_with_backoff(payload, max_retries=6):
delay = 1.0
for attempt in range(max_retries):
r = requests.post("https://aicomp.ai/v1/messages", headers=HEADERS, json=payload, timeout=60)
if r.status_code != 429:
return r
retry_after = float(r.headers.get("retry-after", delay))
time.sleep(retry_after + random.uniform(0, 0.3)) # jitter
delay *= 2
raise RuntimeError("rate limited after retries")Respect retry-after when it is present; fall back to exponential backoff when it is not.
Add jitter so parallel workers do not retry in lockstep.
The fix most people actually need
Before requesting a higher limit, check concurrency on your side. Fanning out hundreds of parallel requests against a per-minute token budget will trip the limit no matter how high the ceiling is. A small semaphore usually resolves it:
from concurrent.futures import ThreadPoolExecutor
with ThreadPoolExecutor(max_workers=8) as ex: # bound the parallelism
results = list(ex.map(process_one, items))Reducing tokens also raises throughput
Under a token-per-minute cap, sending less per request means getting through more requests. Compacting conversation history and trimming irrelevant context is therefore a throughput fix as well as a cost fix — see how to reduce Claude API cost.
Errors you will see
| Status | Meaning | Action |
|---|---|---|
| 429 | Rate limit reached | Back off and retry with jitter |
| 529 | Upstream overloaded | Same retry logic; not caused by your quota |
| 500 | Server error | Retry with backoff; alert if persistent |
| 400 | Invalid request | Do not retry — fix the payload |
FAQ
What are the Claude API rate limits?
They are enforced on three axes at once — requests per minute, input tokens per minute and output tokens per minute — and they are set per model rather than per account. Your effective ceiling is whichever axis you hit first, and it changes as your usage tier changes.
How do I know which limit I hit?
The error response names the limit, and the response headers carry the remaining quota for each axis. Log those headers on every request; guessing is what makes rate limit incidents drag on.
What is the correct way to handle a 429?
Exponential backoff with jitter. Retrying immediately in a tight loop makes the problem worse and can extend the throttling window. Cap the total retry time and surface a clear error after that.
Does a rate limit mean I should switch model?
No. Switching changes the per-token cost, not the limit you are hitting, and different models have different ceilings. Fix the retry logic first, then reduce tokens per request if you are still constrained.
How do I get a higher limit?
Request a tier increase from the provider with real projected volume. In the meantime, concurrency control on your side is the practical lever — most self-inflicted 429s come from unbounded parallelism, not from the ceiling being too low.