API Rate Limiting With Token Buckets and Redis

An API limit should protect shared capacity without turning normal bursts into unexplained failures. A dashboard may load several resources at once, while a background integration sends steady traffic all day. Both can have the same average request rate and very different instantaneous demand. Choosing the policy requires more than adding a counter to middleware.

Separate rate, quota, and concurrency controls

A rate limit controls arrivals over time. A quota limits total consumption within a business period, such as monthly document conversions. A concurrency limit bounds work currently executing. Expensive requests may need all three because a low request rate can still exhaust database connections when each operation runs for a long time.

Use the API Latency Tester to inspect a permitted request and a rejected request against a safe test endpoint. Check their statuses and observed timing. This is useful for individual response inspection, not proof of capacity or a substitute for an authorized load test.

Define a policy in plain language first: an authenticated account may make a steady two requests per second, with a burst capacity of ten. Those numbers are illustrative, not universal recommendations. Choose actual values from downstream capacity, operation cost, and acceptable customer behavior. Document limits where integration developers can find them before deployment.

Choose an algorithm that matches the promise

A fixed window is simple but permits large bursts across a boundary: a caller can consume one window's allowance just before reset and the next immediately afterward. A sliding log is precise but stores more per-request data. A sliding counter approximates smoother behavior with less storage.

A token bucket maintains available tokens and a last-refill timestamp. Tokens accumulate at a defined rate up to a maximum capacity. A request spends tokens if enough remain. The capacity explicitly permits bursts while the refill rate controls sustained throughput.

elapsed = max(0, now - last_refill)
available = min(capacity, stored_tokens + elapsed * refill_per_ms)

if available >= request_cost:
  allow
  stored_tokens = available - request_cost
  retry_after = 0
else:
  deny
  stored_tokens = available
  retry_after = ceil((request_cost - available) / refill_per_ms / 1000)

last_refill = now

This is algorithm pseudocode, not a complete distributed implementation. Validate that refill rate is positive and request cost does not exceed capacity, or a request may never become eligible. Fractional tokens can work, but fixed-point arithmetic may simplify reproducibility across runtimes.

Make distributed decisions atomic

With several API instances, local counters multiply the effective allowance and produce inconsistent results. A shared store such as Redis can coordinate decisions, but reading a value and writing it later introduces races. Perform refill, admission, update, and expiry as one atomic operation.

Redis scripting documentation explains how server-side scripts execute atomically. Keep scripts short because execution blocks other activity on the server. Use a consistent time source, and account for clock movement rather than trusting unrelated application clocks to agree perfectly.

Expire inactive buckets only after they would be fully replenished. Deleting state too soon creates free bursts; never expiring it can accumulate unused identities indefinitely. In clustered Redis, keys used together must satisfy the platform's key-placement rules.

Decide what happens when the limiter fails. Fail-open preserves availability but loses protection. Fail-closed protects constrained dependencies but can turn a Redis incident into an API outage. Some systems use a conservative local fallback. Choose separately for sensitive operations and cheap reads, then exercise the failure path instead of leaving it as an undocumented accident.

Identify callers without creating unfairness

Authenticated limits should usually use an account or application identity rather than an IP alone. A company office, university, or carrier network can place many legitimate people behind one public address. Conversely, a malicious client can rotate addresses or create accounts to evade naive policies.

Combine layered limits where needed: a coarse unauthenticated perimeter, an account budget, and a stricter budget for expensive endpoints. Keep the number of rules understandable so callers can predict the effective policy. Password reset and login endpoints may require additional abuse controls beyond general API limits.

Only trust forwarded IP headers from explicitly configured proxies. Otherwise a caller may choose its own apparent identity. Avoid storing raw personal identifiers in limiter keys when a stable internal identifier or carefully managed pseudonym will do.

Consider cost-weighted tokens for operations such as batch imports, but publish enough detail that clients can budget. Reject unbounded batch sizes before calculating admission. Rate limiting does not replace authorization or payload limits; the OWASP API Security project provides broader guidance on resource-consumption risks.

Return actionable responses and test boundary cases

Return 429 Too Many Requests for a caller exceeding policy. Include a useful body explaining which documented limit applies, without leaking other tenants' activity. MDN's Retry-After reference describes its delay-seconds and HTTP-date forms. Delay-seconds is often easier when clients have imperfect clocks.

Calculate retry delay from the actual deficit rather than always returning sixty seconds. Clients should wait at least that long and may add jitter to avoid synchronized retries. Retrying a write still requires a separate duplicate-prevention strategy; receiving a transient failure does not automatically make every operation safe to replay.

Format representative error bodies with the JSON Formatter, and include policy examples in your backend documentation. Avoid claiming that response headers reserve future capacity: other requests from the same account can consume tokens before the next attempt arrives.

Test simultaneous arrivals, idle refill, exact capacity, oversized costs, expired keys, store outages, and clock anomalies. Monitor allowed and denied traffic by bounded policy identifier, plus limiter latency. A protective layer that adds excessive delay or rejects ordinary workloads needs adjustment even if its counter arithmetic is correct.

Calculate a burst by hand before load testing

Take the illustrative bucket with capacity ten and refill rate two tokens per second. Ten simultaneous unit-cost requests can pass from a full bucket. An eleventh request at that same instant lacks one token, so it becomes eligible after half a second if no other request consumes the refill. With integer delay-seconds, round the advertised wait upward to one second.

After three idle seconds from an empty bucket, six tokens are available. A four-token export can pass, leaving two; another four-token export must wait one second. Writing these examples into deterministic tests catches unit conversion errors between seconds and milliseconds and confirms that weighted requests use the same budget consistently.

Use a controllable clock in algorithm tests rather than real waiting. Advance time to just before, exactly at, and just after eligibility. Then separately test atomic execution with concurrent callers, because correct single-threaded arithmetic does not prove the shared-store implementation is race-free.

Key takeaways

• Match token capacity to acceptable bursts and refill rate to sustainable work.
• Update shared state atomically and define datastore-outage behavior explicitly.
• Combine fair caller identities with operation-specific resource protections.
• Return meaningful retry guidance and test races, boundaries, and failure paths.