Why Local Counters Lie to You
Most rate limiting tutorials show you a token bucket running inside a single process, and it works perfectly. Then you deploy three instances behind a load balancer and the whole thing quietly falls apart. Each instance keeps its own counter in memory, completely unaware of what the other two are doing.
Say your limit is 100 requests per minute per user. With three instances round-robining traffic, each one can independently decide the user is under the limit and let 100 requests through. The user just got 300 requests through a system that promised 100. Nobody's code was wrong on its own — the counters were just never talking to each other.
This is the actual problem with distributed rate limiting. It's not which algorithm you pick. It's getting every instance to agree on one shared number, in real time, without corrupting it under concurrent load.
The Race Condition Hiding in Check-Then-Increment
Moving the counter into Redis feels like the fix, and it mostly is, but the naive version still has a race condition baked in. A common first attempt looks like this:
- Read the current count for a key.
- Compare it against the limit.
- If under the limit, increment and allow the request.
The gap between step 1 and step 3 is where things go wrong. Two requests can both read the count as 99, both decide they're under the limit of 100, and both increment — now you're at 101 and you let in one more request than allowed. Under low traffic this rarely matters. Under a burst, with dozens of requests hitting the same key within milliseconds, this compounds fast.
Redis gives you INCR, which is atomic on its own, so a common pattern is:
count = INCR(key)
if count == 1:
EXPIRE(key, window_seconds)
if count > limit:
reject request
This looks safe because INCR itself can't race. But the EXPIRE call is a separate round trip. If the process crashes, or the network hiccups, between the INCR and the EXPIRE, that key never gets a TTL. It sits in Redis forever, and that user is rate limited permanently until someone manually clears the key. This is a real failure mode people hit in production, not a theoretical edge case.
Making It Atomic with Lua
The fix is to stop making two separate calls and instead push the logic into Redis itself as a single atomic unit. Redis runs Lua scripts as one uninterruptible operation — no other command executes in the middle of it, no matter how many API server instances are hammering Redis at once.
A minimal version looks like this:
local current = redis.call("INCR", KEYS[1])
if current == 1 then
redis.call("EXPIRE", KEYS[1], ARGV[1])
end
if current > tonumber(ARGV[2]) then
return 0
end
return 1
You load this once with SCRIPT LOAD and call it with EVALSHA from every API instance. Because the increment, the TTL-setting, and the limit check all happen inside Redis's single-threaded execution model, there's no window for another instance to sneak in a conflicting write. This is the actual mechanism that makes distributed rate limiting safe — not the choice between token bucket and leaky bucket, but pushing the read-modify-write cycle into somewhere atomic.
If you're already using Redis for caching layers in your API, this same discipline applies — the redis caching strategy for paginated api results runs into similar staleness and race concerns whenever multiple writers touch the same key.
Sliding Windows and the Clock Drift Problem
Fixed windows have a well-known boundary issue — a user can burst at the end of one window and the start of the next and effectively double their limit. Sliding window logs fix this using a Redis sorted set: each request gets ZADD-ed with a timestamp as its score, old entries get trimmed with ZREMRANGEBYSCORE, and ZCARD gives you the count inside the window.
Here's where the multi-server angle bites again. Whose timestamp do you use as the score? If each API server stamps the request with its own local system clock, and your servers aren't perfectly synced — which they never are, NTP drift of a few hundred milliseconds is normal, not exotic — you get inconsistent windows. A server with a slightly fast clock can push entries that look like they're from the future relative to Redis's own trimming logic, and two servers can genuinely disagree on whether a given request falls inside or outside the window.
The fix is boring but important: don't trust the API server's clock at all. Call Redis's own TIME command from inside the Lua script and use that as the timestamp source. Every instance ends up scoring against the same clock, because there's only one clock involved — Redis's. This removes an entire class of bugs that only shows up under real multi-node conditions and is nearly impossible to reproduce on a single dev machine.
Fixed Counter vs Sorted Set: The Real Trade-off
| Approach | Memory cost | Accuracy | Redis load | |---|---|---|---| | Fixed counter + TTL | O(1) per key | Allows boundary bursts | Very low, one INCR per request | | Sorted set sliding log | O(requests in window) | Precise, no boundary burst | Higher, ZADD + ZREMRANGEBYSCORE + ZCARD per request |
Most teams land on a hybrid: a sliding window counter that keeps two fixed windows and weights them, giving you sliding-window accuracy without storing every individual timestamp. It's a reasonable middle ground when your traffic volume makes sorted sets expensive to maintain per key.
When Redis Becomes the New Bottleneck
Once every request from every instance has to round-trip to Redis before it can proceed, you've added a new dependency to your hot path. If your API servers are scaling out under a Kubernetes HPA, each new pod opens its own connection pool to Redis, and under a traffic spike you can exhaust Redis's max client connections before you exhaust anything else. The same connection pooling discipline you'd apply to a database — sized pools, sensible timeouts, reuse instead of open-per-request — matters just as much here, similar to how pgbouncer manages connection pooling for Postgres.
There's also a hot key problem if you're on Redis Cluster. Rate limit keys are usually per-user or per-IP, and a single aggressive client hammers one key on one shard — you can't split a single key's traffic across the cluster no matter how many shards you add. Some teams mitigate this with a coarse local pre-check on each instance (reject obviously way-over-limit traffic without touching Redis at all) and only hit Redis for requests near the boundary, cutting Redis calls significantly without sacrificing correctness.
What Happens When Redis Goes Down
You need a decision made ahead of time, not during an incident: fail open or fail closed. Fail closed means if Redis is unreachable, you reject requests — safe for your backend, painful for users during a Redis blip. Fail open means you let requests through with no limiting — safe for user experience, risky for whatever you were trying to protect in the first place.
A practical middle ground is a local, conservative fallback limiter per instance that kicks in only when Redis calls start timing out, wrapped in a circuit breaker so you're not adding latency to every request while Redis is down. It won't be perfectly accurate across instances, but an approximate limit beats either extreme during an outage.
FAQ
Should each API server keep any local rate limit state at all? Yes, but only as a cheap pre-filter or as a fallback during Redis outages, never as the source of truth. A local check can reject requests that are wildly over any reasonable limit without a network call, saving you Redis load, but the actual accept/reject decision near the limit boundary has to go through the shared atomic counter.
How do I avoid adding Redis latency to every single request?
Batch what you can with pipelining, keep persistent connections instead of opening new ones per request, and use EVALSHA instead of resending the full Lua script body every call. If your limits are generous, the local pre-check approach described above cuts a large chunk of Redis round trips for traffic that's clearly nowhere near its limit.
Do I need Redis Cluster for this, or is a single instance enough? A single well-resourced Redis instance handles a surprisingly high volume of rate limit checks since the operations are tiny and fast. Move to Redis Cluster only when you've measured actual throughput limits or need the availability guarantees, and if you do, plan for the hot key problem on your highest-traffic rate limit keys ahead of time.