redis caching strategy for paginated and filtered api results

Why List Endpoints Break the Standard Caching Playbook

Most caching tutorials show you how to cache a single record: user:123 goes in, user:123 gets invalidated when the user updates their profile. Clean, simple, done.

Paginated and filtered API results don't work that way. A request like /api/products?category=electronics&sort=price_asc&page=2 doesn't map to one entity. It maps to a computed view across potentially thousands of rows, and that view can be invalidated by a write to almost any one of them. This is where most cache-aside implementations start leaking stale data or ballooning in memory, and it's the part nobody covers when they just explain cache-aside versus write-through in the abstract.

The core problem has three parts: how you name the cache key, how long you keep it, and how you get rid of it when the underlying data changes. Get any one of these wrong and you either serve stale pages to users or you defeat the purpose of caching by invalidating too aggressively.

Designing Composite Cache Keys

Every unique combination of filters, sort order, and page number needs its own cache entry, because each one returns a genuinely different payload. The naive approach is to just use the raw query string as the key. Don't do this.

The problem is that ?category=electronics&sort=price_asc and ?sort=price_asc&category=electronics are the same logical query but different strings. If you cache on the raw query string, you'll end up with duplicate cache entries for identical data, wasting memory and quietly lowering your hit rate.

The fix is to normalize before hashing:

• Sort query parameters alphabetically before building the key.
• Lowercase and trim filter values so Electronics and electronics hit the same cache entry.
• Hash the normalized parameter set (SHA-1 or MD5 is fine, this isn't a security context) to keep key length predictable.

A practical key format looks like this:

products:v3:list:page=2:size=20:hash=a1f9c3e2

Where hash is generated from the sorted, normalized filter and sort parameters. The v3 segment is a version number for the whole products resource — that's the piece that makes invalidation manageable, and we'll get to why in a minute.

Keep the key structure human-readable where possible. When you're debugging a production issue at 2am, products:v3:list:page=2:... tells you instantly what's cached. A pure hash of everything tells you nothing.

Why Write-Through Doesn't Map Cleanly to List Endpoints

The classic write-through pattern says: when the primary database updates, immediately update the cache too. That works fine for user:123 — one write, one cache key to refresh.

It falls apart for filtered lists because a single write can affect an unknown number of cached pages. Update the price on one product and suddenly the sort=price_asc page 1, page 2, and page 3 caches for three different category filters might all be stale simultaneously. You can't realistically write-through update every page and filter permutation that record might appear in — you don't even know all of them without recomputing the queries.

This is why list and filter caches almost always end up using cache-aside with short TTLs rather than write-through, even in systems that use write-through for individual entity lookups. The two patterns coexist in the same application, just applied to different data shapes.

Version-Based Invalidation Instead of Pattern Deletion

The naive way to invalidate stale list caches is to run SCAN with a pattern like products:list:* and delete everything that matches. Avoid this in production. SCAN is non-blocking compared to KEYS, but it's still an O(n) operation over your keyspace, and on a busy Redis instance with millions of keys it adds latency and load exactly when you're trying to process a write.

A cleaner approach is a version counter stored as a separate key, incremented on any write that could affect the resource:

INCR products:version

Every list cache key includes the current version at generation time, as shown above with v3. When a write happens, you increment the counter and every previously generated key becomes instantly and permanently unreachable — nobody will ever request products:v3:... again once the version moves to v4. You don't delete anything. The old entries just sit there until their TTL expires or Redis evicts them under memory pressure.

This trades memory for speed. Invalidation becomes a single INCR, O(1), regardless of how many pages or filter combinations exist. The cost is that stale data lingers in Redis memory a bit longer than with explicit deletion, so pair this with a reasonably short TTL — don't rely on version bumps alone to keep memory bounded.

An alternative is tag-based invalidation: maintain a Redis set per resource (tag:products) containing every cache key that was generated from a products query, then on write, pull the set members and delete them explicitly. This gives you precise, immediate invalidation with no lingering stale entries, but it costs an extra write to the tag set on every cache population and an O(n) delete on every invalidation. Use version bumping when writes are frequent and you want cheap invalidation; use tag sets when staleness for even a few seconds is unacceptable.

TTL Tuning: Lists Need Shorter Lifetimes Than Records

Single-entity caches can often afford longer TTLs because you have a clean invalidation path — update the entity, update or clear its key. List and filter caches don't have that luxury, so lean on short TTLs as your safety net rather than your primary strategy.

A reasonable starting point:

• Individual entity lookups (product:456): TTL in the range of tens of minutes to a few hours, since these can be write-through refreshed directly.
• Paginated/filtered list results: TTL in the range of 30 to 120 seconds, relying on version bumps for the fast path and TTL expiry as the fallback.
• Aggregate counts (total results, total pages): cache alongside the list data they describe, never separately, or you'll end up with a page count that doesn't match the actual paginated results.

That last point catches a lot of teams off guard. If your list payload and your total-count payload live in separate cache keys with separate TTLs, you'll eventually serve a response where page 3 exists according to the count but returns an empty array because the underlying data shrank between the two cache expirations.

Handling Cache Stampedes on Popular Filters

Default, unfiltered listing pages — the homepage product grid, the default "all posts" feed — get hit constantly. When that cache key's TTL expires, you can get dozens of concurrent requests all missing at once and hammering the database with the same expensive query simultaneously. This is the classic cache stampede, and it's especially painful on list endpoints because the underlying query (joins, filters, sorting, counting) is usually the most expensive one in the whole API.

Two practical mitigations:

• Lock-based rebuild: before querying the database on a cache miss, attempt SET lock:products:v3:page1 1 NX EX 5. If you get the lock, rebuild the cache. If you don't, wait briefly and retry the cache read instead of hitting the database directly.
• TTL jitter: instead of a flat 60-second TTL, set it to 60 + random(0, 15) seconds. This spreads expiration times across requests instead of letting many keys expire in the same instant, which is common when a bunch of pages get cached around the same time during a traffic spike.

If your database is also under connection pressure from these bursts, it's worth pairing this with proper pgbouncer connection pooling strategies so a stampede doesn't also exhaust your connection limits while the cache rebuilds.

Common Mistakes Worth Calling Out

A few patterns show up repeatedly in real codebases:

• Caching the raw, unsorted query string, doubling memory usage for logically identical requests.
• Using KEYS or unthrottled SCAN for invalidation on a production instance with heavy traffic, causing latency spikes during writes.
• Setting one global TTL for every endpoint regardless of how often the underlying data changes, instead of tuning per resource.
• Forgetting that deep filter forms (multiple checkboxes, ranges, free-text search) create combinatorial key explosion — five filters with four options each is already hundreds of possible cache keys, most of which will only ever be requested once and never again, which is pure cache pollution.

For that last point, consider setting a lower TTL or skipping caching entirely for filter combinations that are unlikely to repeat, like free-text search queries. Cache the common, predictable filter paths — category browsing, default sorts, first few pages — where the hit rate actually pays off, and let rare combinations fall through to the database.

Putting It Together

A workable strategy for most APIs looks like this: version the resource, hash the normalized filter parameters into the key, keep list TTLs short and record TTLs longer, bump the version on write instead of hunting down keys to delete, and add jitter or locking around your hottest unfiltered endpoints. None of this is exotic. It's just the part of cache-aside and write-through that the standard explanations skip over, because the standard explanations are written for single-record lookups, not for the messy, high-cardinality world of real API pagination and filtering.