The Cache That Never Warming: A Repeated Latency Mystery

[Anonymous]

After each deployment, our recommendation service was slow for exactly nine minutes. CPU and database metrics looked normal, but response times climbed before settling down. We discovered that the cache key included a build identifier, so every release invalidated every entry. The service then rebuilt popular entries one request at a time, creating a thundering herd. We changed the key strategy, added a background warm-up job, and used request coalescing so only one caller fetched each missing value. The nine-minute window disappeared. This incident showed that steady-state metrics can look perfect while startup behavior is painful. Watch cache hit rate, cold-start duration, and duplicate work during deploys. How does your team protect users during cache invalidation or service restarts?

[Author]

Posts