The system was slow. But the dashboards looked healthy. We had CPU utilization under 50%. Memory was stable. Error rates were below 1%. Our monitoring alerts weren’t triggering. And yet customers were complaining about slowness. This was the contradiction we…
We had a system that looked healthy. Application CPU was reasonable. Memory was stable. The number of application instances had increased. The load balancer was distributing traffic normally. And yet response times kept getting worse. The first instinct was to…
We had a performance problem. One of the most frequently accessed pieces of data was being read far more often than it changed. The database was handling a large number of repeated reads. The obvious solution was caching. We introduced…
We had a performance problem. One part of the application was doing too much work during a synchronous request. The obvious solution was to move the work out of the request path. Instead of: we changed the flow to: The…
A downstream service started timing out. So we did what seemed like the responsible thing. We added retries. The thinking was straightforward. If a request fails because of a temporary network problem or a momentary downstream slowdown, try again. The…
We had a performance problem. So we did what seemed like the obvious thing. We added more application capacity. The system got slower. At first, the numbers didn’t make sense. The system was serving traffic across multiple regions, and workload…
