We originally introduced Kafka for a simple reason. We needed asynchronous communication. A service would publish an event. Another service would consume it. The producer didn’t need to wait. The consumer could process the event independently. It looked straightforward. Kafka…
At first, the API was just an interface. A way for one component to call another. Nothing more. Then the system grew. The monolith became multiple services. Teams became independent. Deployments became more frequent. And suddenly almost every important business…
The transaction used to be simple. At least, it looked simple from the application’s perspective. A customer placed an order. The system processed the payment. The order was created. Inventory was reserved. The ledger was updated. The customer received a…
The system was becoming increasingly difficult to change. Every time we added a new capability, another service needed to know about it. A payment completed. The ledger needed to update. Notifications needed to be sent. Fraud analytics needed the transaction.…
It was 2AM. Most of the engineering team was asleep. The payment system was processing transactions normally. Then an alert fired. Duplicate charges were increasing. Not thousands. Not enough to immediately bring the entire system down. But enough to tell…
A payment system can be healthy in almost every traditional sense. CPU utilization looks reasonable. The database is responding normally. Kafka is processing messages. Network latency is within limits. There are no obvious application errors. And yet payment latency suddenly…
