Apache Kafka
In depthPlatform
Also known as: Kafka
Distributed event-streaming platform for publishing, storing and processing huge streams of events in real time.
Apache Kafka, explained
Written by EduVerseWhat it is
Apache Kafka is a distributed event-streaming platform. Services write events, such as “order placed”, to topics, and Kafka stores them in order in an append-only log split into partitions. Other services read those events at their own pace and can replay older ones.
Why teams use it
Kafka lets many services react to the same events without calling each other directly, even at very high volumes. Compared with RabbitMQ: RabbitMQ is a classic broker that routes messages to queues and removes them once they’re processed, while Kafka keeps events for a set period so new consumers can read the history.
An example from work
Order confirmations arrive late and someone mentions consumer lag. You run kafka-consumer-groups.sh with --describe for the order-service group and see the LAG column growing on one partition, because a single consumer is stuck retrying a message it can’t process.
Our own explanation, not a quote from the book.
In the book
Sentences from The Software Realm, Decoded that mention Apache Kafka, exactly as printed.
Peter “I keep hearing about Kafka and RabbitMQ.”
Chapter 8 · 8.4 Kafka vs. RabbitMQ: Recording vs. Mailbox · p. 189Open in the free preview
7 more passages about Apache Kafka in the full book
Read every conversation where Apache Kafka comes up, with the interactive slides and demos.
Where it fits
Event streaming, pub/sub messaging, data pipelines
Coverage in the book
Covered in depth: multiple pages with explanations, examples and simulations.
