Download and view it as a Slide show to view animations. This is the powerpoint slide with animations to understand different components of Kafka. Discover the core concepts , architecture and practical use-cases of Kafka.

1. Components of Kafka
By: Arvind Singh Rawat

2. Broker

3. Broker
Place where messages are stored in the actual physical instance
An announcement hall can act as a broker in Real world.

4. Publisher

5. Publisher
System which generates the messages.

6. Consumers

7. Consumers
Consumes the messages delivered by Publisher through Broker.
Like listeners of an announcement in public forum.

8. Message

9. Actual content provided by the Publisher

10. Topics

11. Topics
A Topic is a logical channel or category of messages to which data/messages are published,
like actual category of announcement.

13. Each topic will have its own set of messages, and its own set of publishers and subscribers.

14. Each topic can have multiple partitions and one message can be sent to any one of the partitions.
This will allow parallel message production and consumption.
Partition
A
Partition
B
Partition
C

15. In this case, lets say a Message A is sent to topmost partition.
That message can additionally be replicated for redundancy.
Partition
A
Partition
B
Partition
C

16. Broker 2
In our case, Announcement Hall 2
Broker 1
In our case, Announcement Hall 1
Partitions can be distributed among different Brokers for high availability.

17. Let’s shift our focus to Kafka’s other key feature: Queue.

18. 1, 2, 3!!!
Mic
Check
Hello!
I heard till
Hello!
I heard till
1,2,3!!! I heard till
Mic Check
Kafka maintains the position of a consumer within
a partition of a topic which is called as an Offset.

19. What happens if a Consumer joins after few of the
messages are delivered ?
I am late
!!!

20. Kafka topic retains the message for a
configurable period of time or size and
Consumer can choose to read from the
earliest (all the available messages) or
latest (only messages after it joins)
Hurray !!! I can hear the
older announcements

21. 1. Hello !
2. Mic Check
3. 1, 2, 3 !!!
4. .
5. .
6. .
Historical messages are stored in an
immutable Write-Ahead log.

22. 1. Hello !
2. Mic Check
3. 1, 2, 3 !!!
4. .
5. .
6. .
Hey, I need to drop off, can
you let me know of upcoming
announcements
Yeah, sure !
Consumer Group

23. 1. Hello !
2. Mic Check
3. 1, 2, 3 !!!
4. .
5. .
6. .
Multiple consumers can read from a topic
and thus enables parallel consumption.
Hey, I need to drop off, can
you let me know of upcoming
announcements
Yeah, sure !
Consumer Group

24. 1. Hello !
2. Mic Check
3. 1, 2, 3 !!!
4. .
5. .
6. .
Each consumer group consumes messages
from a single partition, which is not read by
other consumers of same group.
Consumer Group

25. 1. Hello !
2. Mic Check
3. 1, 2, 3 !!!
4. .
5. .
6. .
Even if a consumer leaves the group, other
consumers will take it place to ensure
message consumption.
Consumer Group