What role does ZooKeeper play in a cluster of Kafka?

Data Science, AI and ML
#1

Apache ZooKeeper is a naming registry for distributed applications as well as a distributed, open-source configuration and synchronisation service. It maintains track of the Kafka cluster nodes’ status, as well as Kafka topics, partitions, and so on.

Because the data is distributed throughout ZooKeeper’s groups of nodes, it has a high level of availability and consistency. ZooKeeper conducts an immediate failover migration when a node fails.

In Kafka, ZooKeeper is used to manage service discovery for the Kafka brokers that make up the cluster. When a new broker joins, a broker dies, a topic is deleted, or a topic is created, ZooKeeper notifies Kafka, so that each node in the cluster is aware of the changes. As a result, it gives you a real-time view of the Kafka cluster setup.