What is the use of Zookeeper in Storm?

Data Science, AI and ML
#1

The cluster is coordinated by Storm using Zookeeper. Because Zookeeper isn’t utilised for message forwarding, the strain Storm exerts on it is minimal. Single-node Zookeeper clusters should enough in most instances, however bigger Zookeeper clusters may be required if you require failover or are building large Storm clusters. The instructions for installing Zookeeper may be found here. Observations on Zookeeper deployment:

  • Because Zookeeper is fail-fast and will quit the process if it detects any fault condition, it’s important that you run it under supervision. For more information, go here.

  • Setting up a cron to compress Zookeeper’s data and transaction logs is important. The Zookeeper daemon does not perform this on its own, and Zookeeper will rapidly run out of disc space if you do not set up a cron.