Setup zookeeper

Log replication for mesos with Zookeeper

Zookeeper is a tool to solve log replication, maintaining states of the system to elect mesos leaders for good availability.

Why do we need Zookeeper when Consul can also do log replication, service discovery, leader election?

Mesos did not want to implement yet another paxos algorithm to solve their leader elections. Another reason mesos was born before Consul so they choose to tie up tightly with zookeeper.

We use exhibitor as its a supervisor to zookeeper instance which checks zookeeper and provides a rest interface.

We use container pilot for .... it was too big so created a subpage.

In the code repository cloned you will have a zookeeper folder which has the following scripts too.

DATACENTER=dc1
HOSTNAME=$(hostname -f)

TODO zookeeper is holding data in docker host volumes, change it to s3 bucket

sudo tee /etc/docker/exhibitor <<-EOF
CONSUL_IP=localhost
DC=$DATACENTER
EXHIBITOR_HOSTNAME=$HOSTNAME
EOF

# creating directories for zookeeper to store information
sudo mkdir -p /opt/zk/transactions /opt/zk/snapshots
sudo chmod -R 777 /opt/zk

Execute the exhibitor docker container. Reference : Dockerfile

docker run --restart=always -d --name=exhibitor --net=host --env-file=/etc/docker/exhibitor -v /opt/zk/transactions:/opt/zk/transactions -v /opt/zk/snapshots:/opt/zk/snapshots flyinprogrammer/exhibitor

Check installation

# http://<mesos-master-public-ip>:8500/ui/#/dc1/services
# You should see 3 services registered with consul; consul, zookeeper, exhibitor

Play around !