This is article 26 in the Big Data series. Introduces ZooKeeper distributed coordination framework basic concepts, cluster role division, and 3-node deployment practice.
Complete illustrated version: CSDN Original | Juejin
What is ZooKeeper
ZooKeeper is an Apache open-source highly available distributed coordination service, often compared to “caretaker of distributed systems”. It doesn’t store business data, but provides a lightweight and reliable set of coordination primitives, allowing distributed applications to reach consensus on state.
Consistency Guarantees (Five Properties):
- Sequential Consistency: Updates from same client executed in sent order
- Atomicity: Write operations either all succeed or all fail, no intermediate state
- Single View: Client sees consistent data view regardless of which server connected
- Reliability: Once changes are committed, results persist, won’t be lost
- Timeliness: Within certain time window, client eventually sees latest data
Typical Application Scenarios:
- Distributed Naming Service: Like Dubbo service registry, identify service address via ZNode path
- Distributed Configuration Management: Solr cluster sync configuration, trigger Watcher notification to nodes after changes
- Message Queue (Pub/Sub): Use sequential nodes for ordered message distribution
- Distributed Lock: Sequential ephemeral nodes compete for lock, smallest sequence number gets lock
- Cluster Management: Ephemeral node lifecycle bound to session, automatically unregister when session disconnects
Data Model: ZNode Tree
ZooKeeper organizes data in tree structure similar to Unix filesystem, each node called ZNode.
- Paths separated by
/, root is/ - Each ZNode stores up to ~1 MB data (byte array)
- Also saves version number, timestamp, ACL and other metadata
Cluster Roles: Leader / Follower / Observer
| Role | Responsibilities |
|---|---|
| Leader | Handles all write requests, drives ZAB protocol transaction broadcast |
| Follower | Handles read requests, participates in Leader election voting |
| Observer | Only handles read requests, does not participate in voting, used to horizontally scale read capability |
Leader is the sole write entry point, all write operations must go through Leader before syncing to Followers (and Observers).
ZAB Protocol and Odd Number of Nodes
ZooKeeper uses ZAB (ZooKeeper Atomic Broadcast) protocol to ensure data consistency. Transaction commit requires more than half nodes (N/2 + 1) confirmation, which requires odd number of nodes:
| Cluster Size | Tolerable Failed Nodes |
|---|---|
| 3 nodes | 1 |
| 5 nodes | 2 |
| 7 nodes | 3 |
Production recommends 3 nodes (small cluster) or 5 nodes (high availability cluster).
Install ZooKeeper 3.8.4
This experiment uses 3 cloud servers:
| Host | Spec |
|---|---|
| h121.wzk.icu | 2C4G |
| h122.wzk.icu | 2C4G |
| h123.wzk.icu | 2C2G |
1. Download and Extract
Download apache-zookeeper-3.8.4-bin.tar.gz from Apache mirror, extract to /opt/servers/.
2. Create Data and Log Directories
mkdir -p /opt/servers/apache-zookeeper-3.8.4-bin/data
mkdir -p /opt/servers/apache-zookeeper-3.8.4-bin/logs3. Write zoo.cfg
Copy conf/zoo_sample.cfg to conf/zoo.cfg, key configuration as follows:
# Data directory (stores snapshots)
dataDir=/opt/servers/apache-zookeeper-3.8.4-bin/data
# Log directory (transaction logs)
dataLogDir=/opt/servers/apache-zookeeper-3.8.4-bin/logs
# Client connection port
clientPort=2181
# Base tick time (milliseconds)
tickTime=2000
# Follower initial sync timeout (tickTime multiples)
initLimit=5
# Follower and Leader sync timeout
syncLimit=2
# Auto purge snapshots and logs every hour
autopurge.purgeInterval=1
# Cluster node config: server.id=host:quorumPort:electionPort
server.1=h121.wzk.icu:2888:3888
server.2=h122.wzk.icu:2888:3888
server.3=h123.wzk.icu:2888:38884. Configure myid
Create myid file in each node’s data directory with corresponding node ID:
# Execute on h121
echo 1 > /opt/servers/apache-zookeeper-3.8.4-bin/data/myid
# Execute on h122
echo 2 > /opt/servers/apache-zookeeper-3.8.4-bin/data/myid
# Execute on h123
echo 3 > /opt/servers/apache-zookeeper-3.8.4-bin/data/myid5. Start and Verify
Execute on each of three machines:
/opt/servers/apache-zookeeper-3.8.4-bin/bin/zkServer.sh startCheck each node status:
zkServer.sh statusExpected output shows one as Mode: leader, others as Mode: follower.
Summary
ZooKeeper provides strongly consistent coordination capability in distributed environment through ZAB protocol and Leader centralized write model. 3-node cluster is the most common starting configuration. Mastering zoo.cfg core parameters and myid configuration is foundation for cluster operations. Next article will dive into ZNode data structure and Watcher monitoring mechanism.