TL;DR
- Scenario: Three-node Elasticsearch cluster deployment to achieve high availability and horizontal scaling
- Conclusion: Cluster mode provides data sharding and replica mechanism to ensure high availability
- Output: Complete cluster deployment flow, config explanation, error quick reference
Version Matrix and Environment Prep
| Component | Version |
|---|---|
| Elasticsearch | 7.3.0 (linux-x86_64 tar.gz) |
| Nodes | h121 / h122 / h123 |
| Running User | es_server (non-root) |
Directory and Permission Settings
Execute on Three Machines
mkdir -p /opt/servers/es/{data,logs}
chown -R es_server /opt/servers/es
chown -R es_server /opt/servers/elasticsearch-7.3.0
User Creation and Permissions
useradd es_server
passwd es_server
# Add in /etc/sudoers: es ALL=(ALL) ALL
Core Config elasticsearch.yml
h121 Node Config
cluster.name: wzkicu-es
node.name: h121.wzk.icu
path.data: /opt/servers/es/data
path.logs: /opt/servers/es/logs
network.host: h121.wzk.icu
network.bind_host: h121.wzk.icu
network.publish_host: 114.115.221.144
http.port: 9200
cluster.initial_master_nodes: ["h121.wzk.icu","h122.wzk.icu","h123.wzk.icu"]
discovery.seed_hosts: ["h121.wzk.icu", "h122.wzk.icu","h123.wzk.icu"]
h122 Node Config
cluster.name: wzkicu-es
node.name: h122.wzk.icu
path.data: /opt/servers/es/data
path.logs: /opt/servers/es/logs
network.host: h122.wzk.icu
network.bind_host: h122.wzk.icu
network.publish_host: 114.115.221.145
http.port: 9200
cluster.initial_master_nodes: ["h121.wzk.icu","h122.wzk.icu","h123.wzk.icu"]
discovery.seed_hosts: ["h121.wzk.icu", "h122.wzk.icu","h123.wzk.icu"]
h123 Node Config
cluster.name: wzkicu-es
node.name: h123.wzk.icu
path.data: /opt/servers/es/data
path.logs: /opt/servers/es/logs
network.host: h123.wzk.icu
network.bind_host: h123.wzk.icu
network.publish_host: 114.115.221.146
http.port: 9200
cluster.initial_master_nodes: ["h121.wzk.icu","h122.wzk.icu","h123.wzk.icu"]
discovery.seed_hosts: ["h121.wzk.icu", "h122.wzk.icu","h123.wzk.icu"]
System Parameter Config
Modify sysctl.conf
vim /etc/sysctl.conf
# Add
vm.max_map_count=655360
Apply:
sysctl -p
Modify limits.conf
vim /etc/security/limits.conf
# Add
* soft nofile 65536
* hard nofile 65536
* soft nproc 4096
* hard nproc 4096
JVM Memory Config
Modify jvm.options
vim /opt/servers/elasticsearch-7.3.0/config/jvm.options
# Set
-Xms2g
-Xmx2g
Distribution and Startup
Distribute Config
# Distribute configured ES to each node
scp -r /opt/servers/elasticsearch-7.3.0 es_server@h122.wzk.icu:/opt/servers/
scp -r /opt/servers/elasticsearch-7.3.0 es_server@h123.wzk.icu:/opt/servers/
Start on Each Node
su es_server
/opt/servers/elasticsearch-7.3.0/bin/elasticsearch -d
Verify Cluster Status
Using Elasticsearch Head Tool
Access http://h121.wzk.icu:9200/_plugin/head/ or use curl:
# Cluster health
curl -XGET 'http://h121.wzk.icu:9200/_cluster/health?pretty'
# Node list
curl -XGET 'http://h121.wzk.icu:9200/_cat/nodes?v'
# Shard status
curl -XGET 'http://h121.wzk.icu:9200/_cat/shards?v'
Error Quick Reference
| Symptom | Root Cause | Fix |
|---|---|---|
| vm.max_map_count too low | Kernel parameter not set | Add vm.max_map_count=655360 in /etc/sysctl.conf |
| max file descriptors too low | limits not applied | Set nofile 65536 in limits.conf |
| ES refuses root run | Started as root | Switch to es_server user |
| Node starts but doesn’t join cluster | 9300 not accessible/hostname resolution failed | Check discovery.seed_hosts and network |
| node name already in use | Multiple nodes with duplicate node.name | Assign unique name to each node |
| Cluster Red | Shard allocation failed | Check disk space, node status |
| Split-brain issue | Master node election disagreement | Adjust minimum_master_nodes |