FastDFS Cluster Practice: 3-Node Tracker/Storage + Nginx

TL;DR

Scenario: Deploy tracker, storage, and Nginx on three hosts (h121/122/123) with round-robin upload.
Conclusion: Works with store_lookup=0 and multiple tracker_server entries; key is Nginx+mod_fastdfs routing and monitoring verification.
Output: Reusable cluster config checklist + version matrix + common errors quick reference for deployment and regression.

Version Matrix

Item/Feature	Verified Description
Multi-Tracker (h121/h122/h123) parallel	client.conf/storage.conf both configured with three tracker_server entries, supports routing and fault tolerance.
store_lookup=0 round-robin	Fits multi-storage assembly scenario, default is 0, consistent with config.
Storage three nodes (same group)	Connected and monitorable per examples; sync status based on fdfs_monitor.
Nginx + mod_fastdfs	mod_fastdfs.conf tracker list and Nginx routing config provided.
fdfs_monitor observation and inspection	Runtime screenshots shown, usable for ACTIVE/SYNCING/DISCONNECTED determination.

1. FastDFS Cluster Introduction

1.1 Basic Concepts

FastDFS (Fast Distributed File System) is an open-source lightweight distributed file system developed by Alibaba senior architect Yu Qing. It mainly solves the problems of large-capacity file storage and high-concurrency access, especially suitable for small and medium-sized internet enterprises.

1.2 Core Components

FastDFS cluster consists of two core components:

Tracker Server: Responsible for scheduling and managing the entire file system, recording status information of storage servers. Usually uses primary-backup architecture to ensure high availability. A cluster can deploy multiple tracker servers for load balancing.
Storage Server: Responsible for actual file storage, organized by groups. Each group contains multiple storage servers, using redundant backup mechanism to ensure data security.

1.3 Main Features

High Performance: Uses memory indexing for fast file positioning
High Scalability: Supports online expansion without affecting existing services
High Availability: No single point of failure, automatic failover
Load Balancing: Supports multi-tracker and multi-storage load balancing
File Deduplication: Same file stored only once, saving space

2. Architecture Setup

We need three servers, three trackers and three storages.

h121: tracker + storage group + nginx
h122: tracker + storage group + nginx
h123: tracker + storage group + nginx

3. Cluster Configuration

3.1 Tracker Cluster

vim /etc/fdfs/tracker.conf

Modify configuration:

# 0 round-robin
# 1 specified group
# 2 prefer more remaining storage space
store_lookup=0

h121 h122 h123 all need configuration (default value is 0)

3.2 Storage Cluster

vim /etc/fdfs/storage.conf

Write the following content:

tracker_server = h121.wzk.icu:22122
tracker_server = h122.wzk.icu:22122
tracker_server = h123.wzk.icu:22122

h121 h122 h123 all need configuration.

Then restart tracker and storage:

/usr/bin/fdfs_trackerd /etc/fdfs/tracker.conf restart
/usr/bin/fdfs_storaged /etc/fdfs/storage.conf restart

3.3 FastDFS Access Principle

In FastDFS access mechanism, tracker servers are logically equal, but there is a Leader election mechanism during actual operation.

3.4 Test Upload

vim /etc/fdfs/client.conf

Modify content:

tracker_server = h121.wzk.icu:22122
tracker_server = h122.wzk.icu:22122
tracker_server = h123.wzk.icu:22122

Upload file:

/usr/bin/fdfs_upload_file /etc/fdfs/client.conf ~/wzk.png

4. Nginx Integration

vim /etc/fdfs/mod_fastdfs.conf

Modify content:

tracker_server = h121.wzk.icu:22122
tracker_server = h122.wzk.icu:22122
tracker_server = h123.wzk.icu:22122

5. Error Quick Reference

Symptom	Root Cause	Diagnosis	Fix
Upload error `connect to tracker timeout / Connection refused :22122`	Tracker list/port unreachable, DNS unresolved	`telnet h121 22122`, `dig h121.wzk.icu`	Check port connectivity, fix DNS resolution or tracker config
Download 404 (Nginx hit but file not retrieved)	URL prefix and group mapping inconsistent	Check Nginx logs and mod_fastdfs.conf	Unify prefix, verify `group_name`, `url_have_group_name`
`fdfs_monitor` shows node long-time DISCONNECTED	storage process down/disk full/permission or network issue	`systemctl status fdfs_storaged`, `df -h`	Clear disk, fix permissions; restart storage
New storage sync very slow or stuck	Time drift, bandwidth bottleneck, slow I/O	`date -R` time sync, `iftop`	NTP time sync; rate limit during peak hours/offline pre-warming