Use rsync to efficiently synchronize data between two Linux servers
Efficiently Synchronize Data Between Two Linux Servers Using rsync
In daily operations, it’s common to synchronize files or directories between multiple servers—whether for backing up critical data, deploying static assets, or maintaining consistency between development and production environments. rsync is a powerful and reliable tool for this purpose. This article provides a detailed guide on using rsync to securely and efficiently sync data between two Linux servers.
What is rsync?
rsync (remote sync) is a command-line utility for synchronizing files and directories locally or remotely. It uses a “delta-transfer algorithm” that only transfers the differences between source and destination, drastically reducing bandwidth usage and sync time. rsync also supports compression, permission preservation, symbolic link handling, and resumable transfers.
Prerequisites
Assume we have two servers:
- Source Server: 192.168.1.10, with data at
/data/www - Destination Server: 192.168.1.20, syncing to
/backup/www
Both run Linux (e.g., CentOS, Ubuntu) and have rsync and openssh-server installed.
If rsync isn’t installed:
- Ubuntu/Debian:
sudo apt install rsync- CentOS/RHEL:
sudo yum install rsync
Synchronization Architecture
graph TD
Source[Source Server 192.168.1.10] -->|rsync over SSH| Destination[Destination Server 192.168.1.20]
SSH Key-Based Authentication Setup Flow
For automated sync, configure SSH key-based authentication from the source to the destination server.
graph TD
A[Generate SSH key pair] --> B[Copy public key to destination]
B --> C[Test passwordless login]
C --> D{Login successful?}
D -- Yes --> E[Proceed with rsync setup]
D -- No --> F[Check authorized_keys permissions]
F --> B
Automated Sync Execution Flow
sequenceDiagram
participant Admin as Sysadmin
participant Src as Source Server
participant Dst as Destination Server
Admin->>Src: Create sync script sync-www.sh
Admin->>Src: Configure cron job
Src->>Dst: Automatically run rsync daily at 2 AM
Dst-->>Src: Return sync result
Src->>Admin: Log to /var/log/rsync-www.log
Step 1: Set Up SSH Passwordless Login (Recommended)
Enable key-based SSH authentication from source to destination:
# Run on source server
ssh-keygen -t rsa -b 4096
ssh-copy-id user@192.168.1.20
ssh user@192.168.1.20 # Should connect without password
Step 2: Basic rsync Command
rsync -avz /data/www/ user@192.168.1.20:/backup/www/
Note: The trailing
/in the source path determines whether you sync the directory contents or the directory itself.
Step 3: Common Options
| Option | Description |
|---|---|
-a |
Archive mode (preserves permissions, timestamps, etc.) |
-v |
Verbose output |
-z |
Compress during transfer |
--delete |
Delete extraneous files on destination |
--exclude='*.log' |
Exclude log files |
-P |
Show progress + resume partial transfers |
Step 4: Automation Script + Cron
Create script /usr/local/bin/sync-www.sh:
#!/bin/bash
rsync -avz --delete --exclude='*.log' /data/www/ user@192.168.1.20:/backup/www/ >> /var/log/rsync-www.log 2>&1
echo "[$(date)] Sync completed." >> /var/log/rsync-www.log
Schedule with cron:
0 2 * * * /usr/local/bin/sync-www.sh
Conclusion
Combining rsync + SSH + cron gives you a lightweight, reliable, and automated cross-server sync solution. With Mermaid diagrams, the entire workflow becomes visually clear—ideal for team documentation and knowledge sharing.
Tip: For real-time sync, consider pairing rsync with
inotifywaitto monitor file changes.