rsync Recipes Every Sysadmin Needs
Practical rsync recipes for mirrors, SSH transfers, exclude lists, bandwidth throttling, hard-link snapshots, and safe dry-run verification. Ready to adapt.
Before you start
- ▸rsync installed (apt install rsync / dnf install rsync / pacman -S rsync)
- ▸SSH key-based authentication configured for remote recipes
- ▸Read and write access to both source and destination paths
- ▸Basic familiarity with the Linux command line and file paths
rsync is one of those tools that rewards every hour you invest in learning it. It transfers only changed blocks, preserves permissions and timestamps, works over SSH, and handles everything from laptop backups to multi-terabyte server mirrors. These recipes cover the patterns you will reach for repeatedly, explained with enough detail to adapt them confidently.
Core Flags You Need to Know
Before diving into recipes, understand the flags that appear everywhere:
- -a (--archive): Equivalent to
-rlptgoD. Recurses directories, preserves symlinks, permissions, timestamps, group, owner, and device files. Use this as your default. - -v (--verbose): Shows files being transferred. Add a second
-vfor even more detail. - -z (--compress): Compresses data in transit. Useful over slow or metered links; skip it on gigabit LANs since it burns CPU for no gain.
- --delete: Removes files from the destination that no longer exist in the source. Required for true mirrors.
- -n (--dry-run): Simulates the transfer without touching anything. Always run this first on destructive operations.
- -P: Combines
--progressand--partial, showing per-file progress and resuming interrupted transfers.
Recipe 1: Local Archive Copy
Copy a directory tree to another location on the same machine, preserving everything:
rsync -av /home/alice/projects/ /mnt/backup/projects/
The trailing slash on the source is significant. With a trailing slash, rsync copies the contents of projects/ into the destination. Without it, rsync copies the directory itself, creating /mnt/backup/projects/projects/. The trailing slash on the destination only controls whether rsync creates a subdirectory there, and most people include it for clarity.
Recipe 2: True Mirror with --delete
Make a destination an exact replica of the source, including deletions:
rsync -av --delete /srv/www/ /mnt/mirror/www/
Run a dry-run first when using --delete. A misplaced slash can wipe things you want to keep:
rsync -avn --delete /srv/www/ /mnt/mirror/www/
The output will list what would be transferred or removed. Review it carefully before dropping the -n.
Recipe 3: Sync Over SSH
rsync uses SSH as its transport by default when you provide a remote host. No extra setup is required beyond SSH access:
rsync -avz -e ssh /home/alice/docs/ [email protected]:/home/alice/docs/
The -e ssh flag is technically optional now (SSH is the default transport), but making it explicit lets you pass SSH options:
rsync -avz -e "ssh -p 2222 -i ~/.ssh/backup_key" \
/home/alice/docs/ [email protected]:/home/alice/docs/
For automated backups, set up key-based authentication and restrict the remote key in ~/.ssh/authorized_keys using the command= option to lock it to rsync only.
Pulling from a Remote Host
The source and destination arguments simply swap:
rsync -avz [email protected]:/var/log/app/ /srv/log-archive/app/
Recipe 4: Exclude Files and Directories
Exclude specific patterns inline:
rsync -av --exclude='*.log' --exclude='.git/' /home/alice/projects/ /mnt/backup/projects/
For more than a couple of exclusions, use an exclude file. Create a plain text file with one pattern per line:
cat ~/.rsync-excludes
# Output (edit to suit your needs):
*.log
*.tmp
.git/
.cache/
node_modules/
__pycache__/
*.pyc
Then reference it with --exclude-from:
rsync -av --exclude-from="$HOME/.rsync-excludes" \
/home/alice/projects/ /mnt/backup/projects/
Patterns starting with / are anchored to the transfer root. *.log matches any log file at any depth; /*.log matches only at the top level.
Recipe 5: Bandwidth Limiting
On production systems, unconstrained rsync can saturate a link. Use --bwlimit to cap throughput in KiB/s:
# Limit to 50 MB/s (51200 KiB/s)
rsync -av --bwlimit=51200 /srv/data/ [email protected]:/srv/data/
# Limit to 10 MB/s for a metered link
rsync -avz --bwlimit=10240 /home/alice/ [email protected]:/backup/alice/
Note that --bwlimit sets the average throughput per file, not a strict cap on total socket usage. For stricter shaping, wrap rsync in a tool like trickle or apply traffic control at the network level with tc.
Recipe 6: Dry-Run Before Any Destructive Operation
Make dry-run a habit. Pair it with --itemize-changes (-i) for a detailed change summary:
rsync -avin --delete --exclude-from="$HOME/.rsync-excludes" \
/srv/www/ /mnt/mirror/www/
Each line of output starts with an 11-character status string. A leading > means the file would be sent; *deleting means it would be removed at the destination. Nothing runs until you remove -n.
Recipe 7: Incremental Backup with Hard Links
Using --link-dest, rsync creates a new snapshot directory where unchanged files are hard-linked to the previous backup rather than copied. This gives you full point-in-time restore capability with minimal disk usage:
TODAY=$(date +%F)
LAST=$(ls -1d /mnt/snapshots/????-??-?? 2>/dev/null | tail -1)
rsync -a --delete \
${LAST:+--link-dest="$LAST"} \
/home/alice/ "/mnt/snapshots/$TODAY/"
Each /mnt/snapshots/YYYY-MM-DD/ directory looks like a complete backup but shares unchanged inodes with previous snapshots. Deleting an old snapshot directory does not affect others.
Verification
After a critical sync, verify the result with --checksum mode. This forces rsync to compare MD5/SHA checksums rather than relying on size and modification time:
rsync -acn --itemize-changes /home/alice/projects/ /mnt/backup/projects/
If the output is empty, the two trees are identical at the checksum level. Be aware that --checksum reads every byte of every file on both sides, so it is slow on large datasets.
Troubleshooting
Permission Denied Errors
If rsync fails with permission errors on the remote side, check that the destination directory is writable by the connecting user. On pulls, ensure the source files are readable. When running as a non-root user, rsync cannot preserve ownership (the -o flag inside -a) of files owned by other users; use --no-owner to suppress the error instead of running as root unnecessarily.
"Partial Transfer" Warnings
These appear when a file changes on the source while rsync is reading it (common with live databases or logs). Either pause the application during backup, use application-level snapshots (e.g., mysqldump before rsync), or accept that those files may be inconsistent and re-run rsync immediately after to catch up.
SSH Host Key Checking in Scripts
Automated scripts sometimes fail because the remote host key is not in known_hosts. Do an interactive SSH login once to accept the key, or use the -o StrictHostKeyChecking=accept-new SSH option the first time only. Never use StrictHostKeyChecking=no permanently in production.
rsync Daemon vs SSH Transport
Some setups run rsyncd as a standalone daemon (port 873, paths look like host::module). This is common for anonymous mirrors. For anything involving credentials or sensitive data, always prefer the SSH transport. The daemon mode sends passwords in the clear by default unless wrapped in a VPN or SSH tunnel.
Frequently asked questions
- What is the difference between rsync -a and rsync -r?
- -r recurses into directories only. -a includes -r but also preserves symlinks, permissions, timestamps, group, owner, and device files. Use -a for backups; -r alone is rarely the right choice.
- Why does rsync create a nested directory instead of copying into the destination?
- A missing trailing slash on the source argument causes rsync to copy the directory itself rather than its contents. Compare: rsync -av docs/ /backup/ (copies contents) versus rsync -av docs /backup/ (creates /backup/docs/).
- Is --delete safe to use?
- It is safe when the source and destination are correct. The risk is human error: a wrong source path mirrored to the destination will delete legitimate files. Always run with -n first and consider --delete-delay, which stages deletions until the transfer succeeds, rather than --delete which can delete mid-run.
- Does --bwlimit work for both upload and download?
- Yes, --bwlimit throttles the data rsync itself writes to the socket, so it applies regardless of transfer direction. It is applied per-file, not as a hard socket cap, so brief spikes above the limit are possible.
- Can I resume a failed rsync transfer?
- Use -P (which combines --partial and --progress). --partial keeps partially transferred files at the destination so the next run resumes from where it stopped rather than restarting the file from scratch.
Related guides
Bash Arrays and Associative Arrays
Master bash indexed and associative arrays: declaration, element access, looping, mapfile, namerefs, and practical patterns for real scripting work.
Bash Functions and Variable Scoping
Master Bash function scoping with local variables, source-based libraries, correct use of return codes, and array passing techniques including namerefs.
Bash Loops: for, while and until
Learn all three Bash loop types — for, while, and until — with practical, copy-paste examples covering file iteration, counting, polling, and safe line reading.
Bash Scripting for Beginners
Learn Bash scripting from scratch: shebang lines, variables, conditionals, loops, and arguments, plus a real backup script to tie it all together.