ZFS Snapshots and Send/Receive

ZFS snapshots are instantaneous, space-efficient, and crash-consistent. Combined with zfs send and zfs receive, they form a bulletproof replication pipeline that beats rsync for datasets where consistency matters. This guide walks through creating and managing snapshots, building incremental replication streams to a remote host, and automating retention with zfs-auto-snapshot.

How ZFS Snapshots Work

A snapshot captures the state of a dataset at a single point in time using copy-on-write. When you write new data, the original blocks are preserved for the snapshot; the snapshot itself costs only metadata until blocks are modified. Snapshots are read-only, cannot be accidentally overwritten, and are addressable as pool/dataset@snapshotname.

Because snapshots record the exact list of on-disk blocks at a moment in time, zfs send can serialize that block list into a byte stream. The receiving end reconstructs the dataset exactly — checksums and all — making this fundamentally different from file-level tools.

Prerequisites and Setup

You need ZFS installed on both source and destination hosts. On Debian/Ubuntu, ZFS on Linux ships in the official repos:

sudo apt install zfsutils-linux

On Fedora / RHEL / Rocky, enable the ZFS repository first:

sudo dnf install https://zfsonlinux.org/fedora/zfs-release-2-5$(rpm --eval "%{dist}").noarch.rpm
sudo dnf install zfs
sudo modprobe zfs

On Arch:

sudo pacman -S zfs-utils   # requires archzfs or AUR; follow archzfs repo setup first

Verify your pool is healthy before doing anything else:

sudo zpool status

Creating Atomic Snapshots

A single dataset snapshot:

sudo zfs snapshot tank/data@2025-07-18_manual

Snapshot an entire pool hierarchy recursively with -r. This is atomic across all child datasets:

sudo zfs snapshot -r tank@2025-07-18_pre-upgrade

List snapshots:

zfs list -t snapshot -o name,creation,used,refer -s creation

Output will look similar to:

# NAME                              CREATION              USED  REFER
# tank/data@2025-07-18_manual       Fri Jul 18 09:00 2025   0B  4.20G
# tank@2025-07-18_pre-upgrade       Fri Jul 18 09:01 2025   0B  1.10G

Naming convention matters for automation. Use ISO-style timestamps (YYYY-MM-DD_HHMM) or a prefix that sorts lexicographically, so scripts can identify the oldest snapshot reliably.

Rolling Back and Destroying Snapshots

Roll a dataset back to a snapshot. This destroys any snapshots taken after the target unless you use -r to confirm:

sudo zfs rollback tank/data@2025-07-18_manual

Destroy a specific snapshot when you no longer need it:

sudo zfs destroy tank/data@2025-07-18_manual

Destroy a range of snapshots (ZFS 0.8+):

sudo zfs destroy tank/data@snap1%snap5   # destroys snap1 through snap5 inclusive

Replication with zfs send / zfs receive

Full Initial Send

Send a complete snapshot to a remote host over SSH. The -R flag includes all child datasets and their properties; -p preserves properties alone if you don't need recursion:

sudo zfs send -R tank/data@2025-07-18_manual \
  | ssh backup-host sudo zfs receive -F backup/data

For large datasets, pipe through mbuffer to absorb network jitter and get transfer stats:

sudo zfs send -R tank/data@2025-07-18_manual \
  | mbuffer -s 128k -m 1G \
  | ssh backup-host "mbuffer -s 128k -m 1G | sudo zfs receive -F backup/data"

Incremental Replication

After the initial send, you only need to ship the difference between two snapshots. Create a new snapshot on the source:

sudo zfs snapshot tank/data@2025-07-19_daily

Send the increment from the previous snapshot to the new one:

sudo zfs send -R -i tank/data@2025-07-18_manual tank/data@2025-07-19_daily \
  | ssh backup-host sudo zfs receive -F backup/data

The -i flag means "from this snapshot" (one specific parent). Use -I instead to include all intermediate snapshots if you missed a transfer cycle — this keeps the remote side's history intact.

Important: The parent snapshot referenced by -i must exist on both sides. If the remote is ever out of sync, use zfs list -t snapshot backup/data on the destination to confirm which snapshot to use as the base.

Scripted Nightly Replication

A minimal systemd service + timer pair is the right way to schedule this on modern systems. Create the script at /usr/local/bin/zfs-replicate.sh:

#!/usr/bin/env bash
set -euo pipefail

DATASET="tank/data"
REMOTE="backup-host"
REMOTE_DS="backup/data"
SNAP_NEW="${DATASET}@$(date +%Y-%m-%d_%H%M)"
SNAP_PREV=$(zfs list -t snapshot -H -o name -s creation "${DATASET}" | tail -2 | head -1)

zfs snapshot "${SNAP_NEW}"
zfs send -R -i "${SNAP_PREV}" "${SNAP_NEW}" \
  | ssh "${REMOTE}" zfs receive -F "${REMOTE_DS}"

sudo chmod +x /usr/local/bin/zfs-replicate.sh

Create the systemd unit at /etc/systemd/system/zfs-replicate.service:

[Unit]
Description=ZFS incremental replication to backup-host
After=network-online.target

[Service]
Type=oneshot
ExecStart=/usr/local/bin/zfs-replicate.sh
User=root

And the timer at /etc/systemd/system/zfs-replicate.timer:

[Unit]
Description=Run ZFS replication nightly

[Timer]
OnCalendar=*-*-* 02:00:00
Persistent=true

[Install]
WantedBy=timers.target

sudo systemctl daemon-reload
sudo systemctl enable --now zfs-replicate.timer

Automated Retention with zfs-auto-snapshot

zfs-auto-snapshot creates and expires snapshots on a configurable schedule. It integrates with cron or systemd and is available on all major distros.

# Debian/Ubuntu
sudo apt install zfs-auto-snapshot

# Fedora / RHEL — install from EPEL or clone from GitHub
sudo dnf install epel-release && sudo dnf install zfs-auto-snapshot

# Arch (AUR)
yay -S zfs-auto-snapshot

After installation, it ships cron jobs in /etc/cron.d/zfs-auto-snapshot and /etc/cron.{hourly,daily,weekly,monthly}/. The defaults keep 4 frequent (15-min), 24 hourly, 31 daily, 8 weekly, and 12 monthly snapshots.

Control which datasets participate with the com.sun:auto-snapshot property:

# Enable on a dataset
sudo zfs set com.sun:auto-snapshot=true tank/data

# Disable for a child dataset you want to exclude (e.g., a scratch volume)
sudo zfs set com.sun:auto-snapshot=false tank/scratch

To override retention per schedule, pass flags directly. For example, to keep only 7 daily snapshots on a specific dataset:

sudo zfs-auto-snapshot --keep=7 --label=daily --quiet tank/data

If you prefer systemd timers over cron, disable the cron jobs and create timers following the same pattern shown in the replication section above, calling zfs-auto-snapshot as the ExecStart.

Verification

Confirm replication arrived intact by comparing the most recent snapshot name on both sides:

# Source
zfs list -t snapshot -H -o name -s creation tank/data | tail -1

# Destination (run on backup-host or via SSH)
ssh backup-host zfs list -t snapshot -H -o name -s creation backup/data | tail -1

For a deeper check, compare the written and used properties, or use zfs diff between snapshots to inspect exactly what changed:

zfs diff tank/data@2025-07-18_manual tank/data@2025-07-19_daily

Troubleshooting

• "cannot receive incremental stream: most recent snapshot does not match incremental source" — The remote is ahead of or behind your chosen base. Run zfs list -t snapshot on both sides and find a common snapshot to use as the -i argument.
• Send is unexpectedly large — Check whether the dataset has dedup enabled on the source but not the destination; dedup ratios don't transfer. Also verify you're using -i (single increment) not -I (all intermediates) unless you need history replay.
• zfs-auto-snapshot not creating snapshots — Confirm com.sun:auto-snapshot=true is set and cron / the timer is active: systemctl list-timers or grep auto-snapshot /var/log/syslog.
• Permission denied on receive — The remote user needs zfs allow delegation or must run as root. Grant delegated permissions with sudo zfs allow -u backupuser receive,create,mount backup.