Sanoid / Syncoid

Tools
- sanoid – a policy-driven snapshot management tool for ZFS filesystems.
- syncoid – a tool to facilitates the asynchronous incremental replication of ZFS filesystems.
  - SystemD service and timer

Tools

sanoid – a policy-driven snapshot management tool for ZFS filesystems.

Installation (Ubuntu)

2021-11-28: It seems Ubuntu packages are badly lagging behind so git it and generate the .deb package is better.

https://github.com/jimsalterjrs/sanoid/blob/master/INSTALL.md#debianubuntu

sudo cp /usr/share/doc/sanoid/sanoid.conf.example /etc/sanoid/sanoid.conf

Configuration

We keep the already existing templates.

One key point is that pruning is not done by syncoid but only and always by sanoid. This means sanoid has to be run on the backup datasets as well, but without creating snapshots, only pruning (as set in the template).

Also, the template is called template_something and only something must be use with use_template.

[SAN200/projects]
        use_template = production
        recursive = yes
        process_children_only = yes

[BACKUP/SAN200/projects]
        use_template = backup
        recursive = yes
        process_children_only = yes

https://github.com/jimsalterjrs/sanoid/wiki/Sanoid

System files location

$ find /lib/systemd -name "*sanoid*"
/lib/systemd/system/sanoid-prune.service
/lib/systemd/system/sanoid.service
/lib/systemd/system/sanoid.timer

CAVEAT: post_snapshot_script cannot be used with syncoid

Especially with recursive = yes. This is because there cannot be two zfs send and receive at the same time on the same dataset (details ?).

And, sanoid does not wait for the script completion before continuing. This mean that should the syncoid process take a bit too much time, a new one will be spawned. And for reasons unknown to me yet, a new syncoid process will cancel the previous one (instead of just leaving).

As some of the spawned syncoid will produce errors, the entire sanoid process will fail.

So… no way… this does not work and has to be done independently, it seems.

The good news is that the SystemD service of Type= oneshot can have several Execstart= lines.

syncoid – a tool to facilitates the asynchronous incremental replication of ZFS filesystems.

https://github.com/jimsalterjrs/sanoid/wiki/Syncoid

–identifier=EXTRA	Extra identifier which is included in the snapshot name. Can be used for replicating to multiple targets.
–recursive\|r	Also transfers child datasets
–skip-parent	Skips syncing of the parent dataset. Does nothing without ‘–recursive’ option.
–sendoptions=OPTIONS	Use advanced options for zfs send (the arguments are filtered as needed), e.g. syncoid –sendoptions=”Lc e” sets zfs send -L -c -e …
–recvoptions=OPTIONS	Use advanced options for zfs receive (the arguments are filtered as needed), e.g. syncoid –recvoptions=”ux recordsize o compression=lz4″ sets zfs receive -u -x recordsize -o compression=lz4 …
–compress	Currently accepted options: gzip, pigz-fast, pigz-slow, zstd-fast, zstd-slow, lz4, xz, lzo (default) & none. If the selected compression method is unavailable on the source and destination, no compression will be used

First time it is used, the target dataset must not exist. After that, it must exist and contain snapshots also present on the source.

sudo /usr/sbin/syncoid --recursive --skip-parent SAN200/projects BACKUP/SAN200/projects

$ /usr/sbin/syncoid --identifier=externalUSB storage/documents USBdrive/documents

This will create the following snapshot on both sender and receiver:

{destination_dataset}@syncoid_{identifier}_{hostname}_{extended_date}

USBdrive/documents@syncoid_externalUSB_mycomputer_2021-12-12:12:18:09-GMT01:00

SystemD service and timer

$ more /etc/systemd/system/syncoid.service 
[Unit]
Description=Runs syncoid
Wants=syncoid.timer

[Service]
Type=oneshot
ExecStart=/usr/sbin/syncoid --recursive --skip-parent SAN200/projects BACKUP/SAN200/projects
ExecStart=/usr/sbin/syncoid --recursive --skip-parent SAN300/projects BACKUP/SAN300/projects

[Install]
WantedBy=multi-user.target

It seems the Unit= isn’t necessary if the service and the timer share the same name.

$ more /etc/systemd/system/syncoid.timer 
[Unit]
Description=Runs syncoid

[Timer]
OnCalendar=*-*-* *:10:00
#Unit=syncoid.service

[Install]
WantedBy=timers.target

sudo systemctl daemon-reload
sudo systemctl restart syncoid.timer
systemctl list-timers

man systemd.unit
man systemd.service
man systemd.time