Difference between revisions of "Time Machine Backups"

From OpenZFS on OS X
Jump to: navigation, search
(reformat text and designate commands using the html code tag)
(Add the option to backup via network shares set up with the com.apple.devdisk and com.apple.mimic_hfs properties.)
(4 intermediate revisions by 2 users not shown)
Line 1: Line 1:
== Time Machine Backups ==
+
== Backup to Local Disk ==
 +
There are two approaches to using ZFS for Time Machine Backups - Sparsebundles on a ZFS data set and zvols formatted as HFS+.
 +
 
 +
=== Sparsebundles ===
 
Here's one approach to using ZFS for your Time Machine Backups.   
 
Here's one approach to using ZFS for your Time Machine Backups.   
  
Line 14: Line 17:
 
The hypothesis being that an HFS+ sparsebundle stored on a compressed (gzip, lz4), deduped dataset should
 
The hypothesis being that an HFS+ sparsebundle stored on a compressed (gzip, lz4), deduped dataset should
 
yield a compression ratio > 1.0 (previously observed 1.4 with compression=on, dedup=off, FreeBSD network Time Machine drives).
 
yield a compression ratio > 1.0 (previously observed 1.4 with compression=on, dedup=off, FreeBSD network Time Machine drives).
 +
 +
=== zvols ===
 +
 +
1. Create a zvol to use for Time Machine backups:
 +
 +
<code>sudo zfs create -V [size] [pool]/[volume]</code>
 +
 +
For example, the following will create a 1TB volume named tmdisk1 on a zpool named Backup:
 +
 +
<code>sudo zfs create -V 1T Backup/tmdisk1</code>
 +
 +
2. Get OSX disk number corresponding to the zvol:
 +
 +
<code>ioreg -trn "ZVOL [pool]/[volume] Media" | awk -F'"' '/BSD Name/ {print $4;}'</code>
 +
 +
For example, if a zvol named tmdisk1 was created as shown above, the following will get its disk number and store it in the shell variable DISKID:
 +
 +
<code>ioreg -trn 'ZVOL Backup/tmdisk1 Media' | awk -F'"' '/BSD Name/ {print $4;}'</code>
 +
 +
3. Format the zvol as a HFS+, using its disk number from step 2:
 +
 +
<code>sudo diskutil eraseDisk JHFS+ [OSX Disk Name] [disk number]</code>
 +
 +
For example, if the zvol was disk5 in OSX, the following will format that disk as a HFS+ disk named "TimeMachine":
 +
 +
<code>sudo diskutil eraseDisk JHFS+ "TimeMachine" disk5</code>
 +
 +
4. Set the zvol as a Time Machine destination:
 +
 +
<code>sudo tmutil setdestination -a /Volumes/[OSX Disk Name]</code>
 +
 +
For example, if the zvol was formatted as HFS+ named "TimeMachine" in step 3, the following will set it as a Time Machine destination:
 +
 +
<code>sudo tmutil setdestination -a /Volumes/TimeMachine</code>
 +
 +
5. (Optional) Enable encryption for Time Machine backups on the zvol:
 +
 +
5.1 Get the disk identifier for the HFS+ disk on the zvol:
 +
 +
<code>diskutil list [OSX Disk Name] | awk '/[OSX Disk Name]/ { print $NF;}'</code>
 +
 +
For example, if the zvol was formatted as HFS+ named "TimeMachine" in step 3, the following will get its disk identifier:
 +
 +
<code>diskutil list TimeMachine | awk '/TimeMachine/ { print $NF; }'</code>
 +
 +
5.2 Convert the HFS+ disk to use encryption:
 +
 +
<code>sudo diskutil coreStorage convert [Disk ID] -passphrase</code>
 +
 +
This will ask for the encryption password (which may be stored in the Keychain).  For example, if the if the zvol was formatted as HFS+ named "TimeMachine" in step 3, whose disk identifier was returned as disk5s2 in step 5.1, the following will convert the disk to use encryption:
 +
 +
<code>sudo diskutil coreStorage convert disk5s2 -passphrase</code>
 +
 +
Please note, this process can take a considerable amount of time (several hours for a 1TB volume).  You can check on the progress of the encryption using the following command:
 +
 +
<code>diskutil cs list | grep -e "Conversion" -e "Volume Name"</code>
 +
 +
== Backup to Network Share ==
 +
 +
Time Machine can backup over the network to either HFS+ or APFS target volumes, shared via SMB.  If you try to do this with a ZFS target volume, the client gets an error saying "The selected network backup disk does not support the required capabilities." 
 +
 +
You have two options from this point on:
 +
# You can create an HFS+ or APFS target volume on your zpool.  There are two ways to do that, explained in the section above about local backups: create the volume as a sparse bundle, or as a zvol. 
 +
# You can set the properties <code>com.apple.devdisk</code> and <code>com.apple.mimic_hfs</code> to 'on' on the dataset you wish to put your backups onto, through the <code>zfs set</code> command. (note that if you do this through an already mounted dataset, you may have to unmount it and re-mount it again after issuing the <code>zfs set</code> commands)
 +
 +
You don't need the steps that set the volume as a time machine destination on the server; it's just going to be shared over the network.  When you share the volume on the network, follow the instructions at https://support.apple.com/guide/mac-help/a-shared-folder-time-machine-mac-mchl31533145/mac to make it available as a TimeMachine volume.

Revision as of 17:52, 15 January 2020

Backup to Local Disk

There are two approaches to using ZFS for Time Machine Backups - Sparsebundles on a ZFS data set and zvols formatted as HFS+.

Sparsebundles

Here's one approach to using ZFS for your Time Machine Backups.

Because Time Machine doesn't recognize ZFS datasets as a compatible disk for Time Machine backups, as a work around, we create an HFS+ sparsebundle disk image, store it on a ZFS dataset, and set the mounted image as a backup destination (no "TMShowUnsupportedNetworkVolumes" needed).

1. Create, and mount, a sparsebundle from a ZFS dataset (e.g., with makeImage.sh or Disk Utility.app).

2. Set the sparsebundle as the (active) backup destination:

# tmutil setdestination -a /Volumes/[sparse bundle volume name]

While it has been discussed in heated arguments (e.g., https://github.com/openzfsonosx/zfs/issues/66) I still believe there's at least one ZFS feature I'd like to test with Time Machine: compression.

The hypothesis being that an HFS+ sparsebundle stored on a compressed (gzip, lz4), deduped dataset should yield a compression ratio > 1.0 (previously observed 1.4 with compression=on, dedup=off, FreeBSD network Time Machine drives).

zvols

1. Create a zvol to use for Time Machine backups:

sudo zfs create -V [size] [pool]/[volume]

For example, the following will create a 1TB volume named tmdisk1 on a zpool named Backup:

sudo zfs create -V 1T Backup/tmdisk1

2. Get OSX disk number corresponding to the zvol:

ioreg -trn "ZVOL [pool]/[volume] Media" | awk -F'"' '/BSD Name/ {print $4;}'

For example, if a zvol named tmdisk1 was created as shown above, the following will get its disk number and store it in the shell variable DISKID:

ioreg -trn 'ZVOL Backup/tmdisk1 Media' | awk -F'"' '/BSD Name/ {print $4;}'

3. Format the zvol as a HFS+, using its disk number from step 2:

sudo diskutil eraseDisk JHFS+ [OSX Disk Name] [disk number]

For example, if the zvol was disk5 in OSX, the following will format that disk as a HFS+ disk named "TimeMachine":

sudo diskutil eraseDisk JHFS+ "TimeMachine" disk5

4. Set the zvol as a Time Machine destination:

sudo tmutil setdestination -a /Volumes/[OSX Disk Name]

For example, if the zvol was formatted as HFS+ named "TimeMachine" in step 3, the following will set it as a Time Machine destination:

sudo tmutil setdestination -a /Volumes/TimeMachine

5. (Optional) Enable encryption for Time Machine backups on the zvol:

5.1 Get the disk identifier for the HFS+ disk on the zvol:

diskutil list [OSX Disk Name] | awk '/[OSX Disk Name]/ { print $NF;}'

For example, if the zvol was formatted as HFS+ named "TimeMachine" in step 3, the following will get its disk identifier:

diskutil list TimeMachine | awk '/TimeMachine/ { print $NF; }'

5.2 Convert the HFS+ disk to use encryption:

sudo diskutil coreStorage convert [Disk ID] -passphrase

This will ask for the encryption password (which may be stored in the Keychain). For example, if the if the zvol was formatted as HFS+ named "TimeMachine" in step 3, whose disk identifier was returned as disk5s2 in step 5.1, the following will convert the disk to use encryption:

sudo diskutil coreStorage convert disk5s2 -passphrase

Please note, this process can take a considerable amount of time (several hours for a 1TB volume). You can check on the progress of the encryption using the following command:

diskutil cs list | grep -e "Conversion" -e "Volume Name"

Backup to Network Share

Time Machine can backup over the network to either HFS+ or APFS target volumes, shared via SMB. If you try to do this with a ZFS target volume, the client gets an error saying "The selected network backup disk does not support the required capabilities."

You have two options from this point on:

  1. You can create an HFS+ or APFS target volume on your zpool. There are two ways to do that, explained in the section above about local backups: create the volume as a sparse bundle, or as a zvol.
  2. You can set the properties com.apple.devdisk and com.apple.mimic_hfs to 'on' on the dataset you wish to put your backups onto, through the zfs set command. (note that if you do this through an already mounted dataset, you may have to unmount it and re-mount it again after issuing the zfs set commands)

You don't need the steps that set the volume as a time machine destination on the server; it's just going to be shared over the network. When you share the volume on the network, follow the instructions at https://support.apple.com/guide/mac-help/a-shared-folder-time-machine-mac-mchl31533145/mac to make it available as a TimeMachine volume.