OpenZFS on OS X - User contributions [en]

FAQ

2021-09-07T01:35:52Z

Srirangav: add link time machine backups in the wiki

[[Category:About O3X]]

Besides the questions covered below, you may find [[Documentation]] and [[OpenZFS on OS X]] helpful. Both articles contain a good deal of information about OpenZFS on OS X.

== General ==

===Q) What is OpenZFS on OS X?===
'''A)''' See the article entitled [[OpenZFS on OS X]].

===Q) What does O3X mean?===
'''A)''' O3X = O O O X = '''O'''penZFS '''o'''n '''O'''S '''X'''.

===Q) What version of ZFS do you use?===
'''A)''' OpenZFS. Pool version 5000. File system version 5. Pool version 5000 is pool version 28 plus support for feature flags. We support pool version 5000 and pool versions less than or equal to 28. We do not support the closed-source Oracle Solaris ZFS pool versions 29 and up.

===Q) Can I set up a test pool using files instead of disks?===
'''A)''' Yes. The example uses simple files, its also possible to use disk images.

$ cd /tmp
$ mkfile 10G aaa
$ mkfile 10G bbb
$ mkfile 10G ccc
$ sudo zpool create tank raidz /tmp/aaa /tmp/bbb /tmp/ccc
$

===Q) I compiled from source, how do I know that I am running what I compiled?===
'''A)''' There are a couple of sysctls that can be read to determine what commit was compiled into your custom kexts. Assuming you used zfsadm to build, or that you executed the autoconf, and configure steps before compiling the following works.

$ sysctl -a | grep kext
spl.kext_version: 1.5.2-2_g'''115aa2f'''
zfs.kext_version: 1.5.2-33_g'''9ac66a7'''

$ cd <path to source code>/zfs
$ git log -n 1
commit '''9ac66a7'''1e53636eec04f4718b0b3870a18f07840
Merge: 3326995 890ef86
Author: zadmin <zadmin@jerry.local>
Date: Thu Jun 16 17:19:24 2016 +1000

Merge branch 'master' of https://github.com/openzfsonosx/zfs

$ cd <path to source code>/spl
$ git log -n 2
commit f1ff660a2f1fa340d451c2afa5f726f9bd3e609d
Author: Brendon Humphrey <brendon.humphrey@mac.com>
Date: Sat Jun 18 20:25:09 2016 -0700
...
commit '''115aa2f'''05b6f843e0d39d4f6bf999602db120113
Author: Jorgen Lundman <lundman@lundman.net>
Date: Thu May 12 09:48:31 2016 +0900
...

I have highlighted the relevant text to match. You can see that this machine is running the latest ZFS, and is one commit behind the latest SPL.

If you make a small local change to the code then this technique will not work. One workaround for that is to edit your file(s) and commit them in your repository clone, that way it will have a commit id.

== Best practices ==

===Q) Do I have to use mirrors or raidz? ===
'''A)''' Have to? No. Should you? Virtually always. ZFS will not be able to repair errors it finds unless you have redundancy at the vdev level.

===Q) Can I set copies=2 in lieu of using mirrors or raidz? ===
Setting copies=2 is a poor substitute for vdev-level redundancy. Two copies on a broken drive are worthless. That being said, yes, you can set copies=2. Do so at your own risk.

== Administration ==

===Q) How can I access the .zfs snapshot directories?===
'''A)''' You need to set snapdir visible and manually mount a snapshot.
<syntaxhighlight lang="bash">
$ sudo zfs set snapdir=visible tank/bob
$ sudo zfs mount tank/bob@yesterday
$ ls -l /tank/bob/.zfs/snapshot/yesterday/
</syntaxhighlight>
You can see existing snapshots via:
<syntaxhighlight lang="bash">
$ zfs list -t snapshot
</syntaxhighlight>

===Q) Is .zfs snapdir auto-mounting supported?===
'''A)''' No, not at this time. You must manually "zfs mount" snapshots manually to see them in the snapdir.

===Q) OK, I manually mounted my snapshot but still cannot see it in Finder. What gives?===
'''A)''' Currently mounted snapshots are only visible from Terminal, not from Finder.
<syntaxhighlight lang="bash">
$ ls -l /tank/bob/.zfs/snapshot/yesterday/
</syntaxhighlight>

===Q) Why does OSX server not allow the server storage to be on ZFS?===
'''A)''' OSX Server has been coded in such a way as to only allow the server storage area to be an HFS formatted drive. O3X offers a feature that causes ZFS datasets to identify themselves as HFS. This is sufficient for OSX server to allow storage on a ZFS filesystem. HFS mimic is enabled by setting the com.apple.mimic_hfs property on a per dataset basis.

In addition as of OSX Server 5.x it seems that the Application Store Caching server can only store its cache on HFS. This is a new behaviour. I just used an HFS formatted ZVOL to get around this.

sudo zfs create -o volblocksize=1m -s -V 250g tank/cachingzvol

== Interoperability ==

===Q) How do I create an O3X compatible pool on another OpenZFS platform? ===
'''A)''' Only enable feature flags supported by O3X, as discussed [https://openzfsonosx.org/wiki/Zpool#Feature_flags here].

===Q) Can I import my ZEVO pools?===
'''A)''' Yes. O3X can import pool version 28, which means it can import ZEVO pools.

===Q) Can I import my MacZFS pools?===
'''A)''' Yes. O3X can import pool version 8, which means it can import MacZFS pools.

===Q) Do HFS-only applications such as Photos, iCloud Drive and macOS Server work on ZFS?===
'''A)''' Sometimes. Apple codes some software to only work when stored on HFS. We can't change that. We have provided a property that causes ZFS filesystems to identify themselves as a HFS when enabled. We are unable to guarantee that the application will work 100% correctly as HFS may have specific behaviours that the application depends on, and ZFS may not behave identically.

sudo zfs set com.apple.mimic_hfs=on <dataset>

===Q) Does Spotlight work? ===
'''A)''' Yes. Spotlight works on O3X 1.3.1+.

===Q) Can Time Machine backups be stored on ZFS? ===
'''A)''' Yes. It is possible to host a TimeMachine backup within a SparseImage on ZFS, or an HFS formatted ZVOL On ZFS. See: https://openzfsonosx.org/wiki/Time_Machine_Backups

===Q) Updating crypto to 2.0 ===
'''A)''' If you go from 1.9.4 (or earlier) and use crypto, you will get "mount returns -1". This is due to missing
features userobjquota, groupobjquota and projectquotas. The pool will need "zpool upgrade" before you can mount it.
Please remember this means you can not go back to 1.9.4, as with all pool upgrades.

===Q) Updating crypto to 2.0 from earlier 2.0-rc ===
'''A)''' There was a few early releases of 2.0-rc, where the MAC was calculated incorrectly, and you will see:
rpool/crypt:<0x0>
However, issuing "zpool scrub" will clear this issue (as it re-computes the MAC).

===Q) Uninstalling on Big Sur and later ===
'''A)''' Unfortunately, macOS from BigSur has a tendency to "hold on" to kexts, and here is the best way to make sure the kext is not loaded:
Boot into recovery mode
Launch Terminal
# kmutil trigger-panic-medic --volume-root /Volumes/<YourVolumeName>
As in "/Volumes/MacintoshHD" or similar.
reboot normally.
trigger-panic-medic will remove all 3rd party kexts, so you need to "Approve" any kexts
you want to keep, just make sure to skip zfs.kext.
Clean up all files:
# sudo rm -rf /Library/Extensions/zfs.kext /usr/local/zfs/

== Limitations ==

===Q) Can I use finder permissions aka ACLs? ===
'''A)''' Not yet. There is work to go in this area. See https://github.com/openzfsonosx/zfs/issues/275

===Q) Can I boot my computer off of O3X?===
'''A)''' Yes. See https://openzfsonosx.org/wiki/ZFS_on_Boot

===Q) So if I use O3X, that means I don't need to back up, right? ===
'''A)''' Wrong. Wrong. Wrong.

===Q) Can TimeMachine backup the contents of a ZFS volume? ===
'''A)''' No. We believe that when "Issue 116" is resolved it may be supportable. At the present time TimeMachine excludes ZFS filesystems from the list of available backup targets.

===Q) How does auto-import work ? ===
'''A)''' Auto-import on boot is handled by Launchd, from the file '''/Library/LaunchDaemons/org.openzfsonosx.zpool-import-all.plist'''. Which executes the script
'''/usr/local/libexec/zfs/launchd.d/zpool-import-all.sh''' to import files. You can unload the plist, or if you want to temporarily stop auto-import, simply comment
out the line '''"${ZPOOL}" import -a -d /var/run/disk/by-id''' by placing a '''#''' in front of it.

===Q) I can't see my pools? ===
'''A)''' First make sure that '''diskutil list''' will show the disk(s) used in your pool. If not, you have a hardware and/or OsX problem to solve. Second step is running '''zpool import''' and possibly '''zpool import -d /dev''' to make ZFS inspect all disks on your system, looking for pools. The command will list any pools that it can find, and the state they are in, except for pools already imported (You can run '''zpool list''' to display the imported pools). After that, you would run '''zpool import poolname''' to import the pool, where ''poolname'' is replaced with the name of your pool. If you need further assistance, ask on the forum, irc, or, in the case of panics, create a ticket under the github repository.

===Q) Auto-import fails after Catalina? ===
'''A)''' System Preferences -> Security & Privacy -> Full Disk Access by allowing bash. (bash used by auto-import script, change to 'sh' for lingon.)

Time Machine Backups

2021-08-10T23:43:09Z

Srirangav: /* zvols */

== Backup to Local Disk ==
There are two approaches to using ZFS for Time Machine Backups - Sparsebundles on a ZFS data set and zvols formatted as HFS+.

=== Sparsebundles ===
Here's one approach to using ZFS for your Time Machine Backups.

Because Time Machine doesn't recognize ZFS datasets as a compatible disk for Time Machine backups, as a work around, we create an HFS+ sparsebundle disk image, store it on a ZFS dataset, and set the mounted image as a backup destination (no "TMShowUnsupportedNetworkVolumes" needed).

1. Create, and mount, a sparsebundle from a ZFS dataset (e.g., with makeImage.sh or Disk Utility.app).

2. Set the sparsebundle as the (active) backup destination:

<code># tmutil setdestination -a /Volumes/[sparse bundle volume name]</code>

While it has been discussed in heated arguments (e.g., https://github.com/openzfsonosx/zfs/issues/66) I still believe there's at least one ZFS feature I'd like to test with Time Machine: compression.

The hypothesis being that an HFS+ sparsebundle stored on a compressed (gzip, lz4), deduped dataset should
yield a compression ratio > 1.0 (previously observed 1.4 with compression=on, dedup=off, FreeBSD network Time Machine drives).

=== zvols ===

<ol>

<li>Create a zvol to use for Time Machine backups:

<blockquote>

<code>sudo zfs create -V [size] [pool]/[volume]</code>

</blockquote>

For example, the following will create a 1TB volume named tmdisk1 on a zpool named Backup:

<blockquote>

<code>sudo zfs create -V 1T Backup/tmdisk1</code>

</blockquote>

</li>

<li>Get OSX disk number corresponding to the zvol:

<blockquote>

<code>ioreg -trn "ZVOL [pool]/[volume] Media" | awk -F'"' '/BSD Name/ {print $4;}'</code>

</blockquote>

For example, if a zvol named tmdisk1 was created as shown above, the following will get its disk number and store it in the shell variable DISKID:

<blockquote>

<code>ioreg -trn 'ZVOL Backup/tmdisk1 Media' | awk -F'"' '/BSD Name/ {print $4;}'</code>

</blockquote>

</li>

<li>Format the zvol as a HFS+, using its disk number from the previous step:

<blockquote>

<code>sudo diskutil eraseDisk JHFS+ [OSX Disk Name] [disk number]</code>

</blockquote>

For example, if the zvol was disk5 in OSX, the following will format that disk as a HFS+ disk named "TimeMachine":

<blockquote>

<code>sudo diskutil eraseDisk JHFS+ "TimeMachine" disk5</code>

</blockquote>

For newer systems (Big Sur), because Time Machine now prefers apfs (see
https://support.apple.com/en-au/guide/mac-help/mh15139/mac), the following
commands should create a apfs zvol that Time Machine can use:

<blockquote>

<pre>sudo diskutil eraseDisk JHFS+ [OSX Disk Name] GPT [disk number]
sudo diskutil apfs create [disk number]s[slice number] [OSX Disk Name]</pre>

</blockquote>

For example, if the zvol was disk5 in OSX, and the HFS+ volume on that zvol
was on slice 1 (which can be confirmed using '<code>diskutil list disk5</code>')
the following should format that disk as a apfs disk named "TimeMachine":

<blockquote>

<pre>sudo diskutil eraseDisk JHFS+ "TimeMachine" GPT disk5
sudo diskutil apfs create disk5s1 "TimeMachine"</pre>

</blockquote>

</li>

<li>Set the zvol as a Time Machine destination:

<blockquote>

<code>sudo tmutil setdestination -a /Volumes/[OSX Disk Name]</code>

</blockquote>

For example, if the zvol was formatted as HFS+ named "TimeMachine" in
the previous steps, the following will set it as a Time Machine destination:

<blockquote>

<code>sudo tmutil setdestination -a /Volumes/TimeMachine</code>

</blockquote>

</li>

<li>(Optional) Enable encryption for Time Machine backups on the zvol:

<ul>

<li>For HFS+ formatted zvols, the following procedure should work:

<ol>

<li>Get the disk identifier for the HFS+ disk on the zvol:

<blockquote>

<code>diskutil list [OSX Disk Name] | awk '/[OSX Disk Name]/ { print $NF;}'</code>

</blockquote>

For example, if the zvol was formatted as HFS+ named "TimeMachine" in step 3, the following will get its disk identifier:

<blockquote>

<code>diskutil list TimeMachine | awk '/TimeMachine/ { print $NF; }'</code>

</blockquote>

</li>

<li>Convert the HFS+ disk to use encryption:

<blockquote>

<code>sudo diskutil coreStorage convert [Disk ID] -passphrase</code>

</blockquote>

This will ask for the encryption password (which may be stored in the Keychain). For example, if the if the zvol was formatted as HFS+ named "TimeMachine" in step 3, whose disk identifier was returned as disk5s2 in step 5.1, the following will convert the disk to use encryption:

<blockquote>

<code>sudo diskutil coreStorage convert disk5s2 -passphrase</code>

</blockquote>

Please note, this process can take a considerable amount of time (several hours for a 1TB volume). You can check on the progress of the encryption using the following command:

<blockquote>

<code>diskutil cs list | grep -e "Conversion" -e "Volume Name"</code>

</blockquote>

</li>
</ol>

</li>

<li>For apfs formatted zvols, the following procedure should work (based on
comments in the forums, see https://openzfsonosx.org/forum/viewtopic.php?f=26&t=3612):

<blockquote>

<code>sudo diskutil apfs encryptvolume [Disk ID] -user disk</code>

</blockquote>

This will ask for the encryption password (which may be stored in the Keychain). For example, if the zvol was formatted as apfs named "TimeMachine" in a prior
step, whose disk identifier was returned as disk5s1, the following will convert the disk to use encryption:

<blockquote>

<code>sudo diskutil apfs encryptvolume disk5s1 -user disk</code>

</blockquote>

Based on the forums (see https://openzfsonosx.org/forum/viewtopic.php?f=26&t=3612), it is possible to "bless" the Time Machine
zvol by "open /System/Library/CoreServices/CoreTypes.bundle/Contents/Resources/GenericTimeMachineDiskIcon.icns", select the first icon in the resulting window, copying (⌘-C) the icon, then Get Info (⌘-I) on the Time Machine disk icon, and pasting (⌘-V) the copied image to the Time Machine disk to "bless" it as a Time Machine disk. For some reason, Time Machine won't do that by itself, as it does with normal external volumes when it first starts using them for Time Machine.
</li>
</ul>
</li>
</ol>

== Backup to Network Share ==

Time Machine can backup over the network to either HFS+ or APFS target volumes, shared via SMB. If you try to do this with a ZFS target volume, the client gets an error saying "The selected network backup disk does not support the required capabilities."

You have two options from this point on:
# You can create an HFS+ or APFS target volume on your zpool. There are two ways to do that, explained in the section above about local backups: create the volume as a sparse bundle, or as a zvol.
# You can set the properties <code>com.apple.devdisk</code> and <code>com.apple.mimic_hfs</code> to 'on' on the dataset you wish to put your backups onto, through the <code>zfs set</code> command. (note that if you do this through an already mounted dataset, you may have to unmount it and re-mount it again after issuing the <code>zfs set</code> commands)

You don't need the steps that set the volume as a time machine destination on the server; it's just going to be shared over the network. When you share the volume on the network, follow the instructions at https://support.apple.com/guide/mac-help/a-shared-folder-time-machine-mac-mchl31533145/mac to make it available as a TimeMachine volume.

Encryption

2019-11-06T20:41:59Z

Srirangav:

Encryption is now native to ZFS, and it is recommended to use that for greater flexibility and compatibility. See below. However, the core storage documentation will remain here for those who prefer that method.

== Native ZFS Encryption ==

On a zpool that supports encryption, encryption may be enabled as follows:

<code># zpool set feature@encryption=enabled [pool]</code>

On an zpool that supports encryption, an encrypted zfs dataset may be created as follows:

<code># zfs create -o encryption=on -o keylocation=prompt -o keyformat=passphrase [dataset]</code>

This will prompt for the encryption passphrase for this zfs dataset. Other options for the location of the encryption key and its format can be found in the zfs(1M) manpage.

An encrypted zfs dataset may be mounted as follows:

<code># zfs mount -l [dataset]</code>

This will prompt for the encryption passphrase for this zfs dataset and mount the encrypted dataset; it will not mount any child datasets of the encrypted dataset, but they will be accessible as subdirectories of the encrypted dataset. If the encryption passphrase is stored in the Keychain as a generic password under the name of the dataset, security(1) may be used to retrieve the passphrase as follows:

<code># security find-generic-password -a [dataset] -w | zfs mount -l [dataset]</code>

One common use case for an encrypted volume is a portable backup drive. Typically the user will create snapshots on the source drive in the computer, and then transmit them to the portable drive using zfs send and receive. In this scenario it is helpful if the destination dataset on the portable drive is read only, since then there is no need to roll back the destination dataset on the fly to the last valid snapshot. To make this work with encryption, place the destination dataset as a child dataset of the encrypted dataset in the zpool of the portable drive; because it is a child of the encrypted dataset it, too, will be encrypted. More importantly, it can be made read-only, whereas it appears that the encrypted parent dataset cannot be read-only (probably so that the key can be stored locally in the parent dataset).

Additional helpful information about zfs encryption can be found in the [https://blog.heckel.xyz/2017/01/08/zfs-encryption-openzfs-zfs-on-linux/ How-To: Using ZFS Encryption at Rest in OpenZFS (ZFS on Linux, ZFS on FreeBSD, …)].

== Core Storage (File Vault 2) ==

Although the upstream OpenZFS project lists [http://open-zfs.org/wiki/Projects#Platform_agnostic_encryption_support platform-agnostic encryption support] at the ZFS dataset level as a possible future enhancement, OS X already offers a feature called [http://support.apple.com/kb/ht4790 FileVault 2], which provides built-in support for XTS-AES 128 encryption at the block level as part of Core Storage volume management.

This is the OS X analogue of the following block-level encryption systems on other operating systems that support ZFS:
* FreeBSD: geli
* Linux: LUKS

The overall procedure is, as follows: convert an empty HFS+ partition to use Core Storage and apply Core Storage encryption. Then use the Core Storage Logical Volume as a device in your zpool by supplying it to "zpool create," "zpool add," "zpool attach," etc.

=== Prerequisites ===
Build ZFS [[Install#Installing_from_source|from source]], or wait for the [[Downloads|next installer]], newer than 1.2.0 (for explanation, see original IRC chat).

=== Caveats ===
You may receive a pop-up claiming the disk isn't readable by this computer. This leads to one step that can be confusing: when unlocking the disk (e.g., on startup), the "bug" will make OS X believe the disk wasn't unlocked, and thus "wiggle," presenting the prompt again.

Assuming you entered your password correctly, the encrypted volume should now be unlocked, despite the misleading wiggle, and you can safely close the dialog box by clicking "Cancel." You'll know for sure the volume is unlocked when you proceed to import your pool, or you can check directly by looking for <code>Encryption Status: Unlocked</code> in the output of <code>diskutil coreStorage list</code>.

=== Steps ===
The initial layout, with disk1 being the external disk (counter-intuitively named "Internal HD") intended as encrypted ZFS device.

# diskutil list
/dev/disk0
#: TYPE NAME SIZE IDENTIFIER
0: GUID_partition_scheme *160.0 GB disk0
1: EFI EFI 209.7 MB disk0s1
2: Apple_HFS Macintosh HD 159.7 GB disk0s2
/dev/disk1
#: TYPE NAME SIZE IDENTIFIER
0: GUID_partition_scheme *1.0 TB disk1
1: EFI EFI 209.7 MB disk1s1
2: Apple_HFS Internal HD 999.9 GB disk1s2

We note that disk1s2 is the partition to be encrypted, and we convert it to Core Storage (think LVM), to enable encryption:

# diskutil coreStorage convert /dev/disk1s2
Started CoreStorage operation on disk1s2 Internal HD
Resizing disk to fit Core Storage headers
Creating Core Storage Logical Volume Group
Attempting to unmount disk1s2
Switching disk1s2 to Core Storage
Waiting for Logical Volume to appear
Mounting Logical Volume
Core Storage LVG UUID: 4690972A-484E-42E2-B72D-933A58E41237
Core Storage PV UUID: 22A1A783-01BA-4ABA-B4A3-2A9146506519
Core Storage LV UUID: F6D16BFE-B6E9-4A9B-BC03-E5CD03772C44
Core Storage disk: disk2
Finished CoreStorage operation on disk1s2 Internal HD

Note that we converted the existing unencrypted HFS+ partition.

Next, we encrypt the logical volume, our Core Storage disk, disk2:

:'''Note: If you choose to use diskutil to do the encryption, it will default to the most secure option, which will take a VERY long time with large disks, optionally use the Disk Util UI and set it up as an encrypted HFS+ volume and under advanced change to least secure (good for new disks)'''

# diskutil coreStorage encryptVolume /dev/disk2
New passphrase for existing volume:
Confirm new passphrase:
The Core Storage Logical Volume UUID is F6D16BFE-B6E9-4A9B-BC03-E5CD03772C44
Started CoreStorage operation on disk2 Internal HD
Scheduling encryption of Core Storage Logical Volume
Core Storage LV UUID: F6D16BFE-B6E9-4A9B-BC03-E5CD03772C44
Finished CoreStorage operation on disk2 Internal HD

Note that we used disk2, the logical volume, not disk1s2.

This can and will take a while to complete. You can check the status by issuing:
# diskutil coreStorage list | grep Conversion

Until it's done:
Conversion Status: Complete
Conversion Progress: -none-

Your partition layout should now look like:

# diskutil list
/dev/disk0
#: TYPE NAME SIZE IDENTIFIER
0: GUID_partition_scheme *160.0 GB disk0
1: EFI EFI 209.7 MB disk0s1
2: Apple_HFS Macintosh HD 159.7 GB disk0s2
/dev/disk1
#: TYPE NAME SIZE IDENTIFIER
0: GUID_partition_scheme *1.0 TB disk1
1: EFI EFI 209.7 MB disk1s1
2: Apple_CoreStorage 999.9 GB disk1s2
3: Apple_Boot Boot OS X 134.2 MB disk1s3
/dev/disk2
#: TYPE NAME SIZE IDENTIFIER
0: Apple_HFS *999.5 GB disk2

disk2 being our encrypted, unlocked HFS+ device. If you have yet to be prompted for the passphrase by OS X, now would be a good time to restart your Mac and try it out.

Lastly, we'll prepare the volume for ZFS, by unmounting /dev/disk2:

# mount
...
/dev/disk2 on /Volumes/Internal HD (hfs, local, journaled)
# diskutil unmount "/Volumes/Internal HD"

You can now follow the article on [[Zpool#Creating_a_pool|creating a pool]]. As a simple example, you might

<syntaxhighlight lang="text">
# zpool list
no pools available
# zpool create -f -o ashift=12 ZFS_VOLUME /dev/disk2
# zpool list
ZFS_VOLUME 928G 20.8G 907G 2% 1.00x ONLINE -
</syntaxhighlight>

Also note that you '''don't''' need to worry about changing the partition type in this case to ZFS as described in the [[Suppressing_the_annoying_pop-up|Suppressing the annoying pop-up]] wiki page.

=== Reason to "use latest" ===
<syntaxhighlight lang="text">
<ilovezfs> If you want encryption you have a few options
<ilovezfs> https://github.com/zfsrogue/osx-zfs-crypto
<lundman> :)
<ilovezfs> or you can do what cbreak said, and use an encrypted sparsebundle
<ilovezfs> (I'd give it its own ZFS file system)
<ilovezfs> or you can create a ZVOL, and put an encrypted Core Storage/Filevault 2 HFS+
file system on it
<ilovezfs> or you can put the pool itself on top of Core Storage.
<ilovezfs> The last option you should not do with the installer version.
<ilovezfs> But wait for the next installer if that's the route you want to go
<ilovezfs> or build from source.
<aandy> Ah, interesting. Does FileVault 2 require HFS+? Not that it'd surprise me.
<ilovezfs> No it does not.
<ilovezfs> But it is not possible to set other Content Hints
<ilovezfs> so it will always say HFS+ even if you do put ZFS on your logical volumes.
<ilovezfs> So basically the procedure is to format the volume HFS+.
<ilovezfs> Then run 'diskutil coreStorage convert' on it.
<ilovezfs> Then you can encrypt it.
<ilovezfs> Then you unmount the HFS+
<ilovezfs> and zpool create on the logical volume.
<ilovezfs> And you should be good to go.
<aandy> On the original HFS+ partition, right?
<ilovezfs> Right.
<ilovezfs> But I'd encrypt first
<ilovezfs> then put ZFS on it.
<aandy> Right. Perfect.
<ilovezfs> diskutil coreStorage convert ...
<ilovezfs> diskutil coreStorage encryptVolume ...
<ilovezfs> etc.
<ilovezfs> The reason not to use the installer version, is that it will attempt to
partition the Core Storage Logical Volume.
<ilovezfs> But since 10.8.5 and after, Apple doesn't like that
<ilovezfs> so we added new code to detect Core Storage and not partition if it sees it's
Core Storage.
</syntaxhighlight>

Time Machine Backups

2018-10-19T05:35:05Z

Srirangav: /* zvols */

== Time Machine Backups ==
There are two approaches to using ZFS for Time Machine Backups - Sparsebundles on a ZFS data set and zvols formatted as HFS+.

=== Sparsebundles ===
Here's one approach to using ZFS for your Time Machine Backups.

Because Time Machine doesn't recognize ZFS datasets as a compatible disk for Time Machine backups, as a work around, we create an HFS+ sparsebundle disk image, store it on a ZFS dataset, and set the mounted image as a backup destination (no "TMShowUnsupportedNetworkVolumes" needed).

1. Create, and mount, a sparsebundle from a ZFS dataset (e.g., with makeImage.sh or Disk Utility.app).

2. Set the sparsebundle as the (active) backup destination:

<code># tmutil setdestination -a /Volumes/[sparse bundle volume name]</code>

While it has been discussed in heated arguments (e.g., https://github.com/openzfsonosx/zfs/issues/66) I still believe there's at least one ZFS feature I'd like to test with Time Machine: compression.

The hypothesis being that an HFS+ sparsebundle stored on a compressed (gzip, lz4), deduped dataset should
yield a compression ratio > 1.0 (previously observed 1.4 with compression=on, dedup=off, FreeBSD network Time Machine drives).

=== zvols ===

1. Create a zvol to use for Time Machine backups:

<code>sudo zfs create -V [size] [pool]/[volume]</code>

For example, the following will create a 1TB volume named tmdisk1 on a zpool named Backup:

<code>sudo zfs create -V 1T Backup/tmdisk1</code>

2. Get OSX disk number corresponding to the zvol:

<code>ioreg -trn "ZVOL [pool]/[volume] Media" | awk -F'"' '/BSD Name/ {print $4;}'</code>

For example, if a zvol named tmdisk1 was created as shown above, the following will get its disk number and store it in the shell variable DISKID:

<code>ioreg -trn 'ZVOL Backup/tmdisk1 Media' | awk -F'"' '/BSD Name/ {print $4;}'</code>

3. Format the zvol as a HFS+, using its disk number from step 2:

<code>sudo diskutil eraseDisk JHFS+ [OSX Disk Name] [disk number]</code>

For example, if the zvol was disk5 in OSX, the following will format that disk as a HFS+ disk named "TimeMachine":

<code>sudo diskutil eraseDisk JHFS+ "TimeMachine" disk5</code>

4. Set the zvol as a Time Machine destination:

<code>sudo tmutil setdestination -a /Volumes/[OSX Disk Name]</code>

For example, if the zvol was formatted as HFS+ named "TimeMachine" in step 3, the following will set it as a Time Machine destination:

<code>sudo tmutil setdestination -a /Volumes/TimeMachine</code>

5. (Optional) Enable encryption for Time Machine backups on the zvol:

5.1 Get the disk identifier for the HFS+ disk on the zvol:

<code>diskutil list [OSX Disk Name] | awk '/[OSX Disk Name]/ { print $NF;}'</code>

For example, if the zvol was formatted as HFS+ named "TimeMachine" in step 3, the following will get its disk identifier:

<code>diskutil list TimeMachine | awk '/TimeMachine/ { print $NF; }'</code>

5.2 Convert the HFS+ disk to use encryption:

<code>sudo diskutil coreStorage convert [Disk ID] -passphrase</code>

This will ask for the encryption password (which may be stored in the Keychain). For example, if the if the zvol was formatted as HFS+ named "TimeMachine" in step 3, whose disk identifier was returned as disk5s2 in step 5.1, the following will convert the disk to use encryption:

<code>sudo diskutil coreStorage convert disk5s2 -passphrase</code>

Please note, this process can take a considerable amount of time (several hours for a 1TB volume). You can check on the progress of the encryption using the following command:

<code>diskutil cs list | grep -e "Conversion" -e "Volume Name"</code>

Time Machine Backups

2018-09-19T06:38:21Z

Srirangav:

Time Machine Backups

2018-09-18T01:15:29Z

Srirangav: /* Time Machine Backups */

== Time Machine Backups ==
There are two approaches to using ZFS for Time Machine Backups - Sparsebundles on a ZFS data set and zvols formatted as HFS+.

=== Sparsebundles ===
Here's one approach to using ZFS for your Time Machine Backups.

Because Time Machine doesn't recognize ZFS datasets as a compatible disk for Time Machine backups, as a work around, we create an HFS+ sparsebundle disk image, store it on a ZFS dataset, and set the mounted image as a backup destination (no "TMShowUnsupportedNetworkVolumes" needed).

1. Create, and mount, a sparsebundle from a ZFS dataset (e.g., with makeImage.sh or Disk Utility.app).

2. Set the sparsebundle as the (active) backup destination:

<code># tmutil setdestination -a /Volumes/[sparse bundle volume name]</code>

While it has been discussed in heated arguments (e.g., https://github.com/openzfsonosx/zfs/issues/66) I still believe there's at least one ZFS feature I'd like to test with Time Machine: compression.

The hypothesis being that an HFS+ sparsebundle stored on a compressed (gzip, lz4), deduped dataset should
yield a compression ratio > 1.0 (previously observed 1.4 with compression=on, dedup=off, FreeBSD network Time Machine drives).

=== zvols ===

1. Create a zvol to use for Time Machine backups:

<code>sudo zfs create -V [size] [pool]/[volume]</code>

For example, the following will create a 1TB volume named tmdisk1 on a zpool named Backup:

<code>sudo zfs create -V 1T Backup/tmdisk1</code>

2. Get OSX disk number corresponding to the zvol:

<code>ioreg -trn "ZVOL [pool]/[volume]" | awk -F'"' '/BSD Name/ {print $4;}'</code>

For example, if a zvol named tmdisk1 was created as shown above, the following will get its disk number and store it in the shell variable DISKID:

<code>ioreg -trn 'ZVOL Backup/tmdisk1' | awk -F'"' '/BSD Name/ {print $4;}'</code>

3. Format the zvol as a HFS+, using its disk number from step 2:

<code>sudo diskutil eraseDisk JHFS+ [OSX Disk Name] [disk number]</code>

For example, if the zvol was disk5 in OSX, the following will format that disk as a HFS+ disk named "TimeMachine":

<code>sudo diskutil eraseDisk JHFS+ "TimeMachine" disk5</code>

4. Set the zvol as a Time Machine destination:

<code>sudo tmutil setdestination -a /Volumes/[OSX Disk Name]</code>

For example, if the zvol was formatted as HFS+ named "TimeMachine" in step 3, the following will set it as a Time Machine destination:

<code>sudo tmutil setdestination -a /Volumes/TimeMachine</code>

5. (Optional) Enable encryption for Time Machine backups on the zvol:

5.1 Get the disk identifier for the HFS+ disk on the zvol:

<code>diskutil list [OSX Disk Name] | awk '/[OSX Disk Name]/ { print $NF;}'</code>

For example, if the zvol was formatted as HFS+ named "TimeMachine" in step 3, the following will get its disk identifier:

<code>diskutil list TimeMachine | awk '/TimeMachine/ { print $NF; }'</code>

5.2 Convert the HFS+ disk to use encryption:

<code>sudo diskutil coreStorage convert [Disk ID] -passphrase</code>

This will ask for the encryption password (which may be stored in the Keychain). For example, if the if the zvol was formatted as HFS+ named "TimeMachine" in step 3, whose disk identifier was returned as disk5s2 in step 5.1, the following will convert the disk to use encryption:

<code>sudo diskutil coreStorage convert disk5s2 -passphrase</code>

Please note, this process can take a considerable amount of time (several hours for a 1TB volume).

Time Machine Backups

2018-01-06T02:51:26Z

Srirangav: reformat text and designate commands using the html code tag

== Time Machine Backups ==
Here's one approach to using ZFS for your Time Machine Backups.

Because Time Machine doesn't recognize ZFS datasets as a compatible disk for Time Machine backups, as a work around, we create an HFS+ sparsebundle disk image, store it on a ZFS dataset, and set the mounted image as a backup destination (no "TMShowUnsupportedNetworkVolumes" needed).

1. Create, and mount, a sparsebundle from a ZFS dataset (e.g., with makeImage.sh or Disk Utility.app).

2. Set the sparsebundle as the (active) backup destination:

<code># tmutil setdestination -a /Volumes/[sparse bundle volume name]</code>

While it has been discussed in heated arguments (e.g., https://github.com/openzfsonosx/zfs/issues/66) I still believe there's at least one ZFS feature I'd like to test with Time Machine: compression.

The hypothesis being that an HFS+ sparsebundle stored on a compressed (gzip, lz4), deduped dataset should
yield a compression ratio > 1.0 (previously observed 1.4 with compression=on, dedup=off, FreeBSD network Time Machine drives).

Encryption

2018-01-06T02:45:47Z

Srirangav: move timemachine backups to a separate wikipage

Encryption is now native to ZFS, and it is recommended to use that for greater flexibility and compatibility. See the "zpool create -O encryption=on" feature.

However, the core storage documentation will remain here for those who prefer that method.

== Native ZFS Encryption ==

On a zpool that supports encryption, encryption may be enabled as follows:

<code># zpool set feature@encryption=enabled [pool]</code>

On an zpool that supports encryption, a encrypted zfs dataset may be created as follows:

<code># zfs create -o encryption=on -o keylocation=prompt -o keyformat=passphrase [dataset]</code>

This will prompt for the encryption passphrase for this zfs dataset. Other options for the location of the encryption key and its format can be found in the zfs(1M) manpage.

An encrypted zfs dataset may be mounted as follows:

<code># zfs mount -l [dataset]</code>

This will prompt for the encryption passphrase for this zfs dataset. If the encryption passphrase is stored in the Keychain as a generic password under the name of the dataset, security(1) may be used to retrieve the passphrase as follows:

<code># security find-generic-password -a [dataset] -w | zfs mount -l [dataset]</code>

Additional helpful information about zfs encryption can be found in the [https://blog.heckel.xyz/2017/01/08/zfs-encryption-openzfs-zfs-on-linux/ How-To: Using ZFS Encryption at Rest in OpenZFS (ZFS on Linux, ZFS on FreeBSD, …)].

== Core Storage (File Vault 2) ==

Although the upstream OpenZFS project lists [http://open-zfs.org/wiki/Projects#Platform_agnostic_encryption_support platform-agnostic encryption support] at the ZFS dataset level as a possible future enhancement, OS X already offers a feature called [http://support.apple.com/kb/ht4790 FileVault 2], which provides built-in support for XTS-AES 128 encryption at the block level as part of Core Storage volume management.

This is the OS X analogue of the following block-level encryption systems on other operating systems that support ZFS:
* FreeBSD: geli
* Linux: LUKS

The overall procedure is, as follows: convert an empty HFS+ partition to use Core Storage and apply Core Storage encryption. Then use the Core Storage Logical Volume as a device in your zpool by supplying it to "zpool create," "zpool add," "zpool attach," etc.

=== Prerequisites ===
Build ZFS [[Install#Installing_from_source|from source]], or wait for the [[Downloads|next installer]], newer than 1.2.0 (for explanation, see original IRC chat).

=== Caveats ===
You may receive a pop-up claiming the disk isn't readable by this computer. This leads to one step that can be confusing: when unlocking the disk (e.g., on startup), the "bug" will make OS X believe the disk wasn't unlocked, and thus "wiggle," presenting the prompt again.

Assuming you entered your password correctly, the encrypted volume should now be unlocked, despite the misleading wiggle, and you can safely close the dialog box by clicking "Cancel." You'll know for sure the volume is unlocked when you proceed to import your pool, or you can check directly by looking for <code>Encryption Status: Unlocked</code> in the output of <code>diskutil coreStorage list</code>.

=== Steps ===
The initial layout, with disk1 being the external disk (counter-intuitively named "Internal HD") intended as encrypted ZFS device.

# diskutil list
/dev/disk0
#: TYPE NAME SIZE IDENTIFIER
0: GUID_partition_scheme *160.0 GB disk0
1: EFI EFI 209.7 MB disk0s1
2: Apple_HFS Macintosh HD 159.7 GB disk0s2
/dev/disk1
#: TYPE NAME SIZE IDENTIFIER
0: GUID_partition_scheme *1.0 TB disk1
1: EFI EFI 209.7 MB disk1s1
2: Apple_HFS Internal HD 999.9 GB disk1s2

We note that disk1s2 is the partition to be encrypted, and we convert it to Core Storage (think LVM), to enable encryption:

# diskutil coreStorage convert /dev/disk1s2
Started CoreStorage operation on disk1s2 Internal HD
Resizing disk to fit Core Storage headers
Creating Core Storage Logical Volume Group
Attempting to unmount disk1s2
Switching disk1s2 to Core Storage
Waiting for Logical Volume to appear
Mounting Logical Volume
Core Storage LVG UUID: 4690972A-484E-42E2-B72D-933A58E41237
Core Storage PV UUID: 22A1A783-01BA-4ABA-B4A3-2A9146506519
Core Storage LV UUID: F6D16BFE-B6E9-4A9B-BC03-E5CD03772C44
Core Storage disk: disk2
Finished CoreStorage operation on disk1s2 Internal HD

Note that we converted the existing unencrypted HFS+ partition.

Next, we encrypt the logical volume, our Core Storage disk, disk2:

:'''Note: If you choose to use diskutil to do the encryption, it will default to the most secure option, which will take a VERY long time with large disks, optionally use the Disk Util UI and set it up as an encrypted HFS+ volume and under advanced change to least secure (good for new disks)'''

# diskutil coreStorage encryptVolume /dev/disk2
New passphrase for existing volume:
Confirm new passphrase:
The Core Storage Logical Volume UUID is F6D16BFE-B6E9-4A9B-BC03-E5CD03772C44
Started CoreStorage operation on disk2 Internal HD
Scheduling encryption of Core Storage Logical Volume
Core Storage LV UUID: F6D16BFE-B6E9-4A9B-BC03-E5CD03772C44
Finished CoreStorage operation on disk2 Internal HD

Note that we used disk2, the logical volume, not disk1s2.

This can and will take a while to complete. You can check the status by issuing:
# diskutil coreStorage list | grep Conversion

Until it's done:
Conversion Status: Complete
Conversion Progress: -none-

Your partition layout should now look like:

# diskutil list
/dev/disk0
#: TYPE NAME SIZE IDENTIFIER
0: GUID_partition_scheme *160.0 GB disk0
1: EFI EFI 209.7 MB disk0s1
2: Apple_HFS Macintosh HD 159.7 GB disk0s2
/dev/disk1
#: TYPE NAME SIZE IDENTIFIER
0: GUID_partition_scheme *1.0 TB disk1
1: EFI EFI 209.7 MB disk1s1
2: Apple_CoreStorage 999.9 GB disk1s2
3: Apple_Boot Boot OS X 134.2 MB disk1s3
/dev/disk2
#: TYPE NAME SIZE IDENTIFIER
0: Apple_HFS *999.5 GB disk2

disk2 being our encrypted, unlocked HFS+ device. If you have yet to be prompted for the passphrase by OS X, now would be a good time to restart your Mac and try it out.

Lastly, we'll prepare the volume for ZFS, by unmounting /dev/disk2:

# mount
...
/dev/disk2 on /Volumes/Internal HD (hfs, local, journaled)
# diskutil unmount "/Volumes/Internal HD"

You can now follow the article on [[Zpool#Creating_a_pool|creating a pool]]. As a simple example, you might

<syntaxhighlight lang="text">
# zpool list
no pools available
# zpool create -f -o ashift=12 ZFS_VOLUME /dev/disk2
# zpool list
ZFS_VOLUME 928G 20.8G 907G 2% 1.00x ONLINE -
</syntaxhighlight>

Also note that you '''don't''' need to worry about changing the partition type in this case to ZFS as described in the [[Suppressing_the_annoying_pop-up|Suppressing the annoying pop-up]] wiki page.

=== Reason to "use latest" ===
<syntaxhighlight lang="text">
<ilovezfs> If you want encryption you have a few options
<ilovezfs> https://github.com/zfsrogue/osx-zfs-crypto
<lundman> :)
<ilovezfs> or you can do what cbreak said, and use an encrypted sparsebundle
<ilovezfs> (I'd give it its own ZFS file system)
<ilovezfs> or you can create a ZVOL, and put an encrypted Core Storage/Filevault 2 HFS+
file system on it
<ilovezfs> or you can put the pool itself on top of Core Storage.
<ilovezfs> The last option you should not do with the installer version.
<ilovezfs> But wait for the next installer if that's the route you want to go
<ilovezfs> or build from source.
<aandy> Ah, interesting. Does FileVault 2 require HFS+? Not that it'd surprise me.
<ilovezfs> No it does not.
<ilovezfs> But it is not possible to set other Content Hints
<ilovezfs> so it will always say HFS+ even if you do put ZFS on your logical volumes.
<ilovezfs> So basically the procedure is to format the volume HFS+.
<ilovezfs> Then run 'diskutil coreStorage convert' on it.
<ilovezfs> Then you can encrypt it.
<ilovezfs> Then you unmount the HFS+
<ilovezfs> and zpool create on the logical volume.
<ilovezfs> And you should be good to go.
<aandy> On the original HFS+ partition, right?
<ilovezfs> Right.
<ilovezfs> But I'd encrypt first
<ilovezfs> then put ZFS on it.
<aandy> Right. Perfect.
<ilovezfs> diskutil coreStorage convert ...
<ilovezfs> diskutil coreStorage encryptVolume ...
<ilovezfs> etc.
<ilovezfs> The reason not to use the installer version, is that it will attempt to
partition the Core Storage Logical Volume.
<ilovezfs> But since 10.8.5 and after, Apple doesn't like that
<ilovezfs> so we added new code to detect Core Storage and not partition if it sees it's
Core Storage.
</syntaxhighlight>

Encryption

2018-01-06T02:45:11Z

Srirangav: move timemachine backups to a separate wikipage

Time Machine Backups

2018-01-06T02:43:22Z

Srirangav: move time machine backups from the encryption wiki page to a separate page.

=== Time Machine Backups ===
Here's one approach to using ZFS for your Time Machine Backups:

While it has been discussed in heated arguments (e.g., https://github.com/openzfsonosx/zfs/issues/66) I still believe there's at least one ZFS feature I'd like to test with Time Machine: compression.

The hypothesis being:
an HFS+ sparsebundle stored on a compressed (gzip, lz4), deduped dataset should
yield a compression ratio > 1.0.
(previously observed 1.4 with compression=on, dedup=off, FreeBSD network Time Machine drives).

To work around compatible disks for Time Machine, we create an HFS+ sparsebundle, store it on ZFS, and set the mounted image as a backup destination – no "TMShowUnsupportedNetworkVolumes" needed.

1. Create, and mount, a sparsebundle from your ZFS filesystem (e.g., with makeImage.sh or Disk Utility).

2. Set your sparsebundle as the (active) backup destination # tmutil setdestination -a /Volumes/Time\ Machine\ Backups

Documentation

2018-01-06T02:40:15Z

Srirangav: add new section for time machine backups

[[Category:About O3X]]
[[Category:Getting and installing O3X]]

== Documentation ==

General OpenZFS usage can be found on the [http://open-zfs.org OpenZFS wiki]. The main focus of this documentation is on OS X platform specifics.

* [[zpool#Creating_a_pool|Creating a pool]]
* [[zpool#Unplugging_a_pool|Unplugging a pool]]
* [[zpool#Feature_flags|Feature flags]]
* [[Device names]]
* [[Pool upgrade]]
* [[Memory utilization]]
* [[Creating user privileges]]
* [[Stopping Spotlight etc. from changing ZFS without permission]]
* [[Encryption]]
* [[Bash completion]]
* [[DegradedPool|Degraded pool]]
* [[panic|Kernel panic]]
* [[ZFS on Boot]]
* [[Time Machine Backups]]
* [[Expanding ZFS vdevs]]

Encryption

2018-01-06T02:33:02Z

Srirangav: add instructions on creating native zfs encryption dataset