OpenZFS on my MBP's internal SSD (Macintosh HD - Data?)

New to OpenZFS on OS X (Or ZFS in general)? Ask your questions here!

OpenZFS on my MBP's internal SSD (Macintosh HD - Data?)

Postby pat » Tue May 04, 2021 11:17 am

I have a FreeBSD file server with ZFS on root. After getting used to ZFS for a while, I find myself wanting it on my 2015 MacBook Pro. I just have a single 1 TB SSD, so this won't be about redundancy. I do, however, want to make use of snapshots, and send the snapshots to my FreeBSD file server.

I've read through the wiki and a bunch of forum posts, and the scenarios I come across seem to be either raidz data disks, or zfs on boot. I am interested in more of a "middle ground".

What I am thinking of is keeping the "Macintosh HD" volume at its default read-only APFS, and making the "Macintosh HD - Data" volume ZFS. So what I _think_ this would entail is something like:

1. Back up the volumes separately
2. Boot from my backup, launch disk utility. Partition internal disk to have an APFS partition, and an HFS partition.
3. Restore "Macintosh HD" to APFS partition.
4. Create a ZFS pool from the HFS partition.
5. Restore "Macintosh HD - Data" to the new ZFS pool.

One thing that I think is missing from this is, how will MacOS know to mount the Data pool? I'm assuming that all of the ZFS config and initialization will be on Data. So I have a chicken-and-egg problem.

So that's one approach - create as big of a ZFS partition as possible. Another possibility is to retain the default "Macintosh HD" and "Macintosh HD - Data" volume, but create a separate ZFS partition for my data.

Anyway, does anyone have any guidance for using ZFS for a single, internal drive? I didn't really find anything in the wiki about this scenario, beyond the ZFS on Root (and maybe that's the way to go?)

disk.png
disk.png (123.64 KiB) Viewed 8123 times
pat
 
Posts: 13
Joined: Mon May 03, 2021 11:58 pm

Re: OpenZFS on my MBP's internal SSD (Macintosh HD - Data?)

Postby soongrazer » Mon May 10, 2021 10:38 pm

I have been using OpenZFS on my MacBook Pro 2017 and it works great with the exception of iCloud Drive.

I do not think macOS can boot from a ZFS filesystem so I kept macOS on an APFS volume.

I use the ZFS file system mainly for /Users so that I can take snapshots and backups and create encrypted file systems. I have separate ZFS file systems for each user. I also have two ZFS file systems which are mounted like external disks. I cannot use iCloud Drive. I am using macOS Catalina.

macpool is the name of zpool.
Screenshot 2021-05-11 at 12.28.47.png
Internal SSD with a ZFS partition
Screenshot 2021-05-11 at 12.28.47.png (613.06 KiB) Viewed 8083 times


# zfs list
Code: Select all
NAME                               USED  AVAIL     REFER  MOUNTPOINT
macpool                            300G  35.4G      112K  /Volumes/macpool
macpool/Users                      255G  35.4G     2.73M  /Users
macpool/Users/brother             6.76G  35.4G     6.04G  /Users/brother
macpool/Users/myself               244G  35.4G     69.2G  /Users/myself
macpool/Users/myself/Downloads    52.1G  35.4G     8.54G  /Users/myself/Downloads
macpool/Users/myself/Encrypted    52.3M  35.4G     42.1M  /Users/myself/Encrypted
macpool/Users/visitor             4.09G  35.4G     2.97G  /Users/visitor
macpool/Volumes                   44.0G  35.4G       96K  /Volumes
macpool/Volumes/Scratch           12.0G  35.4G     11.7G  /Volumes/Scratch
macpool/Volumes/Storage           32.0G  35.4G     32.0G  /Volumes/Storage


# zfs get -t filesystem canmount
Code: Select all
NAME                              PROPERTY  VALUE     SOURCE
macpool                           canmount  off       local
macpool/Users                     canmount  on        default
macpool/Users/brother             canmount  noauto    local
macpool/Users/myself              canmount  on        default
macpool/Users/myself/Downloads    canmount  on        default
macpool/Users/myself/Encrypted    canmount  on        default
macpool/Users/visitor             canmount  on        default
macpool/Volumes                   canmount  off       local
macpool/Volumes/Scratch           canmount  noauto    local
macpool/Volumes/Storage           canmount  on        default


I keep /Applications on Data APFS volume. With ZFS file system, the Dock does not behave well. After migrating /Users/* to ZFS, I can still use the installed apps.
To set up like my structure, take the following steps.

1. Back up data to an external drive, and sign out Apple ID if needed.
2. Enable root user. Install OpenZFS.
3. Delete user accounts including the home directories. Do not delete the Data APFS volume.
4. Partition disk. Choose a format other than ZFS dataset.
5. Disable System Integrity Protection to allow mounting on /Users.
6. Log in to root user.
7. Create zpool on the new partition. I set the compression property to on.
8. Create ZFS file system for /Users with mountpoint property set to /Users. You may create descendent file systems for each users.
9. Verify the file systems are mounted after boot with root user.
10. Add user accounts.
11. Log in to user accounts and check if the home directory is a ZFS file system.
12. Restore backed up data.

Important! I have to wait about 5 seconds before logging into a user account with the home directory on ZFS after boot. Otherwise, the user is logged in before the file systems get mounted.
soongrazer
 
Posts: 6
Joined: Sun Dec 06, 2020 5:58 pm

Re: OpenZFS on my MBP's internal SSD (Macintosh HD - Data?)

Postby pat » Tue May 11, 2021 3:49 pm

That is super helpful, thank you.

I am curious about iCloud Drive - what problems have you run in to with it? Have you ever tried mimic HFS/APFS?

Also what happens if you log in before the ZFS pool mounts? Does it cause major problems, or does the login just fail and you try again?

Also it looks like you don’t have the /Users or individual user datasets encrypted - that you have a special Encrypted dataset. Is it possible to use MacOS full-disk encryption along with ZFS (so if I lost the laptop someone couldn’t just pull out the SSD and access files)?
pat
 
Posts: 13
Joined: Mon May 03, 2021 11:58 pm

Re: OpenZFS on my MBP's internal SSD (Macintosh HD - Data?)

Postby soongrazer » Wed May 12, 2021 8:53 am

You are welcome.

iCloud Drive syncing just fails even with HFS/APFS mimicking.

When I log in before the mounting is completed, these happen. Some system services may ask for login keychain password. Some preferences in Appearance and Mission Control are reset. Apple ID may ask to sign in again.

File Vault 2 seems to work only on Apple's file systems. So, it won't work with ZFS I think.

If you encrypt /Users and it's descendants, you will have to use a privileged user whose home directory is not on an encrypted file system to load keys and mount the file systems first.
soongrazer
 
Posts: 6
Joined: Sun Dec 06, 2020 5:58 pm

Re: OpenZFS on my MBP's internal SSD (Macintosh HD - Data?)

Postby pat » Wed May 19, 2021 9:51 pm

Okay very good to know. I use iCloud Drive a bit, and other iCloud stuff a lot (Photos, Messages, Safari bookmark sync, etc) so it sounds like I should avoid ZFS for my home dir and instead just use it for specific data dirs.

Thank you for sharing your experience!
pat
 
Posts: 13
Joined: Mon May 03, 2021 11:58 pm

Re: OpenZFS on my MBP's internal SSD (Macintosh HD - Data?)

Postby soongrazer » Fri May 21, 2021 3:41 am

The affect seems to apply only to iCloud Drive, not other services that uses iCloud. I can still use Calendar, Reminders, Photos, Safari, etc.
soongrazer
 
Posts: 6
Joined: Sun Dec 06, 2020 5:58 pm

Re: OpenZFS on my MBP's internal SSD (Macintosh HD - Data?)

Postby pat » Fri May 21, 2021 7:50 pm

Okay. How do you handle iCloud Drive then (if you use it at all)? Create a folder somewhere on an APFS volume and symlink the iCloud Drive folder to it?

Have you used Dropbox with ZFS? I need some kind of cloud sync - though I’m open to finding alternatives to iCloud Drive / Dropbox.
pat
 
Posts: 13
Joined: Mon May 03, 2021 11:58 pm

Re: OpenZFS on my MBP's internal SSD (Macintosh HD - Data?)

Postby Arne » Fri May 21, 2021 11:11 pm

pat wrote:Okay. How do you handle iCloud Drive then (if you use it at all)? Create a folder somewhere on an APFS volume and symlink the iCloud Drive folder to it?

Have you used Dropbox with ZFS? I need some kind of cloud sync - though I’m open to finding alternatives to iCloud Drive / Dropbox.


I'm using MegaSync and it works with zfs.
My "home" is still on HFS. I just use some folders in my home directory mounted from zfs (Downloads, Music, Mega, ...). So my iCloud drive is on HFS, too.

Maybe a sparse.bundle or sparse.image is a way of using iCloud on zfs. I read about using those so that Timemachine works on zfs.
My system: Mini 2009 (early) with El-Capitan 10.11.6
Arne
 
Posts: 29
Joined: Mon Oct 29, 2018 7:59 am

Re: OpenZFS on my MBP's internal SSD (Macintosh HD - Data?)

Postby nodarkthings » Sat May 22, 2021 2:02 am

pat wrote:Anyway, does anyone have any guidance for using ZFS for a single, internal drive? I didn't really find anything in the wiki about this scenario, beyond the ZFS on Root (and maybe that's the way to go?


I gave up the ZFS on Root thing as there aren't many testimonies of success — and MacOS is already tricky enough for not adding a layer of complexity if not strictly needed. :mrgreen:
I didn't put /Users on ZFS because I like to autologin in my main user, and it was problematic when I started using ZFS (I don't know if it's still nowadays...)
So I mainly use ZFS for data, various datasets for Audio, OS backups, VMs and any heavy static folder from ~/Library/Application Support.
To overcome various issues, I rely on some sparsebundles, especially one for Time Machine that is automatically opened with a user LaunchAgents containing:
Code: Select all
<dict>
   <key>Disabled</key>
   <false/>
   <key>Label</key>
   <string>MyZFS.TM</string>
   <key>ProgramArguments</key>
   <array>
      <string>open</string>
      <string>/Volumes/MyZFSpool/Time Machine.sparsebundle</string>
   </array>
   <key>StartOnMount</key>
   <true/>
</dict>


The use of mountpoints can really improve what you can do — they can be different from their names, in case you don't know; for example, here are a few datasets that I mount where I want:
MyZFS/Videos => mountpoint=/Users/Myself/Movies
MyZFS/OfficeZFS => mountpoint=/Users/Myself/Documents/Données\ utilisateurs\ Microsoft (for MS Office 2011 identities — I didn't have success with MS Office 2016, though, but discovered that the same mountpoint thing exists too in MacOS itself, thanks to the fstab file, and Office 2016 works ok with it).

I also use plenty of symlinks pointing to folders inside the sparsebundles, for Downloads, Dropbox, Pictures, etc. Note that I do this for some reason, but unless you need it, I'd rather suggest that you use mountpoints where possible (as I mentioned above, some app might not appreciate but they're rather few, try and see ;) ).

EDIT: I forgot to mention that I'm still using 10.11, for the time being, but I think most of what I wrote is still valid with recent OSes... :D
nodarkthings
 
Posts: 174
Joined: Mon Jan 26, 2015 10:32 am

Re: OpenZFS on my MBP's internal SSD (Macintosh HD - Data?)

Postby soongrazer » Sat May 22, 2021 8:09 am

pat wrote:Okay. How do you handle iCloud Drive then (if you use it at all)? Create a folder somewhere on an APFS volume and symlink the iCloud Drive folder to it?

Have you used Dropbox with ZFS? I need some kind of cloud sync - though I’m open to finding alternatives to iCloud Drive / Dropbox.


I don't use iCloud Drive to sync with my Mac since I started use ZFS. I hadn't have time to look for the alternatives.

I think iCloud Drive issue is due to the fact that the home folder is not on the same volume as the system's. You can see the message in Storage Management.
Screenshot 2021-05-22 at 22.36.15.png
Storage Management
Screenshot 2021-05-22 at 22.36.15.png (240.21 KiB) Viewed 7776 times
soongrazer
 
Posts: 6
Joined: Sun Dec 06, 2020 5:58 pm

Next

Return to Absolute Beginners Section

Who is online

Users browsing this forum: No registered users and 2 guests