Page 1 of 1

Can't mount ZFS root on boot, mbpro catalina, zfs 1.9.4 & 2

PostPosted: Wed Dec 30, 2020 10:18 am
by derzahl
Hi,I have been having trouble setting up boot from ZFS on OSX catalina(2018 13" macbook pro). Initially did a fresh install of the latest catalina from recovery to a USB drive. Booted from there and installed https://openzfsonosx.org/w/images/6/6f/ ... _1.9.4.dmg for Catalina.

I then followed the ZFS on Boot directions from https://openzfsonosx.org/wiki/ZFS_on_Boot#Booting_ZFS copy/paste word-forword multiple times to rsync and install my working boot of catalina from USB to internal SSD(/dev/disk0). Full disclosure, the one deviation was that I created rpool/ROOT/Catalina with casesensitivity=sensitive, as I have always run OSX on case sensitive ufs, hfs and apfs volumes. However I got codesigning validation errors when booting and the system trying to use mount_zfs. I read forum posts that this was solved sometime back during the 1.9.2 release. hmm.

After doing some digging through the forums I found that OpenZFSonOsX-2.0.0-Catalina-10.15.pkg had actually been release despite not being found on the main download page. So I gave this a try as well, about 4 times, going though the previous wiki document ZFS_on_Boot. Here the results were different. Codesigning was working but the system was unable to mount the root volume. It tries over and over again, finally showing the 'prohibitory symbol'.

The relevent error messages from boot I can see are:

rootvnode is not set something about spa_async_request repeated over and over. No luck finding much nfo on these. Booting in debug ends up doing a full multipage dump over evrything after awhile but I am unsure how to pause the scrolling to make more sense of it or capture it somehow.

Could anyone provide any insight into what I am doing wrong or how to further troubleshoot the issue? Thanks!

Re: Can't mount ZFS root on boot, mbpro catalina, zfs 1.9.4

PostPosted: Thu Dec 31, 2020 11:38 am
by derzahl
From my experience using ZFS root on linux and BSD I also tried to set the mountpoint for rpool/ROOT/Catalina to "/" instead of "legacy" like the directions, since that seemed odd. Also made sure to export the zpool before reboot. Still no go. I'm guessing there should be somewhere in the init process that runs a 'zpool import -a' and 'zfs mount -a' and thats not happening. Any idea where that should be on OSX?

Re: Can't mount ZFS root on boot, mbpro catalina, zfs 1.9.4

PostPosted: Sat Jan 02, 2021 5:22 pm
by abcslayer
I have Catalina boot from ZVOL with OpenZFS 1.9.4 but failed to boot with 2.0 (similar error message you have seen)
How do you install the kext and utilities/scripts of OpenZFS on your Catalina dataset? I think you can double check the permission of them plus check if the prelinkedkernel has been created successfully or not.

Re: Can't mount ZFS root on boot, mbpro catalina, zfs 1.9.4

PostPosted: Wed Jan 06, 2021 4:38 pm
by lundman
Boot should work with 1.9.4, but you are in new territories with 2.0, and perhaps, Catalina. I've not had
a chance to try it myself, even if the code hasn't changed, the OS has and we generally need to tweak things
with each release (with boot support).

Re: Can't mount ZFS root on boot, mbpro catalina, zfs 1.9.4

PostPosted: Thu Jan 07, 2021 9:43 am
by derzahl
lundman wrote:Boot should work with 1.9.4, but you are in new territories with 2.0, and perhaps, Catalina. I've not had
a chance to try it myself, even if the code hasn't changed, the OS has and we generally need to tweak things
with each release (with boot support).


Ah crap. Well I originally tried 1.9.4 and had problems there. It was saying mount_zfs wasnt signed when I tried to boot. I searched a found a thread from a year or so ago with that exact same issues, i saw that you guy released a fix all the way back with 1.9.2 i think. So it made no sense to me that 1.9.4 still would have a signing issue unless it was perhaps because I was using too new of a version of Catalina and they updated their signing process.

So when I realized 2.0 was now released, I thought great! That should have the most up to date signing etc. Any sure enough, I don't get a signing error anymore - it just gives me the vague "rootvnode is not set" over and over. So I now have my zpool created with 2.0 and Artix linux all set up on one dataset, which I have been using, and catalina installed on another.

So does anyone have any suggestions I can try? I would prefer to stay on 2.0 and soldier on then to go back to 1.9.4 and have to create a new zpool. Thanks for the help!

Re: Can't mount ZFS root on boot, mbpro catalina, zfs 1.9.4

PostPosted: Thu Jan 07, 2021 9:56 am
by derzahl
abcslayer wrote:I have Catalina boot from ZVOL with OpenZFS 1.9.4 but failed to boot with 2.0 (similar error message you have seen)
How do you install the kext and utilities/scripts of OpenZFS on your Catalina dataset? I think you can double check the permission of them plus check if the prelinkedkernel has been created successfully or not.


Interesting. Im using a dataset not a zvol. I just followed the wiki here https://openzfsonosx.org/wiki/ZFS_on_Boot ...like 6 times, thinking i mustve done something wrong but got the same results every time.

So I just ran:

kextcache -c "/Volumes/ZFSBoot/System/Library/PrelinkedKernels/prelinkedkernel" -K "/Volumes/ZFSBoot/System/Library/Kernels/kernel" -l -- "/Volumes/ZFSBoot/System/Library/Extensions" "/Volumes/ZFSBoot/Library/Extensions/"

and also forthe apple boot partition:

diskutil mount disk1s3
mkdir -p /Volumes/boot/com.apple.boot.R/System/Library/Caches/com.apple.kext.caches/Startup
kextcache -c "/Volumes/boot/com.apple.boot.R/System/Library/PrelinkedKernels/prelinkedkernel" -K "/System/Library/Kernels/kernel" -l -- "/System/Library/Extensions" "/Library/Extensions/"

Thought I am a little unclear why we have to do this. Which kernel is actually being booted? The one on the hfs+ boot partition or the one on the ZFS root dataset?

I also made sure that both com.apple.Boot.plist files were up to date and correct:
/Volumes/boot/com.apple.boot.R/Library/Preferences/SystemConfiguration/com.apple.Boot.plist
/Volumes/ZFSBoot/Library/Preferences/SystemConfiguration/com.apple.Boot.plist

I even updated the UUID to be correct, even though the wiki says it doesnt matter.

Oh,one thing I did deviate from the wiki on, when I create my zpool i set casesensitivity=sensitive instead of insensitive. But I doubt that would matter right? I've always run OXS with case sensitive root filesystems, even back when ufs was an option, and had no issues

Edit:

so Ive booted from the catalina on USB drive to check the kernels. They look ok to me but im not really an expert on what kernels should look like on OSX. There only odd thing to me is the different sizes:

~ % ls -lah@e /Volumes/boot/com.apple.boot.R/System/Library/PrelinkedKernels/prelinkedkernel
-rw-r--r-- 1 root wheel 17M Dec 29 14:34 /Volumes/boot/com.apple.boot.R/System/Library/PrelinkedKernels/prelinkedkernel
~ % ls -lah@e /Volumes/ZFSBoot/System/Library/PrelinkedKernels/prelinkedkernel
-rw-r--r-- 1 root wheel 27M Dec 29 14:34 /Volumes/ZFSBoot/System/Library/PrelinkedKernels/prelinkedkernel

Also here is the contents of com.apple.Boot.plist:
~ % cat /Volumes/boot/com.apple.boot.R/Library/Preferences/SystemConfiguration/com.apple.Boot.plist
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
<key>Kernel Flags</key>
<string>-v io=0xffffffff debug=0x144 keepsyms=y zfs_boot=rpool</string>
<key>Root UUID</key>
<string>60b29b18-6b58-46f9-8b5d-fef64d36ee69</string>
</dict>
</plist>


~ % cat /Volumes/ZFSBoot/Library/Preferences/SystemConfiguration/com.apple.Boot.plist
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
<key>Kernel Flags</key>
<string>-v keepsyms=y zfs_boot=rpool</string>
<key>Root UUID</key>
<string>60b29b18-6b58-46f9-8b5d-fef64d36ee69</string>
</dict>
</plist>

Re: Can't mount ZFS root on boot, mbpro catalina, zfs 1.9.4

PostPosted: Thu May 06, 2021 4:05 am
by pat
Did you ever get this working? I am also unable to get a fresh Catalina install to boot.