Can't mount ZFS root on boot, mbpro catalina, zfs 1.9.4 & 2

All your general support questions for OpenZFS on OS X.

Can't mount ZFS root on boot, mbpro catalina, zfs 1.9.4 & 2

Postby derzahl » Wed Dec 30, 2020 10:18 am

Hi,I have been having trouble setting up boot from ZFS on OSX catalina(2018 13" macbook pro). Initially did a fresh install of the latest catalina from recovery to a USB drive. Booted from there and installed https://openzfsonosx.org/w/images/6/6f/ ... _1.9.4.dmg for Catalina.

I then followed the ZFS on Boot directions from https://openzfsonosx.org/wiki/ZFS_on_Boot#Booting_ZFS copy/paste word-forword multiple times to rsync and install my working boot of catalina from USB to internal SSD(/dev/disk0). Full disclosure, the one deviation was that I created rpool/ROOT/Catalina with casesensitivity=sensitive, as I have always run OSX on case sensitive ufs, hfs and apfs volumes. However I got codesigning validation errors when booting and the system trying to use mount_zfs. I read forum posts that this was solved sometime back during the 1.9.2 release. hmm.

After doing some digging through the forums I found that OpenZFSonOsX-2.0.0-Catalina-10.15.pkg had actually been release despite not being found on the main download page. So I gave this a try as well, about 4 times, going though the previous wiki document ZFS_on_Boot. Here the results were different. Codesigning was working but the system was unable to mount the root volume. It tries over and over again, finally showing the 'prohibitory symbol'.

The relevent error messages from boot I can see are:

rootvnode is not set something about spa_async_request repeated over and over. No luck finding much nfo on these. Booting in debug ends up doing a full multipage dump over evrything after awhile but I am unsure how to pause the scrolling to make more sense of it or capture it somehow.

Could anyone provide any insight into what I am doing wrong or how to further troubleshoot the issue? Thanks!
derzahl
 
Posts: 7
Joined: Wed Dec 30, 2020 9:51 am

Re: Can't mount ZFS root on boot, mbpro catalina, zfs 1.9.4

Postby derzahl » Thu Dec 31, 2020 11:38 am

From my experience using ZFS root on linux and BSD I also tried to set the mountpoint for rpool/ROOT/Catalina to "/" instead of "legacy" like the directions, since that seemed odd. Also made sure to export the zpool before reboot. Still no go. I'm guessing there should be somewhere in the init process that runs a 'zpool import -a' and 'zfs mount -a' and thats not happening. Any idea where that should be on OSX?
derzahl
 
Posts: 7
Joined: Wed Dec 30, 2020 9:51 am

Re: Can't mount ZFS root on boot, mbpro catalina, zfs 1.9.4

Postby abcslayer » Sat Jan 02, 2021 5:22 pm

I have Catalina boot from ZVOL with OpenZFS 1.9.4 but failed to boot with 2.0 (similar error message you have seen)
How do you install the kext and utilities/scripts of OpenZFS on your Catalina dataset? I think you can double check the permission of them plus check if the prelinkedkernel has been created successfully or not.
abcslayer
 
Posts: 2
Joined: Thu Dec 10, 2020 7:16 am

Re: Can't mount ZFS root on boot, mbpro catalina, zfs 1.9.4

Postby lundman » Wed Jan 06, 2021 4:38 pm

Boot should work with 1.9.4, but you are in new territories with 2.0, and perhaps, Catalina. I've not had
a chance to try it myself, even if the code hasn't changed, the OS has and we generally need to tweak things
with each release (with boot support).
User avatar
lundman
 
Posts: 1337
Joined: Thu Mar 06, 2014 2:05 pm
Location: Tokyo, Japan

Re: Can't mount ZFS root on boot, mbpro catalina, zfs 1.9.4

Postby derzahl » Thu Jan 07, 2021 9:43 am

lundman wrote:Boot should work with 1.9.4, but you are in new territories with 2.0, and perhaps, Catalina. I've not had
a chance to try it myself, even if the code hasn't changed, the OS has and we generally need to tweak things
with each release (with boot support).


Ah crap. Well I originally tried 1.9.4 and had problems there. It was saying mount_zfs wasnt signed when I tried to boot. I searched a found a thread from a year or so ago with that exact same issues, i saw that you guy released a fix all the way back with 1.9.2 i think. So it made no sense to me that 1.9.4 still would have a signing issue unless it was perhaps because I was using too new of a version of Catalina and they updated their signing process.

So when I realized 2.0 was now released, I thought great! That should have the most up to date signing etc. Any sure enough, I don't get a signing error anymore - it just gives me the vague "rootvnode is not set" over and over. So I now have my zpool created with 2.0 and Artix linux all set up on one dataset, which I have been using, and catalina installed on another.

So does anyone have any suggestions I can try? I would prefer to stay on 2.0 and soldier on then to go back to 1.9.4 and have to create a new zpool. Thanks for the help!
derzahl
 
Posts: 7
Joined: Wed Dec 30, 2020 9:51 am

Re: Can't mount ZFS root on boot, mbpro catalina, zfs 1.9.4

Postby derzahl » Thu Jan 07, 2021 9:56 am

abcslayer wrote:I have Catalina boot from ZVOL with OpenZFS 1.9.4 but failed to boot with 2.0 (similar error message you have seen)
How do you install the kext and utilities/scripts of OpenZFS on your Catalina dataset? I think you can double check the permission of them plus check if the prelinkedkernel has been created successfully or not.


Interesting. Im using a dataset not a zvol. I just followed the wiki here https://openzfsonosx.org/wiki/ZFS_on_Boot ...like 6 times, thinking i mustve done something wrong but got the same results every time.

So I just ran:

kextcache -c "/Volumes/ZFSBoot/System/Library/PrelinkedKernels/prelinkedkernel" -K "/Volumes/ZFSBoot/System/Library/Kernels/kernel" -l -- "/Volumes/ZFSBoot/System/Library/Extensions" "/Volumes/ZFSBoot/Library/Extensions/"

and also forthe apple boot partition:

diskutil mount disk1s3
mkdir -p /Volumes/boot/com.apple.boot.R/System/Library/Caches/com.apple.kext.caches/Startup
kextcache -c "/Volumes/boot/com.apple.boot.R/System/Library/PrelinkedKernels/prelinkedkernel" -K "/System/Library/Kernels/kernel" -l -- "/System/Library/Extensions" "/Library/Extensions/"

Thought I am a little unclear why we have to do this. Which kernel is actually being booted? The one on the hfs+ boot partition or the one on the ZFS root dataset?

I also made sure that both com.apple.Boot.plist files were up to date and correct:
/Volumes/boot/com.apple.boot.R/Library/Preferences/SystemConfiguration/com.apple.Boot.plist
/Volumes/ZFSBoot/Library/Preferences/SystemConfiguration/com.apple.Boot.plist

I even updated the UUID to be correct, even though the wiki says it doesnt matter.

Oh,one thing I did deviate from the wiki on, when I create my zpool i set casesensitivity=sensitive instead of insensitive. But I doubt that would matter right? I've always run OXS with case sensitive root filesystems, even back when ufs was an option, and had no issues

Edit:

so Ive booted from the catalina on USB drive to check the kernels. They look ok to me but im not really an expert on what kernels should look like on OSX. There only odd thing to me is the different sizes:

~ % ls -lah@e /Volumes/boot/com.apple.boot.R/System/Library/PrelinkedKernels/prelinkedkernel
-rw-r--r-- 1 root wheel 17M Dec 29 14:34 /Volumes/boot/com.apple.boot.R/System/Library/PrelinkedKernels/prelinkedkernel
~ % ls -lah@e /Volumes/ZFSBoot/System/Library/PrelinkedKernels/prelinkedkernel
-rw-r--r-- 1 root wheel 27M Dec 29 14:34 /Volumes/ZFSBoot/System/Library/PrelinkedKernels/prelinkedkernel

Also here is the contents of com.apple.Boot.plist:
~ % cat /Volumes/boot/com.apple.boot.R/Library/Preferences/SystemConfiguration/com.apple.Boot.plist
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
<key>Kernel Flags</key>
<string>-v io=0xffffffff debug=0x144 keepsyms=y zfs_boot=rpool</string>
<key>Root UUID</key>
<string>60b29b18-6b58-46f9-8b5d-fef64d36ee69</string>
</dict>
</plist>


~ % cat /Volumes/ZFSBoot/Library/Preferences/SystemConfiguration/com.apple.Boot.plist
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
<key>Kernel Flags</key>
<string>-v keepsyms=y zfs_boot=rpool</string>
<key>Root UUID</key>
<string>60b29b18-6b58-46f9-8b5d-fef64d36ee69</string>
</dict>
</plist>
derzahl
 
Posts: 7
Joined: Wed Dec 30, 2020 9:51 am

Re: Can't mount ZFS root on boot, mbpro catalina, zfs 1.9.4

Postby pat » Thu May 06, 2021 4:05 am

Did you ever get this working? I am also unable to get a fresh Catalina install to boot.
pat
 
Posts: 13
Joined: Mon May 03, 2021 11:58 pm


Return to General Help

Who is online

Users browsing this forum: No registered users and 26 guests