Difference between revisions of "ZFS on Boot"
m |
(Move edit boot.plist above creating boot slice, so boot.plist is configured before rsync) |
||
Line 112: | Line 112: | ||
# make install DESTDIR=/stage_dir | # make install DESTDIR=/stage_dir | ||
# rsync -acH /stage_dir/ /Volumes/Capitan/ | # rsync -acH /stage_dir/ /Volumes/Capitan/ | ||
+ | |||
+ | |||
+ | ==== Edit Boot.plist ==== | ||
+ | |||
+ | More examples of the Boot.plist can be found at [[Boot.plist|com.apple.Boot.plist]] | ||
+ | |||
+ | # sudo vim /Volumes/boot/com.apple.boot.R/Library/Preferences/SystemConfiguration/com.apple.Boot.plist | ||
+ | |||
+ | Two keys must be set: '''Kernel Flags''' and '''Root UUID''': | ||
+ | |||
+ | Look for | ||
+ | <key>Kernel Flags</key> | ||
+ | <string></string> | ||
+ | |||
+ | Replace with: | ||
+ | <key>Kernel Flags</key> | ||
+ | <string>-v keepsyms=y zfs_boot=rpool</string> | ||
+ | <key>Root UUID</key> | ||
+ | <string>00000000-0000-0000-0000-000000000000</string> | ||
+ | |||
+ | Any UUID may be used here, I used 5A465320-626F-6F74-2064-657669636500, (which is literally the string "ZFS boot device\0"). | ||
+ | |||
+ | See [[Boot.plist]] for more examples. | ||
+ | |||
+ | [[ZVOL boot]] from HFS+ can be accomplished by using the UUID of the partition containing a standard Mac OS install, for compatibility. | ||
Line 152: | Line 177: | ||
This step does not change the current boot device, unless you add --setBoot. See [[#Blessing the boot device|blessing the boot device]] to change the startup device. | This step does not change the current boot device, unless you add --setBoot. See [[#Blessing the boot device|blessing the boot device]] to change the startup device. | ||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
==== Generate prelinkedkernel ==== | ==== Generate prelinkedkernel ==== |
Revision as of 03:40, 11 July 2016
Contents
ZFS as Boot
Status: Beta Most of the commands listed below must be run as root or with sudo.
Because you have to compile own kexts, they will not be signed. SIP needs to be disabled for Kext signing to work. The next OpenZFSOnOSX will come with --enable-boot compiled KEXTs and be signed.
Disk Layout
Play disk: disk1
I used gptfdisk, but gpt and diskutil can be used also. Create the following partitions on the test boot disk:
Disk /dev/rdisk1: 83886080 sectors, 40.0 GiB Number Start (sector) End (sector) Size Code Name 1 2048 411647 200.0 MiB EF00 EFI System 2 411648 82200575 39.0 GiB BF01 rpool 3 82200576 83886046 823.0 MiB AB00 Apple boot
s1 is not really used, but added for EFI.
s2 is the ZFS pool used for booting. Note: You are not limited to using the name rpool.
s3 is the small HFS boot which will carry the prelinkedkernel to load the kernel with ZFS.
The layout need not be exactly this, but:
The EFI partition should always exist at the beginning of the disk.
The Apple Boot (ab00) partition should directly follow the ZFS partition, and while it can be as small as 50 MB, it is better to use 650MB or more (to imitate a Recovery HD).
Note: at this time, the boot helper partition is not updated by the OS and must be manually updated. It may be possible to script this in a launchd service, or even 'bless' the device so that macOS updates it automatically. Additional development is ongoing.
You cannot use Startup Disk to select the boot disk, see blessing the boot device.
Populating the ZFS Root s2
Root pool
Any pool name may be used - rpool is used here for example. Additional research is needed on atime, casesensitivity, and normalization (and probably more).
# sudo zpool create -f -o ashift=12 -O casesensitivity=insensitive \ -O normalization=formD -O atime=off -O compression=lz4 \ -O mountpoint=none -O canmount=off rpool disk1s2
This step will relabel the disks using InvariantDisks paths (by UUID), so they can be located at boot time.
# sudo zpool export rpool # sudo zpool import -d /var/run/disk/by-id rpool
We strongly recommend NOT using /dev/disk paths due to disk renumbering issues!
Root dataset
Note: com.apple.mimic_hfs is known to NOT be compatible at this time. (It will panic!)
Create ROOT and lower just like IllumOS so that you can easy swap OS snapshots and reboot into something else (for example to run softwareupdate).
# sudo zfs create -o mountpoint=none -o canmount=off rpool/ROOT # sudo zfs create -o mountpoint=legacy rpool/ROOT/Capitan # sudo zpool set bootfs=rpool/ROOT/Capitan rpool
You probably want to relocate your home folder outside of the root dataset, for example rpool/HOME/username, or on ZVOL for compatibility.
# sudo zfs create -o mountpoint=/Users -o canmount=off rpool/HOME # sudo zfs create rpool/HOME/username
Mount it:
# sudo mkdir /Volumes/Capitan # sudo mount_zfs rpool/ROOT/Capitan /Volumes/Capitan
Install macOS
Copy over the OS as you see fit, rsync will generally do. (Remember that etc, tmp and var are symlinks into /private)
# rsync -axH --exclude=".Spotlight-V100" --exclude=".fseventsd" \ --exclude=".vol" / /Volumes/Capitan/
Install ZFS
# sudo mkdir -p /Volumes/Capitan/usr/local # cd /Volumes/Capitan/usr/local # sudo ln -s ../../sbin sbin
Install ZFS with own source repository
# cd $your-zfs-source-area
Install ZFS by getting sources
# Insert lines here, git clone, etc
Install ZFS continued ...
# cd spl # git fetch --all # git checkout spl-boot # ./autogen # ./configure --enable-boot # make # sudo make install DESTDIR=/Volumes/Capitan # cd ../zfs # git fetch --all # git checkout ldi-boot # ./autogen # ./configure --enable-boot # make # sudo make install DESTDIR=/Volumes/Capitan
Or use a stage directory for zfs builds, that can also be done.
# make install DESTDIR=/stage_dir # rsync -acH /stage_dir/ /Volumes/Capitan/
Edit Boot.plist
More examples of the Boot.plist can be found at com.apple.Boot.plist
# sudo vim /Volumes/boot/com.apple.boot.R/Library/Preferences/SystemConfiguration/com.apple.Boot.plist
Two keys must be set: Kernel Flags and Root UUID:
Look for
<key>Kernel Flags</key> <string></string>
Replace with:
<key>Kernel Flags</key> <string>-v keepsyms=y zfs_boot=rpool</string> <key>Root UUID</key> <string>00000000-0000-0000-0000-000000000000</string>
Any UUID may be used here, I used 5A465320-626F-6F74-2064-657669636500, (which is literally the string "ZFS boot device\0").
See Boot.plist for more examples.
ZVOL boot from HFS+ can be accomplished by using the UUID of the partition containing a standard Mac OS install, for compatibility.
Setting up boot slice s3
Format
Journaled HFS+ filesystem, and populated thusly; (Volume named boot here) These can be rsynced from the (normal) booted OS, or from the Capitan mount.
# newfs_hfs -J -v "boot" /dev/disk1s3 # diskutil mount disk1s3 # sudo mdutil -i off /Volumes/boot
Directories
# sudo mkdir -p /Volumes/boot/System/Library/CoreServices # sudo mkdir -p /Volumes/boot/com.apple.boot.R/System/Library/PrelinkedKernels # sudo mkdir -p /Volumes/boot/com.apple.boot.R/Library/Preferences/SystemConfiguration # sudo mkdir -p /Volumes/boot/com.apple.boot.R/usr/standalone/i386
Required Files
# sudo rsync -a /Volumes/Capitan/System/Library/CoreServices/PlatformSupport.plist \ /Volumes/boot/System/Library/CoreServices/PlatformSupport.plist # sudo rsync -a /Volumes/Capitan/System/Library/CoreServices/SystemVersion.plist \ /Volumes/boot/System/Library/CoreServices/SystemVersion.plist # sudo rsync -a /Volumes/Capitan/System/Library/PrelinkedKernels/prelinkedkernel \ /Volumes/boot/com.apple.boot.R/System/Library/PrelinkedKernels/prelinkedkernel # sudo rsync -a /Volumes/Capitan/Library/Preferences/SystemConfiguration/com.apple.Boot.plist \ /Volumes/boot/com.apple.boot.R/Library/Preferences/com.apple.Boot.plist # sudo rsync -a /Volumes/Capitan/usr/standalone/i386/ \ /Volumes/boot/com.apple.boot.R/usr/standalone/i386/
Bless the volume
The boot helper needs to know where to load boot.efi from.
You may use any label in place of ZFS Boot here, it will appear at the option-boot screen.
# sudo bless --folder /Volumes/boot/System/Library/CoreServices \ --file /Volumes/boot/System/Library/CoreServices \ --bootefi /Volumes/Capitan/System/Library/CoreServices/boot.efi \ --label "ZFS Boot"
This step does not change the current boot device, unless you add --setBoot. See blessing the boot device to change the startup device.
Generate prelinkedkernel
Generate caches for kernel
# kextcache -arch x86_64 -local-root -volume-root /Volumes/rpool \ -kernel /Volumes/rpool/System/Library/Kernels/kernel \ -prelinked-kernel /Volumes/rpool/System/Library/PrelinkedKernels/prelinkedkernel \ -- /Volumes/rpool/System/Library/Extensions /Volumes/boot/Library/Extensions
For developers who also have development, or debug kernels, do the same for each one. For example, caches for kernel.development
# kextcache -arch x86_64 -local-root -volume-root /Volumes/rpool \ -kernel /Volumes/rpool/System/Library/Kernels/kernel.development \ -prelinked-kernel /Volumes/rpool/System/Library/PrelinkedKernels/prelinkedkernel.development \ -- /Volumes/rpool/System/Library/Extensions /Volumes/boot/Library/Extensions
Set the boot device
Finally, tell your Mac to use the boot helper on boot. You may want to try the --nextonly option to avoid permanent changes.
# sudo bless --device disk1s3 --setBoot
Booting
Booting on ESX
This is only required if you prefer not to bless the device. If you bless it, it will boot automatically. However, if you skip the blessing part you can easily chose which HDD to boot by using the EFI Boot Shell.
Booting on ESX, hit F2 to get the EFI menu up. One of the options is the "EFI Shell (not supported)". Once the EFI shell has been entered
# map
will list all available maps. For me,
fs0 disk0 EFI partition fs1 disk0 OSX boot fs2 disk0 OSX Recovery fs3 disk1 EFI fs4 disk1 boot HFS fs5 disk1 recovery fs6 disk1 ZFS
So in my case
EFI> fs4: fs4: ls usr System Library fs4: cd System fs4: cd Library fs4: cd CoreServices fs4: boot.efi
Which runs (boots) boot.efi, which then locates and loads the prelinkedkernel. You can specify boot options here too:
fs4: boot.efi zfs_boot=rpool -s -v
To ask it to boot from pool rpool (using bootfs= property) into single-user mode, with verbose output.
Booting on VirtualBox
SIP can be disabled in VirtualBox, but must be disabled before every boot. One way to do this is to install rEFInd into the EFI partition. It is possible to write a startup.nsh script to do so.
VirtualBox does not like the boot helper as created above. It used to function in a previous version (4.3.x?) but at this time, the easiest option I have found is to clone the contents of BaseSystem.dmg from any MacOS recovery partition.
# diskutil mount disk0s3 # hdiutil attach -readonly "/Volumes/Recovery HD/com.apple.recovery.boot/BaseSystem.dmg"
Then
# sudo rsync -axH "/Volumes/OS X Base System/" "/Volumes/boot/"
Or if you prefer asr:
# sudo asr restore -source "/Volumes/OS X Recovery" -target "/Volumes/boot" -erase # diskutil rename "/Volumes/OS X Base System 1" "boot"
And the paths used above will change, as we are using a “real” disk layout with no com.apple.boot.R:
# sudo rsync -a /Volumes/Capitan/System/Library/PrelinkedKernels/prelinkedkernel \ /Volumes/boot/System/Library/PrelinkedKernels/prelinkedkernel # sudo vim /Volumes/boot/Library/Preferences/SystemConfiguration/com.apple.Boot.plist
(See Boot.plist)
Booting on real Macs
Always make extra backups! Test this out using flash drives first!
Create partitions as above. If you already created a boot helper on one disk, you can clone it with Disk Utility or asr and edit the [com.apple.Boot.plist] to put a different pool name in zfs_boot in the Kernel flags.
Hold option while powering on or restarting your Mac.
Look for ZFS Boot as set by bless --label above, and double click.
ZFS will load and search available disks for the pool specified in zfs_boot above. If the pool is not found, or not enough disks are present, ZFS will wait and check additional disks as they appear.
If all is well, the pool will import and MacOS will boot from the Capitan dataset.
After a certain number of attempts, the MacOS loader will show a 'prohibitory symbol' (circle with a line through it). If you see this, hold the power button until your Mac powers off.
If you are stuck in a boot loop, power the machine off, then power on while holding the option key. Select the partition with standard MacOS to reboot normally. Once booted, check and set System Preferences->Startup Disk.
Additional
Mounting other boot environments
# sudo zfs snapshot rpool/ROOT/Capitan1@updating # sudo zfs clone rpool/ROOT/Capitan1@updating rpool/ROOT/Capitan2 # mkdir /Volumes/Capitan2 # sudo mount_zfs rpool/ROOT/Capitan2 /Volumes/Capitan2
Later you can promote
# sudo zfs promote rpool/ROOT/Capitan2
Then if you want
# sudo zfs destroy -r rpool/ROOT/Capitan
Blessing the boot device
Need only be run once, to setup the boot volume:
# sudo bless --folder /Volumes/boot/System/Library/CoreServices \ --file /Volumes/boot/System/Library/CoreServices/boot.efi \ --bootefi /Volumes/Capitan/System/Library/CoreServices/boot.efi
This command actually sets the boot helper to be used on reboot:
# sudo bless --device disk1s3 --setBoot