ZFS on Boot

From OpenZFS on OS X
Revision as of 05:23, 18 June 2016 by Lundman (Talk | contribs)

Jump to: navigation, search

ZFS as Boot

Status: Not Ready


Disk Layout

Play disk: disk1


Created the following partition on the test boot disk:

Number  Start (sector)    End (sector)  Size       Code  Name
   1            2048          411647   200.0 MiB   EF00  EFI System
   2          411648         2508799   1024.0 MiB  AF00  Apple HFS/HFS+
   3         2508800         3737599   600.0 MiB   AB00  Recovery HD
   4         3737600        83886046   38.2 GiB    BF01  Solaris /usr & Mac ZFS
s1 is not really used, but added for EFI.
s2 is the small HFS boot, which will carry the kernel (and the kext caches) to load the kernel with ZFS. This then imports and mounts the ZFS slice.
s3 is currently not used, but created in case we want it in future
s4 is the ZFS pool used for booting.

Setting up boot slice s2

Regular HFS filesystem, and populated thusly; (Volume named boot here) In this case, we have both the kernel and kernel.development. But normal users will only have kernel. These are rsynced from the (normal) booted OS.

# diskutil mount disk1s2
# mdutil -i off /Volumes/boot
# cd /Volumes boot
./usr/standalone/i386/boot.efi
./System/Library/Kernels/kernel
./System/Library/Kernels/kernel.development
./System/Library/PrelinkedKernels/prelinkedkernel
./System/Library/PrelinkedKernels/prelinkedkernel.development
./System/Library/Caches/com.apple.kext.caches/Startup/IOKitPersonalities_x86_64.ioplist.gz
./System/Library/Caches/com.apple.kext.caches/Startup/kernelcache
./System/Library/Caches/com.apple.kext.caches/Startup/kernelcache.development
./System/Library/Caches/com.apple.kext.caches/Startup/KextPropertyValues_OSBundleHelper_x86_64.plist.gz
./System/Library/Caches/com.apple.kext.caches/Startup/KextPropertyValues_PGO_x86_64.plist.gz
./System/Library/CoreServices/boot.efi
./System/Library/CoreServices/SystemVersion.plist
# cat SystemVersion.plist
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
       <key>ProductBuildVersion</key>
       <string>15F34</string>
       <key>ProductCopyright</key>
       <string>1983-2016 Apple Inc.</string>
       <key>ProductName</key>
       <string>Mac OS X</string>
       <key>ProductUserVisibleVersion</key>
       <string>10.11.5</string>
       <key>ProductVersion</key>
       <string>10.11.5</string>
</dict>
</plist>

Now we install the ZFS kexts and rebuild the kernel caches

# mkdir -p /Volumes/boo/Library/Extensions
# cd your-source-directory-for-zfs
# cd spl/module/spl
# make install DESTDIR=/Volumes/boot/
# cd ../../../zfs/module/zfs
# make install DESTDIR=/Volumes/boot/

Technically, that will also install some extra stuff which isn't needed into the boot slice but that fine.

Generate caches for kernel

kextcache -arch x86_64 -local-root -all-loaded -kernel /Volumes/boot/System/Library/Kernels/kernel -prelinked-kernel /Volumes/boot/System/Library/PrelinkedKernels/prelinkedkernel -mkext /Volumes/boot/System/Library/Caches/com.apple.kext.caches/Startup/kernelcache -volume-root / /System/Library/Extensions /Library/Extensions /Volumes/boot/Library/Extensions/

And if needed, caches for kernel.development

kextcache -arch x86_64 -local-root -all-loaded -kernel /Volumes/boot/System/Library/Kernels/kernel.development -prelinked-kernel /Volumes/boot/System/Library/PrelinkedKernels/prelinkedkernel.development -mkext /Volumes/boot/System/Library/Caches/com.apple.kext.caches/Startup/kernelcache.development -volume-root / /System/Library/Extensions /Library/Extensions /Volumes/boot/Library/Extensions/

Here there will be steps for blessing the boot, and setting up EFI etc. But we will use EFI Shell to boot for now.


Populating the ZFS Boot

Create the pool and datasets

# zpool create -f -o ashift=12 -O casesensitivity=insensitive -O normalization=formD -O atime=off -O compression=lz4 rpool disk1s4
# zfs create rpool/ROOT
# zfs create rpool/ROOT/10.11

Create ROOT and lower just like IllumOS so that you can easy swap OS snapshots and reboot into something else

Copy over the OS as you see fit, rsync will generally do. Remember that etc, tmp and var are symlinks into /private

# rsync -ar --exclude="/Volumes/rpool/*" / /Volumes/rpool/ROOT/10.11/

Install ZFS

# cd /Volumes/rpool/ROOT/10.11/
# mkdir -p usr/local
# cd usr/local
# ln -s ../../sbin sbin

# cd $your-zfs-source-area
# cd spl
# make install DESTDIR=/Volumes/rpool/ROOT/10.11/
# cd ../zfs
# make install DESTDIR=/Volumes/rpool/ROOT/10.11/
# cd /Volumes/rpool/ROOT/10.11/
# cd sbin
# mv fsck fsck_real
# cp ../usr/bin/true fsck

When ready, the bootfs needs to be changed to legacy

# zfs set mountpoint=legacy rpool/ROOT/10.11

If you want to change it in future, use inherit to mount it back.

# zfs inherit mountpoint rpool/ROOT/10.11

Once done, remember to change it back to legacy

Booting

Booting on ESX, I hit F2 to get the EFI menu up. One of the options is the "EFI Shell (not supported)". Once the EFI shell has been entered

# map

will list all available maps. For me,

fs0 disk0 EFI partition
fs1 disk0 OSX boot
fs2 disk0 OSX Recovery
fs3 disk1 EFI
fs4 disk1 boot HFS
fs5 disk1 recovery
fs6 disk1 ZFS

So in my case

EFI> fs4:
fs4: ls
usr System Library
fs4: cd System 
fs4: cd Library
fs4: cd CoreServices
fs4: boot.efi rd=disk1s4 -s

Which runs (boots) boot.efi, using rootdev (rd=) as disk1s4 (ZFS Boot). We use -s to go into single user, needed for now.

SingleUser Mode Entered
Issue
fsck
mount -uw /
then exit to boot

We need to remount "/" to be read-write.

/sbin/mount_zfs -u /dev/disk1s4 /
exit

Boot resumes.