Difference between revisions of "Install"

From OpenZFS on OS X
Jump to: navigation, search
(Initial installation from source)
(Installing from source)
(28 intermediate revisions by 5 users not shown)
Line 8: Line 8:
  
 
  $ md5 OpenZFS_on_OS_X_*.dmg
 
  $ md5 OpenZFS_on_OS_X_*.dmg
  $ sha1sum OpenZFS_on_OS_X_*.dmg
+
  $ shasum OpenZFS_on_OS_X_*.dmg
  $ openssl dgst -sha256 OpenZFS_on_OS_X_*.dmg
+
  $ shasum -a 256 OpenZFS_on_OS_X_*.dmg
  
 
Open the .dmg file.
 
Open the .dmg file.
Line 23: Line 23:
 
== Installing from source ==
 
== Installing from source ==
 
===Initial installation from source===
 
===Initial installation from source===
(Adapted from an [http://zerobsd.tumblr.com/post/62586498252/os-x-with-zfs article by ZeroBSD].)
 
  
Before doing anything else, please include <code>keepsyms=y</code> in your boot-args, and, if you are on OS X Yosemite, <code>kext-dev-mode=1</code>. For instance,
+
Before doing anything else, please set your boot-args.
 +
 
 +
On macOS Sierra (10.12), OS X El Capitan (10.11), OS X Mavericks (10.9), and OS X Mountain Lion (10.8) :
 
<syntaxhighlight lang="bash">
 
<syntaxhighlight lang="bash">
sudo nvram boot-args="-v keepsyms=y kext-dev-mode=1"
+
sudo nvram boot-args="-v keepsyms=1"
 
</syntaxhighlight>
 
</syntaxhighlight>
This will take effect the next time you reboot. The argument <code>-v</code> will make your boot screen verbose instead of just showing the Apple logo, the argument <code>keepsyms=y</code> will make your panic reports more useful for us, and the argument <code>kext-dev-mode=1</code> will allow you to autoload unsigned kernel extensions from /System/Library/Extensions on OS X Yosemite.
 
  
To compile on OSX El Capitan, there are still a few tidy-ups required. In the meantime, for Beta 1-3? Enable rootless mode.
+
On OS X Yosemite (10.10):
 
<syntaxhighlight lang="bash">
 
<syntaxhighlight lang="bash">
sudo nvram boot-args="-v keepsyms=y kext-dev-mode=1 rootless=1"
+
sudo nvram boot-args="-v keepsyms=1 kext-dev-mode=1"
 
</syntaxhighlight>
 
</syntaxhighlight>
For the remaining betas, it is necessary to boot into recovery mode, and using the GUI, disable SIP mode.
 
  
For the GM release, it is necessary to boot into recovery mode, and using a shell:
+
 
 +
This will take effect the next time you reboot. The argument <code>-v</code> will make your boot screen verbose instead of just showing the Apple logo, the argument <code>keepsyms=1</code> will make your panic reports more useful for us, and the argument <code>kext-dev-mode=1</code> (OS X Yosemite ONLY) will allow you to load your  custom-built, unsigned kexts kernel extensions. Note that as of OS X El Capitan, kext-dev-mode is obsolete and and does nothing. If you are on macOS Sierra (10.12) or OS X El Capitan, you should remove kext-dev-mode from your boot-args to avoid confusion.
 +
 
 +
If you are a developer, you may want <code>debug=0x144</code> in your boot-args as well, as explained here: [[Development]]
 +
 
 +
 
 +
For macOS Sierra (10.12) and OS X El Capitan (10.11), you also need to boot into the Recovery OS (either the Recovery partition or bootable OS X install media for 10.11) and run
 
<syntaxhighlight lang="bash">
 
<syntaxhighlight lang="bash">
 
csrutil disable
 
csrutil disable
 
</syntaxhighlight>
 
</syntaxhighlight>
 
+
or
We need to change the location that a few things are installed in to be truly compatible with El Capitan, this will be done soon.
+
<syntaxhighlight lang="bash">
 +
csrutil enable --without kext
 +
</syntaxhighlight>
 +
The latter will allow you to load your custom-built, unsigned kernel extensions, while keeping the rest of System Integrity Protection (SIP) in place.
 +
(Depending on your needs, other options you may want are <code>--without fs</code>, <code>--without debug</code>, <code>--without dtrace</code>, or <code>--without nvram</code>. If you specify all of them, that is equivalent to <code>csrutil disable</code>. The <code>--without</code> option takes a comma-delimited list or can be specified multiple times: <code>--without kext,debug,dtrace</code> is equivalent to <code>--without kext --without debug --without dtrace</code>).
  
 
If you have any other implementation of ZFS installed, you must uninstall it and reboot before proceeding further. Similarly, if you have installed the O3X installer version, please follow the [https://openzfsonosx.org/wiki/Uninstall uninstallation directions] before proceeding.
 
If you have any other implementation of ZFS installed, you must uninstall it and reboot before proceeding further. Similarly, if you have installed the O3X installer version, please follow the [https://openzfsonosx.org/wiki/Uninstall uninstallation directions] before proceeding.
Line 50: Line 59:
 
* [https://developer.apple.com/xcode/ Xcode] (from [http://itunes.apple.com/us/app/xcode/id497799835?ls=1&mt=12 Mac App Store] or https://developer.apple.com/downloads/index.action)
 
* [https://developer.apple.com/xcode/ Xcode] (from [http://itunes.apple.com/us/app/xcode/id497799835?ls=1&mt=12 Mac App Store] or https://developer.apple.com/downloads/index.action)
 
* Xcode Command Line Tools (https://developer.apple.com/downloads/index.action)
 
* Xcode Command Line Tools (https://developer.apple.com/downloads/index.action)
* [http://brew.sh/ Homebrew] (or [http://www.macports.org/MacPorts])
+
* [http://brew.sh/ Homebrew] (or [http://www.macports.org/ MacPorts])
  
Note: Xcode 6.3 or later is now required to build due to Broadwell CPU support requirements.
+
Note: Xcode 6.3 or later is now required to build due to Broadwell CPU support requirements. Additionally, as of XCode 8, if you are on 10.11 specifically (unnecessary on 10.12), you need to install the CLT (Command Line Tools):
  
Note: Xcode 7.0 (7A220) is (accidentally) missing the 10.10 SDK. You may unzip https://www.dropbox.com/s/hggp0zgn7hkz38m/MacOSX10.10.sdk.zip in /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/ as a workaround.
+
<syntaxhighlight lang="bash">
 +
xcode-select --install
 +
</syntaxhighlight>
  
 
To install Homebrew:
 
To install Homebrew:
Line 62: Line 73:
 
</syntaxhighlight>
 
</syntaxhighlight>
  
Paste that at a Terminal prompt.
+
If you already have Homebrew installed, you should
 
+
Once Homebrew is installed, we need a couple of things first:
+
 
+
 
<syntaxhighlight lang="text">
 
<syntaxhighlight lang="text">
brew install automake libtool gawk coreutils
+
brew update && brew upgrade
 
</syntaxhighlight>
 
</syntaxhighlight>
  
If you have used brew before, it might pay to check it
+
Once Homebrew is installed or updated, you should check the output of
 
+
 
<syntaxhighlight lang="text">
 
<syntaxhighlight lang="text">
 
brew doctor
 
brew doctor
 
</syntaxhighlight>
 
</syntaxhighlight>
 +
and address any issues it identifies that you know are not false positives.
  
and if prompted
+
If you have used brew before, you should also run
 
+
 
<syntaxhighlight lang="text">
 
<syntaxhighlight lang="text">
 
brew prune
 
brew prune
 
</syntaxhighlight>
 
</syntaxhighlight>
  
Create two folders in your home directory.
+
Once your system is ready to brew, you should install the following:
 +
<syntaxhighlight lang="text">
 +
brew install automake libtool gawk coreutils
 +
</syntaxhighlight>
 +
(Note that autoconf will automatically be installed as well because it is a dependency of automake.)
 +
 
 +
Now that the needed tools are installed, it's time to set up your developer work space. Create two folders in your home directory.
  
 
<syntaxhighlight lang="bash">
 
<syntaxhighlight lang="bash">
Line 90: Line 103:
 
</syntaxhighlight>
 
</syntaxhighlight>
  
Adjust your PATH variable:
+
Now adjust your PATH variable, unless you know it's already set appropriately.
 +
 
 +
If you're on OS X Mavericks or above (>= 10.9), run this:
 +
 
 +
<syntaxhighlight lang="bash">
 +
echo 'export PATH=$HOME/bin:$PATH' >> ~/.bash_profile
 +
</syntaxhighlight>
  
 +
If you're on OS X Mountain Lion (10.8), you will want to run this instead:
 
<syntaxhighlight lang="bash">
 
<syntaxhighlight lang="bash">
 
echo 'export PATH=$HOME/bin:/usr/local/bin:$PATH' >> ~/.bash_profile
 
echo 'export PATH=$HOME/bin:/usr/local/bin:$PATH' >> ~/.bash_profile
 
</syntaxhighlight>
 
</syntaxhighlight>
  
and update your environment by sourcing your profile.
+
Then update your environment by sourcing your profile.
  
 
<syntaxhighlight lang="bash">
 
<syntaxhighlight lang="bash">
Line 210: Line 230:
 
sudo zpool export $poolname
 
sudo zpool export $poolname
 
</syntaxhighlight>
 
</syntaxhighlight>
 +
 +
Alternatively, you can run:
 +
<syntaxhighlight lang="bash">
 +
sudo zpool export -a
 +
</syntaxhighlight>
 +
which will try to export all pools for you.
  
 
Make sure they have exported successfully.
 
Make sure they have exported successfully.
Line 218: Line 244:
  
 
It should say, "no pools available."
 
It should say, "no pools available."
 
Alternatively, you can use this script to export all pools:
 
<syntaxhighlight lang="bash">
 
#!/bin/bash
 
 
if [ x$(which zpool) != x ]
 
then
 
zpool list -H | awk -F '\t' '{print $1;}' | xargs -I '#' sudo zpool export '#'
 
else
 
cd ~/Developer/zfs
 
./cmd.sh zpool list -H | awk -F '\t' '{print $1;}' | xargs -I '#' sudo ./cmd.sh zpool export '#'
 
fi
 
</syntaxhighlight>
 
  
 
Get any [[zfsadm]] updates:
 
Get any [[zfsadm]] updates:
Line 256: Line 269:
  
 
# Assuming the build completed successfully,
 
# Assuming the build completed successfully,
# unload the kexts. If you did not export all of
+
# unload the kexts.
# your pools this will panic:
+
  
 
zfsadm -u
 
zfsadm -u
Line 276: Line 288:
 
# If not, make sure kextd is aware of them
 
# If not, make sure kextd is aware of them
  
sudo touch /System/Library/Extensions
+
sudo touch /Library/Extensions /System/Library/Extensions
 
sudo killall -HUP kextd
 
sudo killall -HUP kextd
  
Line 289: Line 301:
 
sudo kextload -d spl.kext zfs.kext
 
sudo kextload -d spl.kext zfs.kext
 
</syntaxhighlight>
 
</syntaxhighlight>
 
If net.lundman.kernel.dependencies has been updated, which is rarely the case, a reboot will be necessary.
 
  
 
===Uninstalling a source install===
 
===Uninstalling a source install===
Line 316: Line 326:
  
 
cd ~/Developer
 
cd ~/Developer
sudo cp -r zfs/module/zfs/zfs.kext /tmp/o3x/  
+
sudo cp -R zfs/module/zfs/zfs.kext /tmp/o3x/  
sudo cp -r spl/module/spl/spl.kext /tmp/o3x/
+
sudo cp -R spl/module/spl/spl.kext /tmp/o3x/
  
 
cd /tmp/o3x
 
cd /tmp/o3x

Revision as of 15:10, 10 February 2017

Installing the official release

Download the most recent dmg from the Downloads page.

Verify the checksums.

$ md5 OpenZFS_on_OS_X_*.dmg
$ shasum OpenZFS_on_OS_X_*.dmg
$ shasum -a 256 OpenZFS_on_OS_X_*.dmg

Open the .dmg file.

Read ReadMe.rtf.

Start the installer by opening OpenZFS_on_OS_X_x.y.z.pkg.

Follow the prompts.

If you ever want to uninstall, follow the instructions for uninstalling a release version.

Installing from source

Initial installation from source

Before doing anything else, please set your boot-args.

On macOS Sierra (10.12), OS X El Capitan (10.11), OS X Mavericks (10.9), and OS X Mountain Lion (10.8) :

sudo nvram boot-args="-v keepsyms=1"

On OS X Yosemite (10.10):

sudo nvram boot-args="-v keepsyms=1 kext-dev-mode=1"


This will take effect the next time you reboot. The argument -v will make your boot screen verbose instead of just showing the Apple logo, the argument keepsyms=1 will make your panic reports more useful for us, and the argument kext-dev-mode=1 (OS X Yosemite ONLY) will allow you to load your custom-built, unsigned kexts kernel extensions. Note that as of OS X El Capitan, kext-dev-mode is obsolete and and does nothing. If you are on macOS Sierra (10.12) or OS X El Capitan, you should remove kext-dev-mode from your boot-args to avoid confusion.

If you are a developer, you may want debug=0x144 in your boot-args as well, as explained here: Development


For macOS Sierra (10.12) and OS X El Capitan (10.11), you also need to boot into the Recovery OS (either the Recovery partition or bootable OS X install media for 10.11) and run

csrutil disable

or

csrutil enable --without kext

The latter will allow you to load your custom-built, unsigned kernel extensions, while keeping the rest of System Integrity Protection (SIP) in place. (Depending on your needs, other options you may want are --without fs, --without debug, --without dtrace, or --without nvram. If you specify all of them, that is equivalent to csrutil disable. The --without option takes a comma-delimited list or can be specified multiple times: --without kext,debug,dtrace is equivalent to --without kext --without debug --without dtrace).

If you have any other implementation of ZFS installed, you must uninstall it and reboot before proceeding further. Similarly, if you have installed the O3X installer version, please follow the uninstallation directions before proceeding.

Prerequisites:

Note: Xcode 6.3 or later is now required to build due to Broadwell CPU support requirements. Additionally, as of XCode 8, if you are on 10.11 specifically (unnecessary on 10.12), you need to install the CLT (Command Line Tools):

xcode-select --install

To install Homebrew:

ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"

If you already have Homebrew installed, you should

brew update && brew upgrade

Once Homebrew is installed or updated, you should check the output of

brew doctor

and address any issues it identifies that you know are not false positives.

If you have used brew before, you should also run

brew prune

Once your system is ready to brew, you should install the following:

brew install automake libtool gawk coreutils

(Note that autoconf will automatically be installed as well because it is a dependency of automake.)

Now that the needed tools are installed, it's time to set up your developer work space. Create two folders in your home directory.

mkdir -p ~/Developer ~/bin
chmod 700 ~/Developer ~/bin
chmod +a "group:everyone deny delete" ~/Developer ~/bin

Now adjust your PATH variable, unless you know it's already set appropriately.

If you're on OS X Mavericks or above (>= 10.9), run this:

echo 'export PATH=$HOME/bin:$PATH' >> ~/.bash_profile

If you're on OS X Mountain Lion (10.8), you will want to run this instead:

echo 'export PATH=$HOME/bin:/usr/local/bin:$PATH' >> ~/.bash_profile

Then update your environment by sourcing your profile.

source ~/.bash_profile

To acquire the sources and build ZFS, we can use the zfsadm script found here.

cd ~/Developer/
git clone https://gist.github.com/7713854.git zfsadm-repo
cp zfsadm-repo/zfsadm ~/bin/

Now you can can build OpenZFS on OS X:

zfsadm

This will take a few minutes, depending on your hardware. There may be some warnings during the compilation. Do not worry about it unless you see errors.

Before using ZFS, we need to actually install it. If you are a developer and wish not to install ZFS, but rather run it from the compile directory, skip ahead.

cd ~/Developer/spl
sudo make install
cd ~/Developer/zfs
sudo make install

You can check to see if the kernel extensions loaded automatically with

kextstat | grep lundman

You should see something similar to

137    1 0xffffff803f61a800 0x20c      0x20c      net.lundman.kernel.dependencies (10.0.0)
144    1 0xffffff7f82720000 0xd000     0xd000     net.lundman.spl (1.0.0) <137 7 5 4 3 1>
145    0 0xffffff7f8272d000 0x202000   0x202000   net.lundman.zfs (1.0.0) <144 13 7 5 4 3 1>

If not, make sure kextd is aware of them.

sudo touch /System/Library/Extensions
sudo killall -HUP kextd

Now check again.

kextstat | grep lundman

If not, you can load the kexts manually.

cd /System/Library/Extensions
sudo kextload spl.kext
sudo kextload -d spl.kext zfs.kext

Try running

zpool

to see if everything is installed and configured properly.

You can go ahead and create your pools at this point.

Running ZFS from the source build

In the ZFS directory there is a script called load.sh which will load the two kext modules. To run userland binaries from the source tree, use the provided cmd.sh script to set the DYNDL_LIBRARY_PATH variable correctly to find the libraries.

Note that load.sh will also start tail -f /var/log/system.log for your convenience.

$ sudo bash# cd ~/Developer/zfs# ./load.shkernel[0]: SPL: Loaded module v1.3.1-15_g4e2ff66 (DEBUG mode), (ncpu 4, memsize 4294967296, pages 1048576)
kernel[0]: ZFS: Loaded module v1.3.1-230_gb9658da, ZFS pool version 5000, ZFS filesystem version 5
# ./cmd.sh zpool import BOOM# ./cmd.sh zpool listNAME   SIZE  ALLOC   FREE  EXPANDSZ   FRAG    CAP  DEDUP  HEALTH  ALTROOT
BOOM  39.8G   207M  39.5G         -     1%     0%  1.00x  ONLINE  -
# ./cmd.sh zpool export BOOMUnmount successful for /Volumes/BOOM
# kextunload -b net.lundman.zfskernel[0]: ZFS: Unloaded module v1.3.1-230_gb9658da
# kextunload -b net.lundman.splkernel[0]: SPL: Released 524288 bytes from vmem_seg_arena
kernel[0]: SPL: Unloaded module. (os_mem_alloc: 0)

Upgrading a source install

When you want to get the latest commits from the GitHub, here's a quick overview of things you need to run.

First make sure you have exported all of your pools.

zpool list

For every pool listed, run

sudo zpool export $poolname

Alternatively, you can run:

sudo zpool export -a

which will try to export all pools for you.

Make sure they have exported successfully.

zpool status

It should say, "no pools available."

Get any zfsadm updates:

cd ~/Developer
[ -d zfsadm-repo/.git ] && (cd zfsadm-repo ; git pull)
[ ! -d zfsadm-repo/.git ] &&  git clone https://gist.github.com/7713854.git zfsadm-repo
cp zfsadm-repo/zfsadm ~/bin/

Now you should be able to upgrade your ZFS installation.

cd ~/Developer
 
cd spl
make clean
cd ..
 
cd zfs
make clean
cd ..
 
zfsadm
 
# Assuming the build completed successfully,
# unload the kexts.
 
zfsadm -u
 
# Now install the upgrade.
 
cd spl
sudo make install
cd ..
 
cd zfs
sudo make install
 
# And verify they reloaded automatically
 
kextstat | grep lundman
 
# If not, make sure kextd is aware of them
 
sudo touch /Library/Extensions /System/Library/Extensions
sudo killall -HUP kextd
 
# and check again
 
kextstat | grep lundman
 
# if they they still have not loaded automatically
 
cd /System/Library/Extensions
sudo kextload spl.kext
sudo kextload -d spl.kext zfs.kext

Uninstalling a source install

If you ever want to uninstall, follow the instructions for uninstalling a source install.

Using without actually installing (development)

This method is usually appropriate only for Developers.

The procedure is the same as found in the section installing from source except that you never run "make install." Instead you load the kexts manually, and execute the binaries directly from the source tree.

You can load the kexts manually by running

zfsadm -k

By default, zfsadm -k will copy spl.kext and zfs.kext from the source where they were built to /System/Library/Extensions, recursively change the ownership of everything in /System/Library/Extensions/spl.kext and /System/Library/Extensions/zfs.kext to be owned by the user "root" and the group "wheel," and then load the kexts directly from /System/Library/Extensions. If you prefer to use a different directory, use the -i option in zfsadm or edit zfsadm to hard code a different directory.

If you do not wish to use zfsadm, you can do all of this yourself, using whatever target directory you'd like. For example, you might do the following:

cd /tmp
sudo rm -rf o3x
sudo mkdir o3x
 
cd ~/Developer
sudo cp -R zfs/module/zfs/zfs.kext /tmp/o3x/ 
sudo cp -R spl/module/spl/spl.kext /tmp/o3x/
 
cd /tmp/o3x
sudo chown -R *
sudo kextload spl.kext
sudo kextload -d spl.kext zfs.kext

Once the kexts have been loaded, you can test the commands.

~/Developer/zfs/cmd.sh zfs

Migrating old pools (from MacZFS or ZEVO)

First export all of your pools, and uninstall the other implementation. It is all right if you forgot to export your pools before uninstalling. You will just need to use the '-f' option when importing into OpenZFS on OS X.

To find out the pool names, you need to execute the command for pool discovery.

sudo zpool import

This will tell you what pools are available to be imported, but will not actually import anything. You can see that nothing has been imported yet by using the 'zpool status' command.

zpool status

Now that you know what pools are available to be imported, you can actually import a pool by supplying the name or guid that you saw during pool discovery.

sudo zpool import poolname (or guid)

(Notice how this differs from the command for pool discovery.)

If you forgot to export before migrating, you will need to use the '-f' option.

sudo zpool import -f poolname (or guid)

If you want to see the same information you saw during pool discovery, you will now need to use 'zpool status' rather than 'zpool import'.

zpool status

If all pools have been imported, the pool discovery command— 'zpool import' with no pool or guid specified— will return without any output.

sudo zpool import