Auto import only one pool, as quickly as possible

[mkush]

Executive Summary:

I want to auto-import ONE pool consisting of a pair of internal NVMe drives as quickly as possible, since it will contain my home directory, and auto-import nothing else.

More Detail:

Lundman's comment to another thread of mine made me go and look at zpool-import-all.sh for the first time. I've decided that I actually don't want ALL my pools auto-importing, due to the snafu that seems to have borked my backup pool, BUT, I actually DO want one pool to auto-import, and I want to do it as quickly as possible. Some (most?) of this may be based on some faulty assumptions so please tell me if so.

Basically the scenario is that I have a 2019 Mac Pro and it has, internally, a Sonnet board with two NVMe drives that are configured as a ZFS mirror. I would like to store my home folder on that device.

I am ASSUMING that the pool needs to be ready really quickly, because if I try to log on before it is ready, bad stuff will happen. (Correct me if I'm wrong.)

I am ASSUMING that this pool should be ready to import REALLY quickly since these are NVMe drives plugged straight into a PCIe slot inside the machine. Nothing to spin up, etc. (Correct me if I'm wrong.)

Many other disks are also connected to this Mac, externally, via Thunderbolt. These include 8 HDDs and 16 SSDs. I am ASSUMING that all that may take a while to become available, and thus may delay the appearance of the invariant.idle file. (Correct me if I'm wrong.)

Based on the above ASSUMPTIONS, what I would like to do is modify the script so that it no longer waits for the invariant.idle to become available and just imports the ONE pool I care about without further ado. The point of my post is to see what holes there are in my thinking and if there's a better way to accomplish what I want, namely importing ONE mirror pool consisting of a pair of internal NVMe drives as quickly as possible, and importing nothing else.

Specifically:
1. I plan to remove everything from the file except as noted below.
2. What about the /bin/sync line? Remove or modify, and if so, how?
3. Any problem with inserting /usr/sbin/sysctl -w kstat.zfs.darwin.tunable.zfs_arc_max=1000000000 since the conf file isn't getting read properly?
4. I plan to make the import line: "${ZPOOL}" import -d /var/run/disk/by-serial Data

[mkush]

I've been playing with this on and off today but do not have a 100% working solution yet. What I tried to do was, instead of waiting for invariant.idle to appear, was to wait for the symlinks for both my NVMe disks to appear in /var/run/disk/by-serial (or by-id, I've tried both), then once they do, do the import.

It works most of the time but not always. In other words, I reboot, log on, and check for the presence of my mounted pool. Most times it's there, sometimes it's not.

Here's my question: is it possible that those symlinks exist in /var/run/disk/by-x from the PREVIOUS boot still, or by the time this script runs, are they guaranteed to have been freshly generated by InvariantDisks?

If they've been freshly generated and are therefore good data, why would my import ever fail?

I'm considering just putting a loop in the script to check for the presence of my mounted pool in /volumes, repeatedly trying the import until either it works or some timeout is reached.

Probably I'm missing something obvious.

[nodarkthings]

I can only tell you that you're not alone: sometimes, pretty rarely though, one of my pools won't automount (even with my zpool-import-all.sh modified with the -f flag), who knows why... It happens more often when coming back from another boot partition. Then I manually run zpool-import-all.sh and it works... Perhaps I should try to increase the timeout.
As for putting your home folder in a pool, I think the best way is to disable autologin in OSX — I've not done it myself but I believe it leaves time for the pools to mount, then you login manually.

[mkush]

I’ve come to the conclusion that even though I can modify the script to retry mounting the pool until it mounts, I can’t guarantee that it will be there by the time I log on. I was looking for a way to not allow logon until the pool imported but I don’t see an obvious way.

Although I could just wait after booting the computer before logging on, even that’s not a guarantee that the pool is truly imported. So the only “solution” I can think of is to create another user whose account is stored on the boot drive (not ZFS) which is set to auto log on. I can then verify that the pool is imported before logging on to my real account. Clunky but guaranteed to work.

STILL, I’d like to know the answers to the questions I was asking about the auto-import script. Especially whether the presence of the by-serial links means that they have been freshly generated or if they could be from a previous boot and therefore stale.

[mkush]

Sorry for all the posts. I guess I should experiment more before posting. So, I've answered my own question about the entries in /var/run/disk. They are generated each time because /var/run is empty at each boot of MacOS. Therefore, I would conclude that the presence of an entry for each disk in a pool means that that pool can safely be imported. Please correct me if I'm wrong.

[lundman]

Can you " NVMe disks to appear in /var/run/disk/by-serial " AND make sure their time-stamp isn't "too old" ? I do believe they can exist from previous boot if InvD wasn't shut down as expected

[soongrazer]

Hello, I had used ZFS for my home directory long before. I solved the early log in issue by disabling the users whose home directories are on a ZFS volume. Frequently checking if the ZFS volume is mounted and enable the users after that.

I used pwpolicy to enable/disable the users. If you try to log in while the user is disabled, the GUI responds as if incorrect credentials were entered. I think I used a dedicated user account, which is always disabled for logins, just to unlock FileVault on boot. After that, I can just log in to my normal user account.

I think this will help you to some extent. Please don't hesitate to ask me if you have any questions.