OpenZFS-2.0-release report

[theit]

Hmm? Are we saying the pkg isn't notarized? That seems odd, other Cataliina users have installed it..

This seems to be the case, at least on my side...

[abcslayer]

I could not boot macOS Catalina 10.15.7 APFS on ZVOL with OpenZFS 2.0 (latest release by the moment of writing). If I replace the kext by 1.9.4 ones, it can boot and run normally.
The system boots via a HFS+ booter partition. Mac ZFS 1.9.4 has been using from 10.14.x (manually installed) -> 10.15.4 (manually upgraded) -> 10.15.5 -> 10.15.7 (confirmed OK).
If we boot from HFS+ partition (i.e boot from an Installer partition), the 10.15.7 system (minimal macOS system) can boot fine and load the zpool.
For the error details, I have started a Github issue here: https://github.com/openzfsonosx/openzfs/issues/14
I dont know why it happened. Hope it can give you some clues. Anything I can test or check, please tell, I will try to do. At this moment, I have to fallback to 1.9.4
P/S: with OpenZFS 2.0, there is a reading error when the pool is being loaded while pool scub in 1.9.4 give you no error, quite strange.

[lundman]

Any major issues found yet?
* big sur needs keepsym=1
* if you set mimic=apfs, you can't export. Seems it holds some lock, but
mimic=apfs isn't really ready yet, just there for mimic=hfs and future proofing.

[Marco]

Hmm? Are we saying the pkg isn't notarized? That seems odd, other Cataliina users have installed it..

This seems to be the case, at least on my side...

I have just installed Big Sur and then I have tried to update openZFS to version 2. I had the same Finder message when I tried to open the installer by double-clicking on it.

I could override the message when I selected the installer, the right-clicked on it and selected "Open with". The option is then offered to installe even if Apple couldn't perform the checks.

[mike0810]

Only the root datasets are recognized as internal disks. See screenshots. I have two pools, one is a ssd mirrored pool and one a raidz. I created two datasets in the raidz pool. Both are not recognized as internal disks, only the root dataset. This happens with Big sur 11.1

Funny enough, see the screenshot of the iStats Menu. Here it is exactly the opposite, only the root datasets are marked as removable.

michael@sunshine ~ $ zpool status
pool: fast
state: ONLINE
status: Some supported features are not enabled on the pool. The pool can
still be used, but some features are unavailable.
action: Enable all features using 'zpool upgrade'. Once this is done,
the pool may no longer be accessible by software that does not support
the features. See zpool-features(5) for details.
scan: scrub repaired 0B in 00:08:18 with 0 errors on Mon Aug 31 16:13:47 2020
config:

NAME STATE READ WRITE CKSUM
fast ONLINE 0 0 0
mirror-0 ONLINE 0 0 0
media-779FA966-DC8B-9442-88D5-BA2DEABA01D9 ONLINE 0 0 0
media-155BC228-8D34-674A-A6BE-84DDAEA3F6E9 ONLINE 0 0 0
mirror-1 ONLINE 0 0 0
media-35AF8E29-3A9A-B94B-8F41-D960662903D4 ONLINE 0 0 0
media-9981533B-3DE6-F34C-ADD9-C0AEE4DB9673 ONLINE 0 0 0
mirror-2 ONLINE 0 0 0
media-FF6309A7-739D-9642-B94F-6ED53A49E6AB ONLINE 0 0 0
media-0093ECDE-EE0D-B24A-BABA-39E2798109EE ONLINE 0 0 0

errors: No known data errors

pool: tank
state: ONLINE
status: Some supported features are not enabled on the pool. The pool can
still be used, but some features are unavailable.
action: Enable all features using 'zpool upgrade'. Once this is done,
the pool may no longer be accessible by software that does not support
the features. See zpool-features(5) for details.
scan: scrub repaired 0B in 09:52:04 with 0 errors on Tue Sep 1 09:53:54 2020
config:

NAME STATE READ WRITE CKSUM
tank ONLINE 0 0 0
raidz1-0 ONLINE 0 0 0
media-9718BA76-5660-A943-A724-57DC201D33C2 ONLINE 0 0 0
media-31AA2471-FED1-BE47-9664-9128E199E2E5 ONLINE 0 0 0
media-6A6B631A-9277-7241-A95C-694EDB30B3A6 ONLINE 0 0 0
media-723DF783-7F1B-2540-9289-B4488ED4AEE1 ONLINE 0 0 0
logs
media-73C523F3-D705-4F4E-8605-5DFA33D055E7 ONLINE 0 0 0
cache
media-13E48244-C39D-4AE2-84CE-15AC34D06708 ONLINE 0 0 0

errors: No known data errors
michael@sunshine ~ $ zfs list
NAME USED AVAIL REFER MOUNTPOINT
fast 410G 264G 410G /Volumes/fast
tank 4.60T 513G 843K /Volumes/tank
tank/media 1.58T 513G 1.58T /Volumes/tank/media
tank/michael 3.02T 513G 3.02T /Volumes/tank/michael

[mike0810]

Here the screenshot of the disk utility, sorry, 3 attachment limit.

[FunMiles]

I think maybe "zfs recv" of a zvol will deadlock at the end when it tries to create the disk for it. Investigating.

I just tried to do a send/recv to keep a second backup disk in sync and got an error possibly related to this.

bigsur% sudo zfs send -I 14Nov20 quicksand/michel@15Dec20 | sudo zfs recv -F fastsand/michel
libzfs_macos_wrapfd: checking if we need pipe wrap
libzfs_macos_wrapfd: is pipe: work on fd 1
libzfs_macos_wrapfd: readfd 7 (0)
libzfs_macos_wrapfd: writefd 8 (0)
libzfs_macos_wrapfd: spawning thread
pipe_io_relay: thread up: read(7) write(1)
libzfs_macos_wrapfd: checking if we need pipe wrap
libzfs_macos_wrapfd: is pipe: work on fd 0
libzfs_macos_wrapfd: readfd 6 (0)
libzfs_macos_wrapfd: writefd 7 (0)
libzfs_macos_wrapfd: spawning thread
pipe_io_relay: thread up: read(0) write(7)
loop exit
pipe_io_relay: thread done: 30302987528 bytes
loop exit
pipe_io_relay: thread done: 30302987528 bytes
cannot receive: failed to read from stream
bigsur%

I double checked, the snapshot was not completely received and committed.

[lundman]

Trying to replicate the problems you guys are seeing, but not having much luck:

Code: Select all

zfs send -v BOOM@send | zfs recv FUN/new2
full send of BOOM@send estimated size is 676M
total estimated size is 676M
libzfs_macos_wrapfd: checking if we need pipe wrap
libzfs_macos_wrapfd: is pipe: work on fd 1
libzfs_macos_wrapfd: readfd 6 (0)
libzfs_macos_wrapfd: writefd 7 (0)
libzfs_macos_wrapfd: spawning thread
pipe_io_relay: thread up: read(6) write(1)
libzfs_macos_wrapfd: checking if we need pipe wrap
libzfs_macos_wrapfd: is pipe: work on fd 0
libzfs_macos_wrapfd: readfd 6 (0)
libzfs_macos_wrapfd: writefd 7 (0)
libzfs_macos_wrapfd: spawning thread
pipe_io_relay: thread up: read(0) write(7)
TIME        SENT   SNAPSHOT BOOM@send
16:22:27   23.8M   BOOM@send
16:22:28    119M   BOOM@send
16:22:29    123M   BOOM@send
16:22:30    213M   BOOM@send
16:22:31    338M   BOOM@send
16:22:32    444M   BOOM@send
16:22:33    468M   BOOM@send
16:22:34    520M   BOOM@send
16:22:35    601M   BOOM@send
16:22:36    656M   BOOM@send
do_mount calling mount with fstype zfs, rpath /Volumes/FUN/new2, fspec FUN/new2, mflag 0, optptr defaults,atime,strictatime,dev,exec,rw,setuid,nomand,zfsutil, optlen 60, devdisk 0, ispool 0
loop exit
root@lundmans-Mac-2(/Users/lundman) pipe_io_relay: thread done: 724495192 bytes
loop exit
pipe_io_relay: thread done: 724494880 bytes


The new zfs send pipe code only kicks in with pipes (ie "|") so you can try zfs send > file, zfs recv < file.

[FunMiles]

Trying to replicate the problems you guys are seeing, but not having much luck:
[ ... ]

The new zfs send pipe code only kicks in with pipes (ie "|") so you can try zfs send > file, zfs recv < file.

I will try the pipe send again after my next backup. I went around the problem by doing the operation on a Catalina machine with the older code.
If the problem persist on Big Sur, I'll try what you suggest and report.

[FunMiles]

I repeated my backup and duplication. I still got the error message, but it seems that the duplicate is correct. So the error message seems spurious. Maybe I made a mistake when I thought the two disks had differences in my previous experiment.

PS: Every time I do zpool eject <pool>, I get an error "DISK NOT EJECTED PROPERLY". Not major but a bit annoying.