OpenZFS on OS X

Since my upgrade to Big Sur, ZFS has been working fine on my Mac, except when receiving from a send initiated by an Ubuntu server which results in a bad magic number error. I have the latest versions of ZFS on all machines. Ubuntu version is 20.10. Receives on the same Mac were fine before upgrading to Big Sur and the new OpenZFS on OS X.

To confirm the issue, I've been able to send from the same Ubuntu server to another Ubuntu server without problems which would seem to indicate that the problem is not on the sending Ubuntu server. Rsyncs from the same Ubuntu server to my Mac also work without issue so I don't believe there is a true magic number problem on any of my drives.

Further symptom.

I forgot to set up encryption on my filesystem so I wanted to copy it over to another filesystem in the same pool. I issued:

Code: Select all

sudo zfs send -Rw whp_pool/backups@now | sudo zfs receive -su whp_pool/copies/backups

Not sure where I turned on debugging but I received another bad magic number error:

Code: Select all

libzfs_macos_wrapfd: checking if we need pipe wrap
libzfs_macos_wrapfd: is pipe: work on fd 1
libzfs_macos_wrapfd: readfd 7 (0)
libzfs_macos_wrapfd: writefd 8 (0)
libzfs_macos_wrapfd: spawning thread
pipe_io_relay: thread up: read(7) write(1)
libzfs_macos_wrapfd: checking if we need pipe wrap
libzfs_macos_wrapfd: is pipe: work on fd 0
libzfs_macos_wrapfd: readfd 6 (0)
libzfs_macos_wrapfd: writefd 7 (0)
libzfs_macos_wrapfd: spawning thread
pipe_io_relay: thread up: read(0) write(7)
libzfs_macos_wrapfd: checking if we need pipe wrap
libzfs_macos_wrapfd: is pipe: work on fd 8
libzfs_macos_wrapfd: readfd 7 (0)
libzfs_macos_wrapfd: writefd 9 (0)
libzfs_macos_wrapfd: spawning thread
pipe_io_relay: thread up: read(7) write(8)
cannot receive: invalid stream (bad magic number)
warning: cannot send 'whp_pool/backups/users@now': signal received

I am having same issue on macOS Big Sur (11.2.2).

Send one stream at a time, the multi-stream code was broken.

Alternatively, try today's build which fixes it, hopefully.

In short:
Thank you so much. The pi date (2021.03.14) version of 2.0 fixed this problem.

viewtopic.php?f=20&t=3478#p10876

Details:
I got the following errors when sending snapshots of some file systems. I guess it might be problems caused by previous failures. The problem resolved after I destroyed the last snapshot on target file system.

Batch 1 -

Code: Select all

+ zfs send -I earth/Docks@2021-02-28-204659-Sync earth/Docks@2021-03-18-230025-Sync
+ zfs receive -F W/Backups/earth/Docks
cannot receive incremental stream: most recent snapshot of W/Backups/earth/Docks does not
match incremental source
warning: cannot send 'earth/Docks@2021-03-07-184706-Sync': signal received
warning: cannot send 'earth/Docks@2021-03-18-230020-Sync': Broken pipe
warning: cannot send 'earth/Docks@2021-03-18-230025-Sync': Broken pipe

Batch 2 -

Some of file systems will have the following errors when sending snapshots. However, it does not have the same problem after second try (without doing anything).

Code: Select all

+ zfs send -I earth/Life/Dots@2021-02-28-204712-Sync earth/Life/Dots@2021-03-18-230028-Sync
+ zfs receive -F W/Backups/earth/Life/Dots
cannot receive incremental stream: checksum mismatch
+ zfs send -I earth/Life/Pal@2021-02-28-204712-Sync earth/Life/Pal@2021-03-18-230029-Sync
+ zfs receive -F W/Backups/earth/Life/Pal
cannot receive incremental stream: invalid backup stream

References -

Sending and Receiving ZFS Data - Oracle Solaris ZFS Administration Guide
https://docs.oracle.com/cd/E26505_01/ht ... gbchx.html

Disabling access time (atime) updates on ZFS file systems -- Prefetch Technologies
https://prefetch.net/blog/2006/07/25/di ... le-system/

ZFS Best Practices and Caveats | Bigstep Blog
https://bigstep.com/blog/zfs-best-practices-and-caveats

Tested again with 2021-04-03 version
viewtopic.php?f=20&t=3531
OpenZFSonOsX-2.0.0-Big.Sur-11.0.pkg

It works! It can complete all the ZFS send/receive in one batch.

Code: Select all

# zfs version
zfs-2.0.0-rc1_442_g816946801e
zfs-kmod-zfs-2.0.0-rc1-442-g816946801e