rsync -a to case sensitive normalized hangs

[webb]

Code: Select all

rsync -a pool1/dataset1/* pool2/dataset1/.

hangs

• also cases pool1 to disappear from zpool list, although the files are still visible via ls pool1/dataset1
• also hangs the copy process, i.e., hard disk sounds stop and no more changes on destination
• also causes subsequent zpool export to fail
• also prevents soft restart/shutdown; must poweroff with the power button

Workaround: cp -a instead of rsync -a
(My goal is to move dataset1 to new dataset1 + normalization=formD, so I can't use zfs send/recv.)

Non-default settings:

Code: Select all

pool1/dataset1  recordsize       1M         inherited from pool1
pool1/dataset1  compression      lz4        received
pool1/dataset1  atime            off        inherited from pool1
pool1/dataset1  normalization    none       -
pool1/dataset1  casesensitivity  sensitive  -
pool2/dataset1  recordsize       1M         local
pool2/dataset1  compression      lz4        inherited from pool2
pool2/dataset1  atime            off        inherited from pool2
pool2/dataset1  normalization    formD      -
pool2/dataset1  casesensitivity  sensitive  -

Each pool was created to use an entire disk:

Code: Select all

sudo zpool create -f -O atime=off -O maxrecordsize=1M pool1 disk1
sudo zpool create -f -o ashift=12 -O normalization=formD -O compression=lz4 -O atime=off pool2 disk2

None is a boot disk, only data. Boot disk is APFS.

Any suggestions on how I can debug this for myself and/or provide a more informative bug report? For example, I could try rsync -av to find out on which file it hangs and get more information about that file. This happened twice, and at the same point in terms of total amount copied both times, so it probably is one specific file.

OpenZFS on OSX 1.9.4
MacPorts rsync version 3.2.3 protocol version 31
macOS 10.13.6
Mac Pro 4,1 flashed to 5,1
96 GB of RAM with >40% free at time of hang

[webb]

cp -a gave errors for some non-UTF-8 filenames. I was able to fix the issue by converting all filenames in the source dataset from Windows-1250 / ISO-8859-2 to UTF-8:

Code: Select all

find pool1/dataset1 -print0 | xargs -r -0 convmv -f CP1250 -t UTF-8 --notest

then cp -a; rsync -a worked fine (though rsync was at that point only verifying, not copying)

Preferably, bad characters in filenames shouldn't be able to hang the filesystem and cause a zpool to disappear from zpool list. I can do a rollback to repro the error and collect more info if anyone wants to investigate further. Just let me know.

[xenophon]

Curious: how come you use case-sensitive zpools?

Aren't they troublesome in a MacOS environment?

Thank you.

[webb]

> how come you use case-sensitive zpools

The short answer is I didn't consider nor explicitly choose any setting of casesensitivity when creating the zpools, and the default setting was sensitive. Perhaps the best solution is to recreate the zpools as insensitive. I haven't tried this, so I'm not sure if the unexpected behavior will still occur for Windows-1250 / ISO-8859-2 in filenames.