Files corrupted on ZFS

All your general support questions for OpenZFS on OS X.

Files corrupted on ZFS

Postby mtelvers » Tue Feb 21, 2023 7:36 am

I am using ZFS 2.1.6 on macOS Ventura 13.2.1 (Intel) and am experiencing repeatable file corruption in certain circumstances.

I have spent some time tracking down a minimal set of files needed to reproduce this issue. I have staged the three files in a Git repo. Steps to reproduce are:

Code: Select all
git clone https://github.com/mtelvers/zfs-test.git
cd zfs-test
make


The Makefile patches src/dune. The last few lines of the file are shown below:

Code: Select all
(library
 (name base64_rfc2045)
 (modules base64_rfc2045)
 (public_name base64.rfc2045)
 (libraries bytes))


After patching the file, it should end up looking like this and be 244 bytes long.

Code: Select all
(library
 (name base64_rfc2045)
 (modules base64_rfc2045)
 (public_name base64.rfc2045))


However, the file size remains 262 bytes and the original content in those last few bytes is retained.

Code: Select all
(library
 (name base64_rfc2045)
 (modules base64_rfc2045)
 (public_name base64.rfc2045))
libraries bytes))


The key to the failure appears to be files extracted by tar. Looking in the Makefile, you'll see I create an archive of the file and extract it before patching. If you delete these steps, then the patching works as expected.

If you run this on a normal APFS volume, there is no issue, regardless of whether the files are tared or not.

Code: Select all
# zfs version
zfs-macOS-2.1.6-1
zfs-kmod-2.1.6-1
# sw_vers
ProductName:      macOS
ProductVersion:      13.2.1
BuildVersion:      22D68
# patch -v
patch 2.0-12u11-Apple
# tar --version
bsdtar 3.5.3 - libarchive 3.5.3 zlib/1.2.11 liblzma/5.0.5 bz2lib/1.0.8


I cannot reproduce this issue on Monterey.
mtelvers
 
Posts: 6
Joined: Tue Feb 07, 2023 3:06 am

Re: Files corrupted on ZFS

Postby tsmc » Tue Feb 21, 2023 2:46 pm

I can reliably reproduce the problem on macOS Ventura 13.2.1 on ZFS 2.1.6-1 on Intel. I thought perhaps patch had a bug on ZFS, so I tried using `git apply` to do the patching. Using these steps:
Code: Select all
tar -cf src.tar src
tar -xf src.tar
git apply -p1 < 4.02-support.patch


The file is patched (correctly) as:
Code: Select all
(library
 (name base64)
 (modules unsafe base64)
 (public_name base64)
 (libraries result))

(rule
 (copy %{read:../config/which-unsafe-file} unsafe.ml))

(library
 (name base64_rfc2045)
 (modules base64_rfc2045)
 (public_name base64.rfc2045))


It seems like the combination of ZFS, Ventura and patch has a bug.
tsmc
 
Posts: 1
Joined: Tue Feb 21, 2023 2:20 pm

Re: Files corrupted on ZFS

Postby lundman » Tue Feb 21, 2023 5:14 pm

There is some frustration with libarchive using decmpfs compression, and there is nothing we can do to stop it. (we say we dont support it, we say zfs, we say dont-write-that-xattr, it ignores all of it.)

If that is still what this is, can you try with "tar" using the "--nopreserveHFSCompression"?
User avatar
lundman
 
Posts: 1335
Joined: Thu Mar 06, 2014 2:05 pm
Location: Tokyo, Japan

Re: Files corrupted on ZFS

Postby mtelvers » Tue Feb 21, 2023 11:25 pm

Thank you for the comment. Adding "--nopreserveHFSCompression" does not make any difference.

These where my steps, and the file is corrupted again.

Code: Select all
tar -cf src.tar src
tar -xf src.tar --nopreserveHFSCompression
patch -p1 < 4.02-support.patch
mtelvers
 
Posts: 6
Joined: Tue Feb 07, 2023 3:06 am

Re: Files corrupted on ZFS

Postby benmandrew » Wed Feb 22, 2023 6:26 am

I have the same issue, similarly with macOS Ventura 13.2.1 on ZFS 2.1.6-1 on Intel. The "--nopreserveHFSCompression" flag also has no effect for me.
benmandrew
 
Posts: 1
Joined: Wed Feb 22, 2023 6:17 am

Re: Files corrupted on ZFS

Postby lundman » Wed Feb 22, 2023 6:55 pm

User avatar
lundman
 
Posts: 1335
Joined: Thu Mar 06, 2014 2:05 pm
Location: Tokyo, Japan

Re: Files corrupted on ZFS

Postby mtelvers » Wed Feb 22, 2023 11:48 pm

Thanks for the updated version, but unfortunately, it has not addressed the issue.

Working in a newly ZFS volume, I still see the corruption:

Code: Select all
git clone https://github.com/mtelvers/zfs-test.git
cd zfs-test
make


Then checking src/dune shows that the file is corrupt.

Code: Select all
# zfs version
zfs-2.1.99-1734_g3ed0210f3
zfs-kmod-2.1.99-1734_g3ed0210f3
mtelvers
 
Posts: 6
Joined: Tue Feb 07, 2023 3:06 am

Re: Files corrupted on ZFS

Postby lundman » Thu Feb 23, 2023 12:37 am

rats - thanks for checking - I will have dig deeper
User avatar
lundman
 
Posts: 1335
Joined: Thu Mar 06, 2014 2:05 pm
Location: Tokyo, Japan

Re: Files corrupted on ZFS

Postby lundman » Fri Feb 24, 2023 11:53 pm

How does this one run?


OpenZFSonOsX-2.1.99-Catalina-10.15.pkg
(16.39 MiB) Downloaded 308 times
User avatar
lundman
 
Posts: 1335
Joined: Thu Mar 06, 2014 2:05 pm
Location: Tokyo, Japan

Re: Files corrupted on ZFS

Postby mtelvers » Mon Feb 27, 2023 1:34 am

This one runs perfectly.

Thank you very much for your help with this issue.

Mark
mtelvers
 
Posts: 6
Joined: Tue Feb 07, 2023 3:06 am

Next

Return to General Help

Who is online

Users browsing this forum: Google [Bot] and 27 guests