Problems with xattr / v2.1.6 Monterey (Intel)

All your general support questions for OpenZFS on OS X.

Re: Problems with xattr / v2.1.6 Monterey (Intel)

Postby o3x_prokyon » Tue Dec 27, 2022 4:17 am

Setting the "hidden" flag
Code: Select all
chflags hidden "directory"

works as expected on directories and files on the ZFS pool, clearing the flag with "nohidden" too.

Trying to set the user immutable flag
Code: Select all
chflags uchg "directory"

is denied with "operation not permitted" even if called with "sudo" (no idea, if sudo makes any sense here at all...)

Now the interesting part:
Setting the system immutable flag works (sudo is mandatory here):
Code: Select all
sudo chflags schg "directory"

Clearing the flag with "noschg" also works as expected.

More interesting(?): if the system immutable flag is set, setting and clearing the "hidden" flag even works without sudo! But that seems to be a normal behavior: on APFS the "hidden" flag can be set and cleared if the system immutable flag is set, too. But in contrast to ZFS, setting and clearing the user immutable flag works on APFS (without "sudo") as expected.

Oh, by the way... my ZFS pool is set to "com.apple.mimic=apfs" but the "mount" command says it is "hfs". As I mentioned in an earlier post, changing "com.apple.mimic" to "hfs" has obviously no effect. At least, I can't see any...

The last thing I tried was to "unlock" a directory on the ZFS pool in Finder (Lock symbol in the lower right corner of the info-window):
lock.png
lock.png (72.34 KiB) Viewed 195695 times

but nothing happens. On APFS a dialog box opens, asking for the password. So maybe this phenomenon is also related to ACLs???

My acl settings are:
aclmode=discard
aclinherit=restricted
acltype=nfsv4

One more thing: owners are not ignored on any disk, including the ZFS pool and even
Code: Select all
zfs get all

states: com.apple.ignoreowner=off (default)
So this might not be the reason, why setting the user immutable flag doesn't work. I'm clueless...
o3x_prokyon
 
Posts: 33
Joined: Thu Dec 01, 2022 1:17 pm

Re: Problems with xattr / v2.1.6 Monterey (Intel)

Postby o3x_prokyon » Tue Dec 27, 2022 9:56 am

Another interesting result:

Almost every file on my ZFS pool which has been "rsynced", now has an extended attribute which is not present on the local files:
com.apple.system.Security

In this post https://discussions.apple.com/thread/253756427 on the apple community, someone said, that "disk utility" drops an error on that xattr:
warning: found orphan xattr (id xxxxxxx, name com.apple.system.Security)

Rebooting solved this issue for that user, but that's not the case with ZFS.

Repairing a ZFS volume with "disk utility" is no option, so I tried to remove that xattr with
Code: Select all
xattr -d com.apple.system.Security

but again it's not permitted even with "sudo". The file is read-/writable, SIP is disabled so I have no idea what keeps the xattr from being removed.

For testing I just switched the compatibility tunable:
Code: Select all
kstat.zfs.darwin.tunable.zfs.xattr_compat=1

Rsync now again claims "attribute not found" (see my first post), but clears the xattr "com.apple.system.Security", leaving only those xattrs of the source files!!

So, maybe it's better to let that tunable set to "1" after all??? Locking directories and changing folder icons is still not possible, but that's not so relevant (for me), as this is "only" a backup.
o3x_prokyon
 
Posts: 33
Joined: Thu Dec 01, 2022 1:17 pm

Re: Problems with xattr / v2.1.6 Monterey (Intel)

Postby roemer » Thu Jan 05, 2023 4:00 am

I can confirm this with 2.1.6 on Monterey (M1):

I tried the backup bouncer test scripts (https://github.com/n8gray/Backup-Bouncer) between an APFS file system and a ZFS volume, using ChronoSync for the sync.
The verifying check then shows that finder locks, BSD flags, ACLs and resource forks [empty ones] are not synced, respectively not supported on ZFS 2.1.6:
Code: Select all
Backup-Bouncer-master % sudo ./bbouncer verify -d /Volumes/TestSource /Volumes/zfstest
Verifying:    basic-permissions ... ok (Critical)
Verifying:           timestamps ... ok (Critical)
Verifying:             symlinks ... ok (Critical)
Verifying:    symlink-ownership ... ok
Verifying:            hardlinks ... ok (Important)
Verifying:       resource-forks ...
   Sub-test:             on files ... FAIL (Critical)                           [at least if using Posix API]
   Sub-test:  on hardlinked files ... FAIL (Important)
Verifying:         finder-flags ... ok (Critical)
Verifying:         finder-locks ... FAIL
Verifying:        creation-date ... ok
Verifying:            bsd-flags ... FAIL
Verifying:       extended-attrs ...
   Sub-test:             on files ... ok (Important)
   Sub-test:       on directories ... ok (Important)
   Sub-test:          on symlinks ... ok
Verifying: access-control-lists ...
   Sub-test:             on files ... FAIL (Important)
   Sub-test:              on dirs ... FAIL (Important)
Verifying:          combo-tests ...
   Sub-test:  xattrs + rsrc forks ... FAIL


In order to pass the extended-attrs test, I set "kstat.zfs.darwin.tunable.zfs.xattr_compat=1" and "com.apple.mimic=apfs" on the zfs volume.
Code: Select all
Backup-Bouncer-master % sysctl kstat.zfs.darwin.tunable.zfs.xattr_compat                           
kstat.zfs.darwin.tunable.zfs.xattr_compat: 1
Backup-Bouncer-master % zfs get all tank/zfstest
NAME             PROPERTY                   VALUE                 SOURCE
tank/zfstest     xattr                      on                    inherited from tank
tank/zfstest     aclmode                    passthrough           local
tank/zfstest     aclinherit                 restricted            default
tank/zfstest     acltype                    nfsv4                 default
...
tank/zfstest     com.apple.browse           on                    default
tank/zfstest     com.apple.ignoreowner      off                   default
tank/zfstest     com.apple.mimic            apfs                  local
tank/zfstest     com.apple.devdisk          poolonly              default


For comparison, when I use the same source volume and the same ChronoSync script with an HFS+ target volume, all tests are passed.

By the way: When using ChronoSync, the target files have no additional "com.apple.system.Security" attribute as reported above.

[updated 12 Feb 23 to account for bugs in bbouncer]
Last edited by roemer on Sat Feb 11, 2023 9:40 pm, edited 1 time in total.
roemer
 
Posts: 73
Joined: Sat Mar 15, 2014 2:32 pm

Re: Problems with xattr / v2.1.6 Monterey (Intel)

Postby roemer » Sat Feb 11, 2023 9:38 pm

Ok, some weeks and several rsync sessions later, I have a clearer picture of what is going on.
Overall, the meta-data situation with 2.1.6 seems as follows:

Extended Attributes
Supported by ZFS 2.1.6 and fastest if using the -xattr=sa option when creating a dataset.
Two Exceptions: com.apple.FinderInfo and com.apple.ResourceFork extended attributes that are empty (all 0) get dropped / ignored by ZFS 2.1.6.
(actually, it might be that any empty xattr gets dropped - I saw some empty 'com.apple.assetsd.title' attributes which I couldn't sync to ZFS 2.1.6.
On the other hand, on Ventura I can write an all-0 xattr as long as it is not com.apple.FinderInfo - only then this also fails on APFS)

Resource Forks
Resource forks are actually stored as com.apple.ResourceFork extended attribute and as long as they are not empty, they are supported by all extended attribute-based methods. For example, a rsync --xattrs correctly copies resource forks as long as they are not empty or all 0s. Empty resource forks get simply ignored by ZFS.
Important: The Posix mapping of Apple resource forks to a special file path (FILENAME/..namedfork/rsrc) seems unsupported by ZFS 2.1.6, and any software trying to do so (e.g. Finder, or the Backup Bouncer scripts I reported above) fails.

com.apple.FinderInfo
Similar situation: get copied correctly as extended attributes unless they are empty or all-0s. Worthwhile noting, ZFS 2.1.6 zeros out some internal extended fields in the FinderInfo structures. So after copying onto 2.1.6 datasets, bytes 16 to 24 of the FinderInfo attributes are always 0 even if they were set in the original; and if the masking of those bytes makes the whole FinderInfo structure all 0, it doesn't get stored at all. While this doesn't seem to affect any of the 'official' usage of FinderInfo attributes, e.g. for tagging a file with colour flags in Finder, it is quite annoying if you try to rsync from an older ZFS dataset which had such FinderInfo attributes on some of its files...

This also can have the strange side-effect that when accessing an existing dataset which contains all-0 com.apple.FinderInfo attributes, you can list them in a shell, but you cannot read them:
Code: Select all
ls -l@ "/Volumes/PhotoSnapshot/Photos Library.photoslibrary/Photo Database"
-rw-rw-rw-@ 1 user  group  696  9 May  2015 /Volumes/PhotoSnapshot/Photos Library.photoslibrary/Photo Database
   com.apple.FinderInfo   32   [<-- note, non-empty but all zero]
   com.apple.quarantine   46
xattr -lx "/Volumes/PhotoSnapshot/Photos Library.photoslibrary/Photo Database"
   com.apple.FinderInfo:
   00000000
   com.apple.quarantine:
   00000000  30 30 38 36 3B 35 38 38 30 33 39 33 33 3B 50 68  |0086;58803933;Ph|
   00000010  6F 74 6F 20 4C 69 62 72 61 72 79 20 4D 69 67 72  |oto Library Migr|
   00000020  61 74 69 6F 6E 20 55 74 69 6C 69 74 79 3B        |ation Utility;|
xattr -px com.apple.FinderInfo "/Volumes/PhotoSnapshot/Photos Library.photoslibrary/Photo Database"
[nothing]


BSD Flags
BSD flags seem supported on ZFS 2.1.6 except the user immutable (uchg) and the archived (arch) flags. The uchg is the flag which is used by Finder to 'Lock' a file or directory... This also means that the shell commands SetFile -a l FILENAME and chflags uchg,arch FILENAME do not work.
Warning: I think rsync v3.2.7 has a bug in its --fileflags option which mangles the BSD flags during copy... I had problems with it during remote syncs.

ACLs
I have not been successful in getting any ACLs copied or manually set on a ZFS 2.1.6 dataset, regardless of which settings I tried. It seems ACLs are currently unsupported.

What does this mean? Does any of this matter?

I would say yes: This thread started with the problem that user-defined Finder Icons and locking of files in Finder does not work on ZFS 2.1.6.
The reason is that ZFS 2.1.6 does not support the 'uchg' BSD flag (which is used by Finder to lock a file) and that Finder seems to store icons as a named resource fork using the "FILENAME/..namedfork/rsrc" special pathname which is not supported by ZFS 2.1.6.
roemer
 
Posts: 73
Joined: Sat Mar 15, 2014 2:32 pm

Re: Problems with xattr / v2.1.6 Monterey (Intel)

Postby o3x_prokyon » Mon Feb 13, 2023 12:32 pm

@roemer: Thanks for your detailed work and your findings. That rsync 3.2.7 has some problems with fileflags, is something I noticed, too. So in the first place I thought rsync was the only problem, but then I tried to set the immutable flag directly in finder as well as with the chflags command in terminal.

I think, your results and conclusions are a lot more useful for the further development, than mine. I'm sure, the upcoming versions of ooo (and rsync) will fix these issues. For my backups the missing flags and xattrs are a minor problem, but in another environment they might be essential.
o3x_prokyon
 
Posts: 33
Joined: Thu Dec 01, 2022 1:17 pm

Re: Problems with xattr / v2.1.6 Monterey (Intel)

Postby lundman » Wed Feb 15, 2023 3:43 pm

Excellent summary - which would you say should be corrected first?
User avatar
lundman
 
Posts: 1335
Joined: Thu Mar 06, 2014 2:05 pm
Location: Tokyo, Japan

Re: Problems with xattr / v2.1.6 Monterey (Intel)

Postby lundman » Thu Feb 16, 2023 12:26 am

User avatar
lundman
 
Posts: 1335
Joined: Thu Mar 06, 2014 2:05 pm
Location: Tokyo, Japan

Re: Problems with xattr / v2.1.6 Monterey (Intel)

Postby roemer » Thu Feb 16, 2023 2:44 am

Many thanks - this was fast!

The patch is actually much more substantial than I expected; I thought the problem is with this check in zfs_vnops_os.c:zfs_setattr() (lines 2061 - 2069) which is done _after_ the immutable flag was set in zfs_setbsdflags(). Is this a problem?
Code: Select all
   /*
    * Immutable files can only alter immutable bit and atime
    */
   if ((zp->z_pflags & ZFS_IMMUTABLE) &&
       ((mask & (ATTR_SIZE|ATTR_UID|ATTR_GID|ATTR_MTIME|ATTR_MODE)) ||
       ((mask & ATTR_XVATTR) && XVA_ISSET_REQ(xvap, XAT_CREATETIME)))) {
      ZFS_EXIT(zfsvfs);
      return (SET_ERROR(EPERM));
   }

To answer the previous question about which meta-data to fix first: Besides the BSD flags, I think support for '/..namedfork/rsrc' to access the resource fork extended attribute would be great as I suspect this affects the Finder for setting folder icons...
roemer
 
Posts: 73
Joined: Sat Mar 15, 2014 2:32 pm

Re: Problems with xattr / v2.1.6 Monterey (Intel)

Postby o3x_prokyon » Thu Feb 16, 2023 9:12 am



Hey, thanks for the quick fix! So, I should try to compile the patched version... or will there be an "official" new version?
o3x_prokyon
 
Posts: 33
Joined: Thu Dec 01, 2022 1:17 pm

Re: Problems with xattr / v2.1.6 Monterey (Intel)

Postby lundman » Thu Feb 16, 2023 3:27 pm

roemer wrote:Many thanks - this was fast!

The patch is actually much more substantial than I expected; I thought the problem is with this check in zfs_vnops_os.c:zfs_setattr() (lines 2061 - 2069) which is done _after_ the immutable flag was set in zfs_setbsdflags(). Is this a problem?
Code: Select all
   /*
    * Immutable files can only alter immutable bit and atime
    */
   if ((zp->z_pflags & ZFS_IMMUTABLE) &&
       ((mask & (ATTR_SIZE|ATTR_UID|ATTR_GID|ATTR_MTIME|ATTR_MODE)) ||
       ((mask & ATTR_XVATTR) && XVA_ISSET_REQ(xvap, XAT_CREATETIME)))) {
      ZFS_EXIT(zfsvfs);
      return (SET_ERROR(EPERM));
   }



That is actually the problem, we set ZFS_IMMUTABLE before we call into zfs_setattr(), so then that code there would fail, and return EPERM.
This highlighted there is no good way to pass those extra flags into zfs_setattr() since it is 32bit only, changing that would be a whole thing,
since vattr is a XNU defined struct.

But really, while I was poking at it, it is clear we were handling all the z_pflags seperately to the zfs_setattr() call, and relying on "luck" that
the z_pflags gets synced to disk "some time later by some other change to the same file". So I figured I might as well to it right, the way
those extra flags are "supposed" to be passed into zfs_setattr() is via xvattr - extra vattrs. I've ignored that before, so it was time.

Looking at Linux, I copied the way they handled it, translating from XNU flags, to ZFS flags, to XVA flags to pass into zfs_setattr(), just
like all the other platforms. Now ZFS_IMMUTABLE is set "inside" zfs_setattr(), instead of before the call. The change to z_pflags is synced to disk
as part of the zfs_setattr().
User avatar
lundman
 
Posts: 1335
Joined: Thu Mar 06, 2014 2:05 pm
Location: Tokyo, Japan

PreviousNext

Return to General Help

Who is online

Users browsing this forum: No registered users and 30 guests