Duplicated files become hidden in Finder

All your general support questions for OpenZFS on OS X.

Re: Duplicated files become hidden in Finder

Postby evereasy » Mon Apr 05, 2021 1:49 am

That's great! Big Sur, that is what I am using on my MacBook. Can not wait to test it!
evereasy
 
Posts: 30
Joined: Sat Feb 27, 2021 6:38 am

Re: Duplicated files become hidden in Finder

Postby lundman » Mon Apr 05, 2021 1:52 am

intel?
User avatar
lundman
 
Posts: 1335
Joined: Thu Mar 06, 2014 2:05 pm
Location: Tokyo, Japan

Re: Duplicated files become hidden in Finder

Postby evereasy » Mon Apr 05, 2021 1:54 am

Sorry, Yes. Intel.

Thank you so much.
evereasy
 
Posts: 30
Joined: Sat Feb 27, 2021 6:38 am

Re: Duplicated files become hidden in Finder

Postby lundman » Mon Apr 05, 2021 2:20 am

OK give that a whirl - I believe I tagged it rc2 as well
User avatar
lundman
 
Posts: 1335
Joined: Thu Mar 06, 2014 2:05 pm
Location: Tokyo, Japan

Re: Duplicated files become hidden in Finder

Postby evereasy » Mon Apr 05, 2021 5:31 am

Tested with the following version:
viewtopic.php?f=20&t=3533
OpenZFSonOsX-2.0.0-Big.Sur-11.0.pkg
Code: Select all
# zfs version
zfs-2.0.0-rc1_444_ge0b34dd9c7
zfs-kmod-zfs-2.0.0-rc1-444-ge0b34dd9c7

Results:
1. The xattr -l errors are gone. ie. no more "[Errno 34] Result too large"
2. The duplicated files (using Finder/duplicate) are still hidden and with a color tag.
3. Need to run xattr -c to clear extended attributes.

Code: Select all
# xattr -l time\ copy.txt
com.apple.FinderInfo:
00000000  01 00 08 00 C1 FF FF FF A0 A9 7F 86 C1 FF FF FF  |................|
00000010  00 00 00 00 00 00 00 00 08 00 0B 00 00 00 00 00  |................|
00000020
# xattr -c time\ copy.txt
# xattr -l time\ copy.txt

So the question is why the duplicated files get these extended attributes?
evereasy
 
Posts: 30
Joined: Sat Feb 27, 2021 6:38 am

Re: Duplicated files become hidden in Finder

Postby lundman » Mon Apr 05, 2021 1:50 pm

Finder duplicate adds the finderinfo, that is fine. But there is a bunch of special handling of it, HFS seems to "zero" out some fields, which is perhaps what we should be doing. We are also supposed to say "its not there" if it is all zeros.
User avatar
lundman
 
Posts: 1335
Joined: Thu Mar 06, 2014 2:05 pm
Location: Tokyo, Japan

Re: Duplicated files become hidden in Finder

Postby evereasy » Tue Apr 06, 2021 9:26 am

Thank you so much for sharing this information. These 32 bytes are really interesting after browse through GitHub - dropbox/finderinfo-rust

When saving files from Powerpoint, I am having the same problem as Finder.

I did some experiment to set the Green tag, on APFS this will work, on ZFS this will not work.
$a is a path to an APFS volume
$z is a path to an ZFS volume
Code: Select all
$ xattr -wx com.apple.FinderInfo "0000000000000000000400000000000000000000000000000000000000000000" $a/Presentation1.pptx
$ xattr -l $a/Presentation1.pptx
com.apple.FinderInfo:
00000000  00 00 00 00 00 00 00 00 00 04 00 00 00 00 00 00  |................|
00000010  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  |................|
00000020
$ xattr -wx com.apple.FinderInfo "0000000000000000000400000000000000000000000000000000000000000000" $z/Presentation1.pptx
$ xattr -l $z/Presentation1.pptx

Here are the outputs of xattr -l of saved files (using Excel, Word, Powerpoint).
Code: Select all
$ xattr -l *
Book1.xlsx: com.apple.quarantine: 0086;606c89cc;Microsoft Excel;
Doc1.docx: com.apple.metadata:_kMDItemUserTags:
00000000  62 70 6C 69 73 74 30 30 A0 08 00 00 00 00 00 00  |bplist00........|
00000010  01 01 00 00 00 00 00 00 00 01 00 00 00 00 00 00  |................|
00000020  00 00 00 00 00 00 00 00 00 09                    |..........|
0000002a
Doc1.docx: com.apple.quarantine: 0082;606c89b1;Microsoft Word;
Presentation1.pptx: com.apple.metadata:_kMDItemUserTags:
00000000  62 70 6C 69 73 74 30 30 A0 08 00 00 00 00 00 00  |bplist00........|
00000010  01 01 00 00 00 00 00 00 00 01 00 00 00 00 00 00  |................|
00000020  00 00 00 00 00 00 00 00 00 09                    |..........|
0000002a
Presentation1.pptx: com.apple.FinderInfo:
00000000  00 00 00 00 2E 97 01 00 60 F8 D1 83 93 FF FF FF  |........`.......|
00000010  00 00 00 00 00 00 00 00 51 4B 9A A2 00 00 00 00  |........QK......|
00000020
Presentation1.pptx: com.apple.quarantine: 0086;606c8963;Microsoft PowerPoint;

With google search, I managed to create a shell script to clear extended attributes from Finder so the duplicated files can appear as "normal" files. I am using it as a workaround for right now.
Code: Select all
for f in "$@"
do
   date
   echo "$f"
   /usr/bin/xattr -l "$f"
   /usr/bin/xattr -c "$f"
done >> /tmp/xattr.log 2>&1

Here is the content of /tmp/xattr.log, the com.apple.FinderInfo is not the same when duplicating files ... :-( :
Code: Select all
Tue Apr  6 22:19:59 CST 2021
/Volumes/earth/Testbed/time copy 2.txt
com.apple.FinderInfo:
00000000  01 00 08 00 C1 FF FF FF D0 71 A4 07 C2 FF FF FF  |.........q......|
00000010  00 00 00 00 00 00 00 00 08 00 0B 00 00 00 00 00  |................|
00000020
Tue Apr  6 22:19:59 CST 2021
/Volumes/earth/Testbed/time copy 3.txt
com.apple.FinderInfo:
00000000  01 00 08 00 A1 FF FF FF 08 79 35 08 C2 FF FF FF  |.........y5.....|
00000010  00 00 00 00 00 00 00 00 08 00 0B 00 00 00 00 00  |................|
00000020
Tue Apr  6 22:19:59 CST 2021
/Volumes/earth/Testbed/time copy 4.txt
com.apple.FinderInfo:
00000000  01 00 08 00 C1 FF FF FF E0 59 29 2E C1 FF FF FF  |.........Y).....|
00000010  00 00 00 00 00 00 00 00 08 00 0B 00 00 00 00 00  |................|
00000020
Tue Apr  6 22:20:00 CST 2021
/Volumes/earth/Testbed/time copy 5.txt
com.apple.FinderInfo:
00000000  01 00 08 00 C1 FF FF FF 40 49 A6 4D C1 FF FF FF  |........@I.M....|
00000010  00 00 00 00 00 00 00 00 08 00 0B 00 00 00 00 00  |................|
00000020
Tue Apr  6 22:20:00 CST 2021
/Volumes/earth/Testbed/time copy 6.txt
com.apple.FinderInfo:
00000000  01 00 08 00 C1 FF FF FF 28 62 81 07 C2 FF FF FF  |........(b......|
00000010  00 00 00 00 00 00 00 00 08 00 0B 00 00 00 00 00  |................|
00000020
Tue Apr  6 22:20:00 CST 2021
/Volumes/earth/Testbed/time copy 7.txt
com.apple.FinderInfo:
00000000  01 00 08 00 C1 FF FF FF 40 F2 9A 30 C1 FF FF FF  |........@..0....|
00000010  00 00 00 00 00 00 00 00 08 00 0B 00 00 00 00 00  |................|
00000020
Tue Apr  6 22:20:01 CST 2021
/Volumes/earth/Testbed/time copy 8.txt
com.apple.FinderInfo:
00000000  01 00 08 00 C1 FF FF FF 80 89 B2 2F C1 FF FF FF  |.........../....|
00000010  00 00 00 00 00 00 00 00 08 00 0B 00 00 00 00 00  |................|
00000020
Tue Apr  6 22:20:01 CST 2021
/Volumes/earth/Testbed/time copy 9.txt
com.apple.FinderInfo:
00000000  01 00 08 00 C1 FF FF FF 30 79 43 49 C1 FF FF FF  |........0yCI....|
00000010  00 00 00 00 00 00 00 00 08 00 0B 00 00 00 00 00  |................|
00000020
Tue Apr  6 22:20:01 CST 2021
/Volumes/earth/Testbed/time copy.txt
com.apple.FinderInfo:
00000000  01 00 08 00 C1 FF FF FF E0 48 9C 30 C1 FF FF FF  |.........H.0....|
00000010  00 00 00 00 00 00 00 00 08 00 0B 00 00 00 00 00  |................|
00000020
Tue Apr  6 22:20:02 CST 2021
/Volumes/earth/Testbed/time.txt

References -
xattr: com.apple.FinderInfo, information for the Finder – The Eclectic Light Company
https://eclecticlight.co/2017/12/19/xat ... he-finder/

Create Context Menu Items in Mac Finder
https://davidwalsh.name/mac-context-menu

MAC OS filesystem: extended xattr FinderInfo,quarantine,ResourseFork TextEncoding
https://www.real-world-systems.com/docs/xattr.1.html

GitHub - dropbox/finderinfo-rust
https://github.com/dropbox/finderinfo-rust
evereasy
 
Posts: 30
Joined: Sat Feb 27, 2021 6:38 am

Re: Duplicated files become hidden in Finder

Postby lundman » Tue Apr 06, 2021 4:10 pm

Have you found what we do wrong? HFS source say to zero out the ADDEDTIME area, which I think we do, and we are supposed to scribble something elsewhere in the struct, but maybe we miss (off by one?) But if Tags are set on when duplicating on ZFS, are you seeing that we modify a byte we shouldn't ? If you find things we do and can let me know, I can fix them. I appreciate the debugging you have done.

Like:

Code: Select all
com.apple.FinderInfo:
00000000  01 00 08 00 C1 FF FF FF E0 48 9C 30 C1 FF FF FF  |.........H.0....|
ZFS-set-this-bit-and-shouldnt-----^^
User avatar
lundman
 
Posts: 1335
Joined: Thu Mar 06, 2014 2:05 pm
Location: Tokyo, Japan

Re: Duplicated files become hidden in Finder

Postby lundman » Thu Apr 08, 2021 12:45 am

This is odd. When duplicating an empty file, I receive:

<zfs`zfs_vnop_setxattr (zfs_vnops_osx.c:3746)> 08 00 07 00 88 ff ff ff 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 c8 38 88 90 00 00 00 00

then Tag Red (and Purple turns on as well)
<zfs`zfs_vnop_setxattr (zfs_vnops_osx.c:3746)> 01 00 00 00 03 00 00 00 00 26 d9 2d 80 ff ff ff 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

Then remove Red (File gets Hidden)

<zfs`zfs_vnop_setxattr (zfs_vnops_osx.c:3746)> 00 00 00 00 00 00 00 00 f0 91 2f 29 80 ff ff ff 00 00 00 00 00 00 00 00 01 00 00 00 00 00 00 00

These are the binary that ZFS receives from XNU, before we mess with them, so it is confusing how some fields have shifted. Surely
it can't do partial writes to finderinfo.
User avatar
lundman
 
Posts: 1335
Joined: Thu Mar 06, 2014 2:05 pm
Location: Tokyo, Japan

Re: Duplicated files become hidden in Finder

Postby evereasy » Thu Apr 08, 2021 6:49 am

Thank you for working on this issue. I was trying the finderinfo-rust. I am able to use 'cargo build' to build the binary. However, the binary failed to read extended attributes of files on APFS volume via xattr . That's what I have so far. I'll do more experiments tomorrow and at the weekend. One of the idea is to just set a single bit and observe the results ...

GitHub - dropbox/finderinfo-rust
https://github.com/dropbox/finderinfo-rust
evereasy
 
Posts: 30
Joined: Sat Feb 27, 2021 6:38 am

PreviousNext

Return to General Help

Who is online

Users browsing this forum: Google [Bot] and 28 guests

cron