UTF8 file name horror with rsync and terminal

All your general support questions for OpenZFS on OS X.

UTF8 file name horror with rsync and terminal

Postby sir_brickalot » Wed Sep 13, 2017 9:45 am

I ran into a problem where certain files got resyced on local drives (destinations could be HFS+, FAT32 or ExFAT) with rsync over and over when using the --delete option and the common denominators were utf-8 special characters (ä,ö,ü,&) in the file name.
The solution to this problem was, (after days of trial&error) to use 'rsync -a --iconv=UTF8,UTF8-MAC /src/ /dest/'

Files created by terminal commands i.e. $touch ~/test/testfile_ä.txt would run into the syncing problem, while files created with i.e. TextEdit > Save... > testfile_ü.txt would not.

Questions:
What is the reason for this?
How do I have to set up the Terminal so it uses the correct filename codepage?
How du I have to set up my pool correctly to prevent this problem in the future?

I can provide more information if necessary.
To recreate the problem:
Code: Select all
$mkdir /tank/test/
$touch /tank/test/fileäöü.txt
$mkdir /drive_hfs/test/
$rsync  -rtPvhi /tank/test/ /drive_hfs/test/
sending incremental file list
.d..t....... ./
>f++++++++++ fileäöü.txt
              0 100%    0.00kB/s    0:00:00 (xfr#1, to-chk=0/2)
sent 98 bytes  received 63 bytes  322.00 bytes/sec

$rsync  -rtPvhi /tank/test/ /drive_hfs/test/ --delete
sending incremental file list
*deleting   fileäöü.txt
>f++++++++++ fileäöü.txt
              0 100%    0.00kB/s    0:00:00 (xfr#1, to-chk=0/2)
sent 98 bytes  received 56 bytes  308.00 bytes/sec

$rsync  -rtPvhi /tank/test/ /drive_hfs/test/ --delete --iconv=UTF8,UTF8-MAC
sending incremental file list

sent 55 bytes  received 12 bytes  134.00 bytes/sec
total size is 0  speedup is 0.00



My zpool options:
Code: Select all
sudo zpool create -f -o ashift=12 -O casesensitivity=insensitive -O atime=off -O normalization=formD -O compression=lz4 -O utf8only=on ZFS_POOLNAME mirror 'diskX' 'diskY'

rsync version: 3.1.2
sir_brickalot
 
Posts: 23
Joined: Tue Apr 11, 2017 11:13 am

Return to General Help

Who is online

Users browsing this forum: No registered users and 28 guests