Finder copy of large filesystems failure...

Here you can discuss every aspect of OpenZFS on OS X. Note: not for support requests!

Finder copy of large filesystems failure...

Postby tangles » Mon Mar 23, 2015 11:37 pm

This is more so just a post of my experience as I'm not able to test now and get some log data to properly examine.

Last weekend I performed a Finder copy of pool filesystems from one pool to another. Both pools were connected to the same host via SAS and SATA.
10.10.2
ZFS 1.3.1 RC5
GB X58 UD7
Xeon QC @ 3.2Ghz
24GB DDR1600MHz
RocketRaid 2744 PCIe 16X Lane HBA (set to JBOD), (6 x 4TB disks)
Addonics 6G 10-port SATA PCIe 2X Lane HBA (non RAID), (8 x 2TB disks)

I've since disconnected the 2nd pool and connected it up to an xServe, so the hostnames differ accordingly in the pasted output below.
setup:
Code: Select all
HacProServer:~ madmin$ zpool status
  pool: ztank
 state: ONLINE
  scrub: scrub completed after 5h14m with 0 errors on Sun Mar 15 14:40:38 2015
config:

   NAME        STATE     READ WRITE CKSUM
   ztank       ONLINE       0     0     0
     mirror-0  ONLINE       0     0     0
       disk6   ONLINE       0     0     0
       disk5   ONLINE       0     0     0
     mirror-1  ONLINE       0     0     0
       disk4   ONLINE       0     0     0
       disk3   ONLINE       0     0     0
     mirror-2  ONLINE       0     0     0
       disk2   ONLINE       0     0     0
       disk1   ONLINE       0     0     0

errors: No known data errors
HacProServer:~ madmin$

The above is using 4TB drives and below is using 2TB disks
Code: Select all
Xserve:~ badmin$ zpool status
  pool: btank
 state: ONLINE
scrub: scrub completed after 9h16m with 0 errors on Sun Feb 22 23:06:44 2015
config:

   NAME        STATE     READ WRITE CKSUM
   ztank       ONLINE       0     0     0
     raidz-0   ONLINE       0     0     0
       disk10  ONLINE       0     0     0
       disk9   ONLINE       0     0     0
       disk8   ONLINE       0     0     0
       disk7   ONLINE       0     0     0
     raidz-1   ONLINE       0     0     0
       disk14  ONLINE       0     0     0
       disk13  ONLINE       0     0     0
       disk12  ONLINE       0     0     0
       disk11  ONLINE       0     0     0

errors: No known data errors
Xserve:~ badmin$

filesystems on ztank are:
Code: Select all
madmin$ zfs list
NAME             USED  AVAIL  REFER  MOUNTPOINT
ztank           7.63T  3.08T   460K  /Volumes/ztank
ztank/Comedy     201G  3.08T   201G  /Volumes/ztank/Comedy
ztank/Docos      749G  3.08T   749G  /Volumes/ztank/Docos
ztank/Files     74.6G  3.08T  74.6G  /Volumes/ztank/Files
ztank/Movies    2.53T  3.08T  2.53T  /Volumes/ztank/Movies
ztank/Music      155G  3.08T   155G  /Volumes/ztank/Music
ztank/Pictures   154G  3.08T   154G  /Volumes/ztank/Pictures
ztank/Sport      119G  3.08T   119G  /Volumes/ztank/Sport
ztank/TVShows   2.65T  3.08T  2.65T  /Volumes/ztank/TVShows
ztank/Torrents   933G  3.08T   933G  /Volumes/ztank/Torrents
ztank/Video      120G  3.08T   120G  /Volumes/ztank/Video
HacProServer:~ madmin$

the btank pool has exactly the same filesystems, just an underscore at the start of each of the names to get around the Finder bug of two filesystems of same name but on different pools.
therefore
Code: Select all
NAME         
btank         
btank/_Comedy 
btank/_Docos   
btank/_Files   
btank/_Movies 
btank/_Music   
btank/_Pictures
btank/_Sport   
btank/_TVShows
btank/_Torrents
btank/_Video   

I started with the Comedy filesystem and it transferred over in about 20mins, and so I started with the next file system being Docos.
As it was late in the night I let it run during the night and intended to inspect it after some shut-eye.

In the morning, I switched the monitor back on and the system was totally locked up and frozen screen. The time in the menubar indicated that it happened about 30mins after I kicked off the transfer.

After rebooting, destroying the _Docos filesystem and recreating it and kicking off the transfer, sure enough it locked up again.

I tried another filesystem being Movies. it locked up too. I tried a smaller filesystem such as Video, it copied over fine, as did Sport, Pictures and Music.
So it seemed that small filesystems were transferring okay.

What I did notice is that MenuMeters in the menu bar was at times showing that the system only had 54MB out of 24GB free! This scared me a bit but I figured ZFS should release memory appropriately when he OS wants it back.

Not being 100% confident that ZFS was releasing memory fast enough, before kicking off another Finder copy of the larger filesystems, I did some homework and found the kstat.zfs.darwin.tunable.zfs_arc_max command for RC5 so I set it to use 16GB using:
Code: Select all
sudo sysctl -w kstat.zfs.darwin.tunable.zfs_arc_max=17179869184

and
Code: Select all
$ sudo sysctl -w kstat.spl.misc.spl_misc.simulate_pressure=<amount of ram to release in bytes>

to release the memory immediately.

I kicked off the Finder copy of ztank/Docos to btank/_Docos and it successfully transferred, as did the remaining larger filesystems of TVShows, Movies and Torrents.

I was watching arcstat.pl during this and everything seemed to behave itself.

So, I'm curious if anyone else has encountered this when using the Finder to copy over large filesystems?

In case you're wondering, this backup is my 2nd backup which is why I don't use zfs send/recv (plus I was lazy) and only do this backup about every 90 days.

Regards,
tangles
 
Posts: 69
Joined: Tue Jun 17, 2014 6:54 am

Re: Finder copy of large filesystems failure...

Postby mjt5282 » Thu Mar 26, 2015 6:17 am

my Finder also acts very weird with all sorts of Finder-ish error messages when I have my backup pool loaded at the same time as my primary pool (for backup, like you).

I have purchased a second Supermicro chassis and MB and will switch over my copy to Freenas instead of mac. probably will leave a smallish emerg copy on my mac zfs pool.

I use rsync to copy from primary to backup. Finder copy too unreliable for me.
mjt5282
 
Posts: 6
Joined: Wed Mar 19, 2014 8:56 am


Return to General Discussions

Who is online

Users browsing this forum: No registered users and 1 guest