Performance - again ;-)

Moderators: jhartley, MSR734, nola

Performance - again ;-)

Post by jollyjinx » Thu Jan 03, 2013 8:23 am

I've been using ZFS for a year now and have not looked back using HFS+ again even though ZFS is slower than HFS+.

I'm always using SSD's with a partition to boot and the rest of the SSD for zfs (my home). I'm using a seperate ZIL partition on the SSD ( 1GB) so any synchronos writes can be send immediately. This has made the general interactive performance for me much better - seems that some Prorgrams do use fsync().

A year ago I even purchased more RAM as ZEVO used it back then. Since Feb or March last year it seems that ZEVO no longer uses all available RAM which is a pity. I'm now using metadata only caching for ARC and I create a ramdisk with a few gigabytes ( "hdid -nomount ram://10000000" (5GB); "zpool add tank cache /dev/..." ) for ARC2 which can store both metadata and data.
I do rarely reboot machines so it's not a problem for me to recreate the ramdisk after boot and remove the old cache device from the pool.

Additional note: As I'm using a couple of 2TB 2.5" USB3 WD Passport drives, be aware that those are using 4k Sectors and run best with a zpool created with "-o ashift=12"

Patrick aka Jolly
jollyjinx Offline


 
Posts: 60
Joined: Sun Sep 16, 2012 12:40 pm
Location: Munich - Germany

Re: Performance - again ;-)

Post by zslg01 » Sun Jan 06, 2013 10:49 am

I have noticed a real performance hit when changing meta-data (chmod, chown, chirp, etc) on large groups of files. I expect that the cache is getting a real workout in this case and is pretty poor compared to FreeBSD and/or Linux. Its probably the result of having ZFS grafted on to OS X and not a core part of the system,
Pools also seem to leak memory slowly -- wired memory creeps up over a few days.
zslg01 Offline


 
Posts: 18
Joined: Wed Nov 14, 2012 7:34 pm

Re: Performance - again ;-)

Post by mk01 » Sun Jan 06, 2013 4:33 pm

zfs cache device (does not need to be ssd of hounders GBs - on my linux storage server I use 50GB usb2 disk with 30MB/s max) greatly improves this. as well as any kind of random operation.

and will help even with your updates. every update start with reads. time to finish is only fragment of previous looooongly running process.

edit: almost forgot the point, we wan't to split the IO. so separate device is the key (not it's speed itself).
mk01 Offline


 
Posts: 65
Joined: Mon Sep 17, 2012 1:16 am

Link and thoughts

Post by grahamperrin » Sun Jan 13, 2013 5:42 pm

zslg01 wrote:… changing meta-data (chmod, chown, chirp, etc) on large groups of files.


some actions take longer than expected to complete

I expect that the cache is getting a real workout …


As simply listing the contents of directories – without changing metadata – may take longer than expected, my guess is that the sense of a workout arises:

  • more from the time taken to traverse things
  • less from the time taken to change metadata.
grahamperrin Offline

User avatar
 
Posts: 1596
Joined: Fri Sep 14, 2012 10:21 pm
Location: Brighton and Hove, United Kingdom

Re: Performance - again ;-)

Post by raattgift » Tue Apr 16, 2013 8:17 am

Traversal (and also changing metadata) will create a series of atime updates if the atime property is set to on (the default) in the dataset.

Atime updates are more expensive for zfs than in 10.8's journaled HFS.

As noted elsewhere, the subsystem memory is WIRED to avoid being paged out or otherwise disturbed by the OS. Uncommitted writes are always WIRED, but otherwise zfs will release memory (ARC evictions etc.) and unwire allocation slabs whenever there is memory pressure.

No memory pressure -> WIRED will grow.

"we wan't to split the IO. so separate device is the key (not it's speed itself)."

Many workloads are seek-limited rather than bandwidth limited; moving some operations off one path (spindle, sometimes bus) to another will almost always help. Adding cache vdevs move the operations with the most seeking, so even a fairly slow (in terms of bandwidth) device with low to zero seek latency will help.

However, some workloads really are bandwidth-limited, but those are almost always highly sequential and cache vdevs are rarely involved with any sequential operations.

Very slow busses (like USB 2) are often both bandwidth and seek limited; adding low-latency USB 2 flash sticks to a pool with disks on the same USB bus may adversely affect performance. However, USB 2 mass storage has so many problems that using USB 2 at all is usually a bad idea.
USB 3 is not so bad.

Using a RAM disk as a cache vdev is pretty inventive, even though you have to do zpool remove / zpool add at each boot.
(I am not in love with that because it is easy to forget the "cache" in "zpool add", which can really mess up your day, although of course proper scripting means less chance of typing in something hard to undo).

Do you have -- or can you share -- the zpool iostat -v output for the pool (at least for the cache vdevs) with a ramdisk as a cache vdev versus the same pool using a cache vdev of the sort more commonly used, like an SSD slice or a USB3 memory stick ? If there's a stark difference in the IOPS and bandwidth served up by the cache vdevs as the L2ARC heats up, it'd be interesting to see if the overall reads issued because of UBC misses has also increased.
raattgift Offline


 
Posts: 98
Joined: Mon Sep 24, 2012 11:18 pm


Return to General Discussion

Who is online

Users browsing this forum: ilovezfs and 0 guests

cron