Existing pool -- compression level gzip-9

Moderators: jhartley, MSR734, nola

Existing pool -- compression level gzip-9

Post by caronni » Sat Sep 15, 2012 5:29 am

Hey there,

first of all, thank you for releasing ZEVO ZFS, and together with the MacZFS community keeping ZFS for OSX alive. I feared for the worst when tenscomplement shuttered its doors, and am much relieved to see things continue.

Second, I understand limiting the pool size and disabling dedup, however I think the limiting of gzip compression to gzip-6 is a bad choice. I've been using ZEVO with gzip-9 for the past several months, and was perfectly happy with the space-speed tradeoff that I received that way. Could you please reconsider this, and not limit the choice of compression options?

In a related matter, if I do 'zfs get all' on a FS that I created with zevo from tenscomplement, the 'compression' option is not shown at all:

Code: Select all
~ caronni$ zfs get all y |grep comp
y  compressratio         1.21x                  -
~ caronni$ zfs get compression y
NAME     PROPERTY     VALUE     SOURCE
y  compression  -         -
~ caronni$ zpool create z /Users/caronni/zfile
~ caronni$ zfs get all z |grep comp
z     compressratio         1.00x                  -
z     compression           off                    default


Did I loose the compression setting on the old pool 'y' by importing it into greenbytes zevo? Maybe there is something here that you want to fix.

Be it as it may, please simply reinstate gzip-7 .. gzip-9 and the whole issue becomes moot ;-)

Germano
caronni Offline


 
Posts: 7
Joined: Sat Sep 15, 2012 4:45 am

Re: Existing pool -- compression level gzip-9

Post by grahamperrin » Sun Sep 16, 2012 3:03 pm

Currently at http://www.getgreenbytes.com/zevo/

… some ZFS features like deduplication and CPU-intensive gzip-9 compression are not currently feasible in their current state on Mac OS X.


An example, where dataset tall used compression=gzip-9 before installing ZEVO 1.1:

Code: Select all
macbookpro08-centrim:~ gjp22$ zfs get compression tall
NAME  PROPERTY     VALUE     SOURCE
tall  compression  -         -
macbookpro08-centrim:~ gjp22$ zfs set compression=gzip-6 tall
macbookpro08-centrim:~ gjp22$ zfs get compression tall
NAME  PROPERTY     VALUE     SOURCE
tall  compression  gzip      local
macbookpro08-centrim:~ gjp22$ zfs set compression=gzip-5 tall
macbookpro08-centrim:~ gjp22$ zfs get compression tall
NAME  PROPERTY     VALUE     SOURCE
tall  compression  gzip-5    local
macbookpro08-centrim:~ gjp22$ zfs set compression=gzip-6 tall
macbookpro08-centrim:~ gjp22$ zfs get compression tall
NAME  PROPERTY     VALUE     SOURCE
tall  compression  gzip      local
grahamperrin Offline

User avatar
 
Posts: 1596
Joined: Fri Sep 14, 2012 10:21 pm
Location: Brighton and Hove, United Kingdom

Re: Existing pool -- compression level gzip-9

Post by caronni » Sun Sep 16, 2012 3:22 pm

… some ZFS features like deduplication and CPU-intensive gzip-9 compression are not currently feasible in their current state on Mac OS X.

This is not entirely correct.

Deduplication is feasible but hideously expensive since you can't keep all hashes in memory for reasonable datasets. I understand and endorse that this has been disabled both for differentiation and support purposes. On a side note, disabling this makes the difference between Zevo 1.1 and MacZFS 0.74+ notably smaller.

Gzip-9 worked perfectly well, with tangible but not fatal performance impacts on the CPU side. I hope it comes back :-)

An example, where dataset tall used compression=gzip-9 before installing ZEVO 1.1:
...

Your example showed how you downgraded the zfs dataset to gzip-6 and gzip-5 and how that works nicely. You also show that in the beginning the compression property shows no value (as it is set to gzip-9 on-disk). So, all is well...

Germano

p.s. There is an open question here: When compression is set on-disk to be gzip-9, what does the zevo 1.1 implementation do? Write uncompressed data? Write gzip-6 compressed data? Or write gzip-9 compressed data?
caronni Offline


 
Posts: 7
Joined: Sat Sep 15, 2012 4:45 am

compression=gzip-9 where memory is less than 8 GB

Post by grahamperrin » Sun Sep 16, 2012 9:27 pm

caronni wrote:Gzip-9 worked perfectly well, with tangible but not fatal performance impacts on the CPU side.


With 8 GB memory (the amount that is currently recommended) I had the same experience – tangible but not fatal. MacBookPro5,2.

(gzip-9 only for datasets that I used for archive or backup purposes, and I wrote to those datasets only at times when no other uses of the computer were essential. YMMV.)

compression=gzip-9 where memory is less than 8 GB

I wonder whether writing to, or reading from such datasets was unreasonably problematic on systems with 4 GB or less.
grahamperrin Offline

User avatar
 
Posts: 1596
Joined: Fri Sep 14, 2012 10:21 pm
Location: Brighton and Hove, United Kingdom

Re: Existing pool -- compression level gzip-9

Post by caronni » Sun Sep 16, 2012 10:22 pm

compression=gzip-9 where memory is less than 8 GB


What is the source of your statement? Not doubting, just wondering. I have not read about a memory constraint in conjunction with compression yet.
caronni Offline


 
Posts: 7
Joined: Sat Sep 15, 2012 4:45 am

compression=gzip-9 where memory is less than 8 GB

Post by grahamperrin » Tue Sep 18, 2012 12:44 am

caronni wrote:just wondering


Same here – just wondering.

With 8 GB memory in the MacBookPro5,2, the impact of compression=gzip-9 (whilst writing to such datasets) was sometimes enough to make me refrain from typing – too many keystrokes were unrecognised.

I imagine that with less memory, the impact would be more severe.
grahamperrin Offline

User avatar
 
Posts: 1596
Joined: Fri Sep 14, 2012 10:21 pm
Location: Brighton and Hove, United Kingdom

Re: Existing pool -- compression level gzip-9

Post by dch » Tue Sep 18, 2012 12:36 pm

FWIW I use SSDs with gzip-9 compression to store & run VMWare Fusion OS images, with 4GiB of RAM available for OSX. Under MacZFS, things perform just fine, and with compression + snapshots is a great combo for me. Are the rest of you using SSDs or something else? I'm using a MBP 2011 i7 8core so I have "some cycles to spare".
dch Offline


 
Posts: 2
Joined: Mon Sep 17, 2012 9:11 am

Cross reference

Post by grahamperrin » Thu Mar 28, 2013 11:24 am

At viewtopic.php?p=4408#p4408

raattgift wrote:… REALLLY consider avoiding the dataset compression=gzip-9 for a variety of reasons. It is likely to cause you performance problems without any significant space advantage over the standard lzjb (compression=on) except in highly unusual circumstances (recordsize-aligned data that is written once in a single-thread stream and read extremely infrequently).

If your data is really that compressible, you will always gain from using gzip-9 on the userland side rather on the filesystem side.

xar(1) is your Apple-supported friend.


For the primary file system in that topic, compression is usually on (not gzip-9):

Code: Select all
macbookpro08-centrim:~ gjp22$ zpool history zhandy | grep compression | grep -v ocket | grep -v opt
2012-12-06.18:00:52 zpool create -o ashift=12 -O casesensitivity=insensitive -O normalization=formD -O atime=off -O snapdir=visible -O compression=gzip-9 zhandy /dev/disk2
2012-12-08.18:20:55 zfs set compression=on zhandy
2013-02-27.06:27:28 zfs set compression=off zhandy
2013-03-05.01:58:09 zfs set compression=on zhandy
2013-03-09.00:09:15 zfs set compression=gzip-9 zhandy
2013-03-09.06:12:58 zfs set compression=on zhandy


I can't recall the reason for raising the compression level for those few hours on 2013-03-09 but for the use case, it made sense.

That file system aside, I do often prefer compression=gzip-9 for a file system where backups are received, and backups are scheduled by me (not automated) so any disruption to my workflow is negligible.

See also: Compression of bands within the bundle in this post under Contemplating ZFS.
grahamperrin Offline

User avatar
 
Posts: 1596
Joined: Fri Sep 14, 2012 10:21 pm
Location: Brighton and Hove, United Kingdom

Re: Existing pool -- compression level gzip-9

Post by keister » Thu Mar 28, 2013 11:47 am

Gzip -9 compression levels are very expensive processor wise,
While gzip -1 to gzip -5 still gives good compression with much lower processor usage.

Run a test with your dataset using gzip -9 and another with gzip -1 and compare compressed file sizes.
Also watch the performance processor monitor while running each test and you can see the difference.
Its a balancing act.

While working as a system admin on an IBM unix mainframe the gzip -9 just killed us and we had to back off that level.
keister Offline


 
Posts: 3
Joined: Mon Feb 04, 2013 5:47 pm

Re: Existing pool -- compression level gzip-9

Post by grahamperrin » Thu Mar 28, 2013 12:05 pm

keister wrote:Gzip -9 compression levels are very expensive processor wise …


+1

Extremely noticeable, with my current MacBookPro5,2, during most periods when I choose to write to file systems with that level of compression. So I make those writes at times that will be least disruptive to me.

I never compared with level 1, but between 5 and 9 AFAIR there was a worthwhile difference in the resulting compressratio, where a significant proportion of the writes were backups of bands of sparse bundle disk images. YMMV.
grahamperrin Offline

User avatar
 
Posts: 1596
Joined: Fri Sep 14, 2012 10:21 pm
Location: Brighton and Hove, United Kingdom


Return to General Discussion

Who is online

Users browsing this forum: No registered users and 1 guest

cron