A Mirror of RAID-Z's?

New to OpenZFS on OS X (Or ZFS in general)? Ask your questions here!

Re: A Mirror of RAID-Z's?

Postby Haravikk » Thu Apr 16, 2015 12:50 am

rattlehead wrote:For the 6TB of data, the recommended minimum RAM was like 24GB or so, counting only the memory dedicated to ZFS, I do not have that much, so performance would be worse, I knew that. But the host system became practically useless, GUI feedback for moving the mouse took seconds, etc.

Wow, I knew it recommended quite a bit of RAM, but I didn't expect it to be that bad; this was why I asking about caching though as I'm also only on 16gb like you, but I haven't been able to find any good articles on whether a cache disk/partition/file can be configured to offload the de-duplication table onto. If it's something I can enable later though then that's good to know, as I'll just leave it until I can find out more, or run a test once I've thinned down my cable forest enough to hook up a spare disk for testing :)

I did find reference to a feature that can improve deduplication performance, but it seems OpenZFS doesn't support it; which is the ability to specify a less demanding checksum as part of the algorithm,verify notation, but it looks like while OpenZFS supports the notation, it only supports SHA-256 for checksums at the moment. Shame, as several examples using dedup=fletcher4,verify seem to show much lower memory requirements due to the smaller checksum size, and it remains safe (because collisions are verified before the blocks are linked).

rattlehead wrote: compression for me also made things worse.

That's surprising; which compression algorithm were you using? I wouldn't have thought that ZFS compression could result in larger sizes at all, as surely that would break the block size? Most compression algorithms I know would only store the compressed data if it's smaller than the original, otherwise the original is written as-is. For example, I had to port the LZMA algorithm a while back, and it structures data into discreet units with a tiny, ~10 bytes or so, header at the start that describes the mode of compression (if any) and the size of the uncompressed data (which, in the case of ZFS, should always be the same, e.g- 4kb), so at most an incompressible piece of data would only add a few bytes of overhead for this header. This overhead might add up eventually, but it shouldn't be more than a couple of gigabytes to every terabyte (max).

Of course you're right that not all data will benefit, and I'm currently undecided as de-duplication is the kind of "compression" that would likely benefit me the most since I'll just be using OS X's Time Machine for backup (onto an unencrypted disk image) and as you may know Time Machine only operates at the file level, so it often copies a whole file even if only a few bytes have changed, which seems like the kind of thing that de-duplication would handle well (since the unchanged blocks would be linked together), assuming I can avoid or diminish the huge RAM requirements somehow. But yeah, I'm not sure what portion of my backups would benefit from compression, I might have a hunt around and see if there are any compression tools that could do a "dry run" at my data and tell me how much space I could theoretically save.
Last edited by Haravikk on Thu Apr 16, 2015 2:22 am, edited 1 time in total.
Haravikk
 
Posts: 74
Joined: Tue Mar 17, 2015 4:52 am

Re: A Mirror of RAID-Z's?

Postby rattlehead » Thu Apr 16, 2015 1:23 am

Good luck!
And reports are very welcome ;-)
rattlehead
 
Posts: 12
Joined: Tue Apr 14, 2015 11:54 am

Re: A Mirror of RAID-Z's?

Postby realfolkblues12 » Fri Apr 17, 2015 3:08 pm

Haravikk wrote:Since I'm only going to be backing up periodically to this array, I'm currently trying to decide whether to leave it running 24/7 or to instead only switch it on when a new weekly backup is needed


It all depends. Is the machine on all the time anyways? if so the drives will spin down so it's not a big power draw difference. if you do decide to use deduplication it is a big ram hog so that might influence wether or not you keep the pool imported.
realfolkblues12
 
Posts: 34
Joined: Thu Feb 12, 2015 10:41 pm

Re: A Mirror of RAID-Z's?

Postby Haravikk » Sat Apr 18, 2015 2:45 am

realfolkblues12 wrote:
Haravikk wrote:Since I'm only going to be backing up periodically to this array, I'm currently trying to decide whether to leave it running 24/7 or to instead only switch it on when a new weekly backup is needed

It all depends. Is the machine on all the time anyways? if so the drives will spin down so it's not a big power draw difference.

The drives are all in an external JBOD enclosure (well, technically it supports RAID but the controllers aren't very reliable during failures, which isn't a great feature ;))

realfolkblues12 wrote:if you do decide to use deduplication it is a big ram hog so that might influence wether or not you keep the pool imported.

This is partly why I was asking about cache drives; I've found a few good articles that state that the de-duplication tables are treated as metadata. Since I shouldn't need much in the way of data caching, this hopefully means I could designate a modest amount of RAM for ARC (say… 2gb) and then part of my SSD as a metadata only l2ARC, I'm just curious to know what kind of performance I might expect, as it seems that the de-duplication table will be very much random access so it'll end up coming from the SSD only? My SSD is a Samsung 850 EVO, so it pretty much maxes out the performance of SATA III on sequential and random access, I'm just curious whether this would take enough of an edge off de-duplications hardware requirements to make it viable?

It may affect the decision to keep the array on 24/7 or not actually, if the caches used by a zpool aren't persistent (i.e- will the de-duplication table need to reload every time the array is reconnected, or it will still be found in the cache first?)
Haravikk
 
Posts: 74
Joined: Tue Mar 17, 2015 4:52 am

Re: A Mirror of RAID-Z's?

Postby realfolkblues12 » Sat Apr 18, 2015 12:20 pm

Dedupe can be turned on or off without issue. So you might just play with it and see how it works on your system.

https://blogs.oracle.com/bonwick/entry/zfs_dedup
realfolkblues12
 
Posts: 34
Joined: Thu Feb 12, 2015 10:41 pm

Re: A Mirror of RAID-Z's?

Postby ilovezfs » Sat Apr 18, 2015 12:28 pm

realfolkblues12 wrote:Dedupe can be turned on or off without issue. So you might just play with it and see how it works on your system.

https://blogs.oracle.com/bonwick/entry/zfs_dedup

Setting dedup=off only applies going forward. You have to destroy all the deduplicated data, including snapshots, to reverse the effects of having had dedup=on previously.
ilovezfs
 
Posts: 232
Joined: Thu Mar 06, 2014 7:58 am

Re: A Mirror of RAID-Z's?

Postby realfolkblues12 » Sat Apr 18, 2015 1:20 pm

ilovezfs wrote:
realfolkblues12 wrote:Dedupe can be turned on or off without issue. So you might just play with it and see how it works on your system.

https://blogs.oracle.com/bonwick/entry/zfs_dedup

Setting dedup=off only applies going forward. You have to destroy all the deduplicated data, including snapshots, to reverse the effects of having had dedup=on previously.


Thats interesting to know. Does it still affect performance? Sense all the hashing etc has already been performed I'm not sure if it would?
realfolkblues12
 
Posts: 34
Joined: Thu Feb 12, 2015 10:41 pm

Re: A Mirror of RAID-Z's?

Postby ilovezfs » Sun Apr 19, 2015 3:01 am

Yes, it still affects performance, to which I was referring when I said "reversing the effects."

Here's rottegift's comments on IRC regarding the same subject:
3:27 AM <rottegift> it’s the nuking part that gets people into trouble with deduping — zfs destroy is hugely IOPS-intensive, at least two random reads and two sync writes extra per dmu object.
3:28 AM <rottegift> so if you turn it on for a big dataset and decide it’s not what you want, you may be stuck, if the backing store is spinning disks in something like a raidz1.
3:29 AM <rottegift> additionally, while it seems like it doesn’t cost you much to have a bunch of deduped data in an old snapshot or dataset, it bites you during scans (like during resilvers), ‘cause it gets handled first, in all of its random reading glory. good way to cause a second disk to fail once you’ve replaced one that stopped working.
ilovezfs
 
Posts: 232
Joined: Thu Mar 06, 2014 7:58 am

Re: A Mirror of RAID-Z's?

Postby Haravikk » Mon Oct 19, 2015 11:20 pm

So, I finally got around to setting this up, and everything is working great except for one minor issue.

For those interested, I decided to do as realfolkblue12 recommended and paired off all drives of equal size into mirrors, and the performance and capacity are pretty good. I've then taken the resulting zpool and created a zvol with as close to 100% of the capacity as I could manage, which I've formatted as HFS+ for Time Machine to backup onto.

In the ended I opted to create only the one core storage device (from the single zvol) rather than for every single disk. Unfortunately this means that I lost the ability to enable compression/de-duplication (since core storage encryption will render these largely useless), but it's just a lot simpler to setup and manage.


Anyway, the problem I'm having is that when I import my zpool, I end up with two volumes, one for the zpool itself, and one for the file-system on my zvol; is there a way to prevent the zpool "volume" from appearing?

Put another way, my setup looks like:

MyPool (9tb) -> /Volumes/MyPool
MyPool/Backup (~9tb) -> /Volumes/Backup

How do I import MyPool, without ending up with a mounted device under /Volumes/MyPool? Since I probably won't be using snapshots I don't think I have any need for it anyway, plus all the usual methods for hiding it in the Finder don't seem to work properly, so I end up with this redundant device in my sidebar that I then have to eject; I could do that with a script I suppose, but I'm just wondering if there's a way to mount only MyPool/Backup as part of a zpool command?
Haravikk
 
Posts: 74
Joined: Tue Mar 17, 2015 4:52 am

Previous

Return to Absolute Beginners Section

Who is online

Users browsing this forum: No registered users and 16 guests