Expected Throughput?

Moderators: jhartley, MSR734, nola

Expected Throughput?

Post by mkush » Wed Sep 26, 2012 8:36 pm

I posted two days ago asking for opinions on my proposal to build a Mac Pro with six HDDs in a RAIDZ (or 2) pool. At that time I had not installed Zevo CE. Now I have and am doing some testing. I want to get a feel of things before I invest in an expensive Mac.

Currently, as a proof of concept, I have 2 LaCie 2big Thunderbolt enclosures daisy-chained to my Mac. The drives are four Hitachi 7K4000 4TB units. My zpool is a RAIDZ since I only have four drives.

The drives are AF, so I did use the -o ashift=12. For what it matters, the full command was:

Code: Select all
zudo zpool create -f -o ashift=12 -O casesensitivity=insensitive zp raidz /dev/disk1 /dev/disk2 /dev/disk3 /dev/disk4

By the way, any other common options I should have included?

At first, I hooked this to a Mac mini server that I have and ran this command:

Code: Select all
time dd if=/dev/zero of=/volumes/zp/test1 bs=1024000 count=10000

I've used on that on OpenIndiana previously to get a rough idea of what a disk subsystem is capable of.

So on the mini, I got maybe 180MB/s at first, and then for whatever reason after a reboot get as high as 235MB/s. Between reboots, displeased with the mini's first results, I tested on my retina MacBook Pro and got 252MB/s.

So, question is, is that about what should be expected?

More importantly, assuming I go to the Mac Pro, which will increase the number of drives to six, and have them directly attached to on-board SATA ports, but only support SATA 2, should I expect to see substantially better results? Not that those are so bad.
mkush Offline


 
Posts: 34
Joined: Tue Sep 25, 2012 4:36 pm

Re: Expected Throughput?

Post by si-ghan-bi » Thu Sep 27, 2012 4:07 am

The transfer rate for big files should increase linearly with (n-1) or (n-2) disks, but the transfer rate (and IOPS) for small files do not increase at all or just a bit with RAIDZ or Z2.
If you care about random read performance for small files, much better a mirror of 2 RAIDZ, each one with 3 disks. If you care about "big numbers" to show around (or if you really need high transfer rate for big files for video, ...) then you can use the configuration you planned with RAIDZ2.

Don't use RAIDZ with six drives!! If one disk breaks, you have to read (for example) 5x2TB, it means 10 TB to rebuild the latest 2 TB. The probability to encounter read errors (UNRECOVERABLE) is 1E-14 per bit, it means you have almost 50% probability that the rebuilding FAILS. That's a lot.
http://www.zdnet.com/blog/storage/why-r ... n-2009/162

Disclaimer: I'm not an expert but I think I read a lot about it and I think I understood how it works. If I'm wrong, please correct me with details on my mistakes.
si-ghan-bi Offline


 
Posts: 145
Joined: Sat Sep 15, 2012 5:55 am

Re: Expected Throughput?

Post by audiophil » Thu Sep 27, 2012 4:45 am

Your 180-250MB/s doesn't sound out of the realm of what I would expect from a single raidz vdev, w/ those 7k4000 drives. Those drives are individually supposed to do 160MB/s-ish, at least with big files. To oversimplify this, a single raidz vdev is going to be roughly as fast as one of the hard drives in it and will be faster with larger files.

They should be fine on a Mac pro's internal sata ports as well.
audiophil Offline


 
Posts: 15
Joined: Sat Sep 15, 2012 1:51 pm

Re: Expected Throughput?

Post by mkush » Thu Sep 27, 2012 7:58 am

si-ghan-bi wrote:If you care about random read performance for small files, much better a mirror of 2 RAIDZ, each one with 3 disks. If you care about "big numbers" to show around (or if you really need high transfer rate for big files for video, ...) then you can use the configuration you planned with RAIDZ2.

I mostly care about big files (raw photos and HD video), so sounds like the Z2 should be a good choice.
si-ghan-bi wrote:Don't use RAIDZ with six drives!! If one disk breaks, you have to read (for example) 5x2TB, it means 10 TB to rebuild the latest 2 TB. The probability to encounter read errors (UNRECOVERABLE) is 1E-14 per bit, it means you have almost 50% probability that the rebuilding FAILS. That's a lot.
http://www.zdnet.com/blog/storage/why-r ... n-2009/162

Well, that's a good caution. You suggest I use Z2 then, correct? That was my likely direction.
mkush Offline


 
Posts: 34
Joined: Tue Sep 25, 2012 4:36 pm

Re: Expected Throughput?

Post by mkush » Thu Sep 27, 2012 8:00 am

audiophil wrote:Those drives are individually supposed to do 160MB/s-ish, at least with big files.

I can confirm they do do around 160MB/s individually. Nice drives, I really like them.
mkush Offline


 
Posts: 34
Joined: Tue Sep 25, 2012 4:36 pm

Re: Expected Throughput?

Post by si-ghan-bi » Thu Sep 27, 2012 10:08 am

Sorry for the mistake, I mixed mirror and stripe and their properties. Of course making a mirror of two RAIDZ would give you only 40% of the total space, it's nonsense. Either stripe or RAIDZ2, but the latter is a good choice.
si-ghan-bi Offline


 
Posts: 145
Joined: Sat Sep 15, 2012 5:55 am


Return to General Discussion

Who is online

Users browsing this forum: ilovezfs and 1 guest

cron