External Drives?

New to OpenZFS on OS X (Or ZFS in general)? Ask your questions here!

External Drives?

Postby Tucker28 » Wed Sep 28, 2016 4:51 pm

Hi,
I have an 8-core Mac Pro that I want to install ZFS on for my file server. The FB/ECC memory on this will benefit me greatly, I think. But the Mac Pro's only have 4 HDD bays. I am planning on purchasing an SSD card to run the OS and Apps so I can save the 4 bays for ZFS. However, I also want to use RaidZ2, and with 4 bays, that means I only get half the capacity. I have a NexStar HX4R that I will use in individual mode. I have read up enough to know to never use hardware RAID, but to let ZFS manage it itself. If I could use that instead, that would give me 8 total drives, allowing me to use 75% of my storage space. I am toying with the idea of getting another NexStar, but this time, just the HX4, as it's cheaper and I won't be using the hardware RAID anyway and giving me 12 drives. I wouldn't even think of doing this if you could add HDD's to an array, but since you can't, I want to future proof the heck out of this thing.

I do know that to see the individual drives, that I would need an eSata card with multiport capabilities. My question is, would this work? Will ZFS be able to use RaidZ2 when half the drives are in the case and half external? As far as I can tell, eSata is capable of this, but I'm not 100% sure. The external enclosure does also can use USB 3.0 as well as eSata, but I think I remember reading somewhere that ZFS and USB don't get along. Is this true?

Would something else work better than this solution?

Thanks,
Scott
Tucker28
 
Posts: 5
Joined: Wed Sep 28, 2016 4:27 pm

Re: External Drives?

Postby Brendon » Sat Oct 01, 2016 12:56 pm

ZFS does not care about the nature of your drives, we do not have device specific device drivers in our code, that is all provided by the OS. If you create a pool with a bunch of internal and external drives it will work fine - unless you unplug the externals!

Note that raidz* datasets are only as fast as your slowest drive as far as I recall.

Unless someone has specifically used those enclosures, you won't get much commentary as to the validity of your design - it should all work in principle, but ZFS will very quickly tell you if your hardware is not up to scratch. Physically it looks quite neat though, and WAY cheaper than thunderbolt gear.

- Brendon
Brendon
 
Posts: 286
Joined: Thu Mar 06, 2014 12:51 pm

Re: External Drives?

Postby tangles » Mon Oct 03, 2016 7:16 pm

My mate John uses one of these just for HFS disks.
It's connected to my old Sil3132 HBA in his 2008 cMP.
PM works fine but is a tad slow.

You have 2 extra SATA ports for a total of 6 in your MacPro.

I've ripped out the PATA optical drives and placed a SSD for boot up the top, which leaves me with 5 SATA ports to play with internally...
It's a bit of a tight squeeze to thread two SATA cables up to the top area, but it's a 30min effort.

Have u considered selling your old disks and invest in larger ones so u have no external enclosure at all?
tangles
 
Posts: 195
Joined: Tue Jun 17, 2014 6:54 am

Re: External Drives?

Postby Tucker28 » Mon Oct 03, 2016 8:02 pm

I have thought about it, but as I'm going to be using RaidZ2 (and since I can't add drives to the array later), I want to future proof this as much as possible and keep my parity to storage ratio percentage at 25% or lower. Loosing 2 drives to parity means I'll need 8 total which leads me to the external bays with multi port eSata.

I lost my entire video library last year, and while in some respects I'm ok with because I can re-rip stuff when I get the storage, I also lost my home movies. And I had no backups. I still have the drive and ProSoft's Data Recovery software so when I get a drive large enough, hopefully, I'll be able to get at least the home movies transferred. If not, I'll be heading back to the VHS tapes again.

Sorry, off on a tangent there. Anyway, I need RaidZ2 + ZFS to guard against that happening again. The home movies, whether they can be salvaged or re-ripped, I'll make sure to have backed up to iCloud, but I can't back up the entire library, so I'm counting on ZFS and RaidZ2 to keep me safe. Which is also the reason I wanted the Mac Pro... ECC memory. I'm kinda paranoid now. Lol.

How did you get a SATA drive connected to a PATA interface? Even though you have it working, isn't it painfully slow? What kind of speed are you seeing on that?

What I had planned on was replacing the drive in it with an SSD for the OS and apps. 2 of the bays for part of the ZFS storage pool, plus 2 external units connected to a multi port card. That will give me a maximum of 10 external drives, plus the 2 inside. The last internal bay will be another SSD for the storage pool cache.

I know it's going to be a slow build, and I'd love to start and then expand later. But while ZFS let's you expand, it only lets you expand by adding another pool (terminology?). I can't add to an existing RaidZ2 array. So it's expand as I go which will cost me more storage space in the long run due to parity... or build it first so I save more storage for actual files, but that's longer to build because of money.

Or am I making this more complicated than it needs to be?

Sincerely,
Scott
Tucker28
 
Posts: 5
Joined: Wed Sep 28, 2016 4:27 pm

Re: External Drives?

Postby macz » Tue Oct 04, 2016 11:25 am

What you need is a backup... Not a raidz2. Or a raidz5 if it existed.. Data in a single place is vulnerable to loss... Period.

I run a stripped primary pool with no parity (but parity checking) for speed and 100% use of my online spinners.. Being zfs replicated on a weekly basis to an offline raidz. This pool can be expanded by one drive at a time if I want but I add pairs to ensure I can saturate my network and backup cabling.

When my primary reaches 6 drives ( I only by them 2 at a time as needed and they get cheaper every year) I go and get 2 of the largest available at the time and 2 more of whatever size I am using, send-receive over to the new 2 drive pool, destroy the 6 and build another 8 drive raidz which is added to the existing backup array.. You can expand a raidz, a raidz at a time

Some people prefer building arrays as stripped mirrors... So they can not only add vdevs to the pool in pairs but also expand the original drives by replacing drives with larger ones.. But it has 100% overhead and rebuilding thrashes drives.. Enterprise does it so they can keep a pool up during maintenance but I see no reason to do that at home. Also zfs suffers from fragmentation over time so a periodic rebuild from a backup cleans things up nicely on that front as well.

Bottom line.. Is there are many ways to skin the zfs cat ... But having you data in one place and one place only is not one of them... Regardless of parity..

My opinion of course!!
macz
 
Posts: 53
Joined: Wed Feb 03, 2016 4:54 am

Re: External Drives?

Postby Tucker28 » Tue Oct 04, 2016 11:33 am

I think I see what you're saying. Please correct me if I misunderstood.

You start small with both ZFS pool as well as back up.
Then when you need to increase storage, you buy two drives, one for the pool and one for the backup.
You increase the size of the RaidZ2 by 1 drive by rebuilding it with the extra drive
Then do a restore from backup
Then increase the size of the backup in the same way
Then start backup up like normal.

Is that correct?
Tucker28
 
Posts: 5
Joined: Wed Sep 28, 2016 4:27 pm

Re: External Drives?

Postby tangles » Mon Oct 17, 2016 1:29 pm

I think he just has two pools in total Tucker...

And adds 2 drives (largest available) as a mirror each time. (Backup not mentioned)

U definitely need a backup. RAIDz2 is no backup, only redundancy..

macz is also correct about fragmentation. I'm up to 5 pairs mirrored 4TB drives and I'm about to blow everything away and restore via backup.

My read speeds have dropped down to 60Mb/s over the network and yet write speeds saturate my gigabit switch... (Xeon with 24GB RAM)

ZIL & Cache made no difference as most of my data is video. It would be nice if ZFS could read % full of vdevs and auto move/distribute blocks in the background (eg when reach certain idle state) to promote maximum vdev performance.
I guess this is a low priority feature though when enterprise has copious Ram/ZIL/Cache to play with... :roll:

I'm waiting for sierra drivers to start again and get back my I/O...
tangles
 
Posts: 195
Joined: Tue Jun 17, 2014 6:54 am

Re: External Drives?

Postby Tucker28 » Mon Oct 17, 2016 5:50 pm

My server is going to hold primarily video as well, so what you are saying is that the SSD for the cache is not going to benefit me then?

I do want to do RaidZ2, but I'm thinking considering the replies that what I need to do is start out small, with a backup with large archive drives, and then slowly work my way up, from single drives, to maybe mirrors, to RaidZ1 and then finally to RaidZ2 once I have enough drives. From what you guys are telling me about fragmentation, I'm going to have to restore every now and then anyway, so that will be the perfect time to add a drive into the array and restore from backup.

I admit to being perplexed on the fragmentation. Most of my files aren't going to be modified after they go into the pool. And modification is going to be done on the system drive and then the finished product will go into the pool for reads. I'll have to research ZFS fragmentation some more.
Tucker28
 
Posts: 5
Joined: Wed Sep 28, 2016 4:27 pm

Re: External Drives?

Postby tangles » Tue Oct 18, 2016 1:30 pm

I'm the only user reading/writing to my pool over the network Tucker, which is why ZIL/Cache offered no noteable improvement for me. (I used a PCIe x4 128GB flash HBA)
Having more users would obviously utilise the ZIL/Cache.

I used to use RAIDz in the past, I.e 5 sets of RAIDz with only 3 disks in each set. The reason at the time was for better read/write speeds as the more vdevs u have, the faster yr pool will be.

I started with 250GB drives, then 1TB drives, and changed to a mirrored pool for the 4TB drives I'm using now. I still have 5 vdevs in my pool though and resilvering is a lot faster now as it only ever involves two disks instead of the whole pool.

I don't think I'd use any RAID1|2|3 these days and just stick with using mirrors.

I played around with using different default block sizes not long ago, but I didn't notice any benefits over gigabit, even with jumbo frames. I have 10GB nics (Chelseo cards) but only use them to push my backups to another pool so I suspect that's where larger block sizes may help.

Cheers
tangles
 
Posts: 195
Joined: Tue Jun 17, 2014 6:54 am

Re: External Drives?

Postby Tucker28 » Tue Oct 18, 2016 2:25 pm

Good information, thanks. I'm glad you mentioned backing up other stuff to it, as I had been considering using it for that purpose as well. I already have a Time Capsule that is enough space for my machines, but thought about running a once weekly back up to the pools or have Time Machine alternate between the Time Capsule and the pool. That's where I'll get into fragmentation most likely.
Tucker28
 
Posts: 5
Joined: Wed Sep 28, 2016 4:27 pm


Return to Absolute Beginners Section

Who is online

Users browsing this forum: No registered users and 3 guests