Is anyone using 03X with usable performance, and if so, how?

All your general support questions for OpenZFS on OS X.

Is anyone using 03X with usable performance, and if so, how?

Postby DanielSmedegaardBuus » Tue Jan 28, 2020 12:10 am

I apologise for the perhaps slightly confrontative subject, but honestly this is the question that's been on my mind for quite some time now.

I've tried many times over the last handful of years to use ZFS on all of my Macs, and every time performance has been abysmal, and most often accompanied by kernel panics, system hangs and reboots galore. At one point I had to import my pool at that time as read-only between crashes just to get data off of it — importing as read/write would instacrash the Mac.

As I'm writing this, I've once more given up, and am currently moving data off of the second-to-last pool I have on the 16GB i7 Mac mini, a 2TB single-drive pool used for scratch data, such as completed torrents. Thing is, I'm not doing it with O3X — I've fired up a 4GB VM on the Mac with an Ubuntu live cd, and use rsync to move data from the pool to an HFS+ volume on the host. It's ticking along at 22 MB/s. Pretty slow for USB3, but okay for for compressed ZFS on a VM, I guess. Only thing is, the exact same operation run natively on the host using O3X gives me read speeds ranging from 500 kB/s to just over 2 MB/s. Same source, same target, same command, just not inside a VM with a single core and a quarter the memory allotted to it.

Again, I apologise for being that guy bitching over free and open software. I hate doing that, because I really do genuinely appreciate all the time put into this project, and despite all my grievances I really am grateful for any attempt at improving and porting ZFS everywhere.

It's just... I'm at that point. It cannot just be me. Seriously. It's the same result I get every time I try to use O3X, no matter the machine or disks I've thrown at it. Over several years and over a dozen attempts, it always — and quickly — ends up functionally defective. Whether it be the unusable performance, or the kernel panics, something is seriously wrong here. Please, if I'm being ungrateful and bitchy, I apologise. I don't want to sob, whine, yell, and bang the door. I'd much prefer for this software to actually work.

It's just that at this point, I really want to know if it actually works — truly — for any one of you? Can you truthfully use O3X on a serious daily basis as a storage driver, besides running tests or occasionally doing backups or something like that. And if so, how? Because all my attempts over these years have been unfruitful.

I still have one RAID-Z pool that I'd like to keep running on this mini, for local backups. It doesn't have to perform well (and by golly, it doesn't), just as long as it doesn't crash the machine, and everything's been very stable on this 16GB mini with no crashes yet, even if ZFS disrespects my ARC limits by a factor of five. But it'd be awesome if it could perform even at a quarter of the speed of ZFS running in a VM. That would make it able to achieve read speeds to match my internet upload and thus backup speed limit to the cloud.
DanielSmedegaardBuus
 
Posts: 38
Joined: Thu Aug 28, 2014 11:00 pm

Re: Is anyone using 03X with usable performance, and if so,

Postby nodarkthings » Tue Jan 28, 2020 2:31 am

Hi! I don't use raidz but I've been using ZFS for 5 years now (a few pools and datasets, some containing sparsebundles to work around some issues with the Finder (one of them for Time Machine), one dataset containing VMs) — the total is not even 1Tb.
I don't even have an entire drive dedicated to ZFS, only slices.
It's perfectly usable, never had a crash, AFAIR. The feature I like most is Mountpoints, it allows to work around some issues in MacOS (such as moving Office 2016 users to a different place).
I can't say what's wrong with your setup, but I imagine you triple-checked your procedure when creating pools and followed the instructions in the wiki to the letter? Checked messages in the Console and Terminal?
I hope someone more savvy will help you.
nodarkthings
 
Posts: 174
Joined: Mon Jan 26, 2015 10:32 am

Re: Is anyone using 03X with usable performance, and if so,

Postby Sharko » Tue Jan 28, 2020 5:41 pm

Coming up on three years as my daily driver filesystem for all my user data, both in Mac Pro with ECC memory and a MacBook Air without ECC. I haven't used RAIDZ on my O3X backup pools, preferring to run a two disk mirror, and a couple of single disk pools. I've run into, ahem, quirks, here and there, documented in my posts to the forum here, but no show-stoppers and nothing that forced me to rebuild from a backup. My main data pool is on SSD; I do notice that it is/can be slow for certain operations (deletions mostly), but not so bad that I'd give up the data security. I've always been cautious about upgrading pools, so perhaps that is relevant. I did have some kernel panics back in Yosemite / o3x 1.5.2 era, but haven't had any in quite a while (full disclosure: did have a case a week ago where the machine became unresponsive in one application, then spread to other applications until I finally had to do the 8 second power button shutdown - but that's unusual for me). All the usual Apple first-party apps work for me, though I don't run Time Machine.

Edit: should add that I use a variety of partitioning and encryption schemes. Some of my disks are still FileVault encrypted slices, some other backups are ZFS-native encryption where ZFS owns the whole disk.
Sharko
 
Posts: 230
Joined: Thu May 12, 2016 12:19 pm

Re: Is anyone using 03X with usable performance, and if so,

Postby DanielSmedegaardBuus » Wed Jan 29, 2020 12:17 am

Thank you all for the input.

It's quite interesting, though probably not too surprising, that my experiences don't seem to be shared by anyone else. If they were, why would anyone be using it? :D

It does make me rethink what the heck is going on here, because I'm not lying when I describe my endless pains with O3X. I've used it with two different Airs, one 2013 and one 2015 edition, a mini 2015 and a mini Server 2011 (the one currently running, from which the behaviour described in the OP stems), one Hackintosh, and my awful 2016 MBP.

The fact that I can so easily demonstrate the abysmal performance difference by attaching my 2TB USB drive to a VM and go from <=2 MB/s read speeds to >20 MB/s read speeds does make me wonder about one thing: Have I ever run O3X on a drive that wasn't attached via USB? I don't think so. I'm not 100% certain, and I do remember trying to get a ZFS boot drive working on a Hackintosh, but I didn't succeed. There's a very high probability that all of my experiences involve USB storage.

I'm wondering now, reading about your experiences (does every one of you use SATA storage?), if there's some issue with 03X and USB specifically. Clearly, it's not a physical issue with the ports, as demonstrated by my example of just re-assigning the 2TB USB drive to a VM while it's attached to the same physical port and increasing read performance tenfold inside the Linux VM. Oh, and in case you're thinking "hang on — doesn't the 2011 server only have USB 2.0 ports," then yes, that's correct, but mine also has a Belkin Thunderbolt dock with USB 3.0 ports.

This machine was previously used as a workstation, and I'm planning on adding two 500 GB spindles to the internal SATA ports, so I think I'll ZFS-ify that storage when that happens and see what kind of performance I'll be getting when USB is not in the picture.

Thanks again for your input, and please do share if you have any USB-related experiences as well :)
DanielSmedegaardBuus
 
Posts: 38
Joined: Thu Aug 28, 2014 11:00 pm

Re: Is anyone using 03X with usable performance, and if so,

Postby nodarkthings » Wed Jan 29, 2020 3:00 am

When I started using ZFS there were known issues with USB.
I even had one drive and a USB key dead after my first trials, but I believe that was rather because they were old and/or of poor quality.
Since then, I've (carefully) always ZFSed my internal SATAs only... :mrgreen:
I believe from what I've read in the forum that some people use USB now, but I think you're right and that is probably where your issue lies.
Maybe you should open a new thread pointing at USB, describing precisely your setup and software/hardware?
nodarkthings
 
Posts: 174
Joined: Mon Jan 26, 2015 10:32 am

Re: Is anyone using 03X with usable performance, and if so,

Postby Sharko » Wed Jan 29, 2020 9:27 am

Interesting thoughts regarding USB. For much of my time with ZFS my connections have either been internal (SATA or NVMe), or eSATA (for my mirrored backup). About a year ago I bought a pair of 2TB USB3 portable drives, and I've been using those as rotating off-site backups. I do recall that the initial replication was pretty slow on those, though I didn't record how slow it was. Every week I do a ZFS send of a snapshot of my main data, which usually involves the transfer of one to five gigabytes of changed data; I always bracket my send / receive command with the Unix date command before and after, and that send/receive task usually takes between two and four minutes. So if you figure it is sending somewhere around a gigabyte per minute then the transfer speed over USB3 is somewhere on the order of 15 MB/sec, not great, but not as bad as you seem to be experiencing.
Sharko
 
Posts: 230
Joined: Thu May 12, 2016 12:19 pm

Re: Is anyone using 03X with usable performance, and if so,

Postby tangles » Wed Jan 29, 2020 2:25 pm

I stopped using ZFS on macOS. It was just too slow when writing to disks.

My experience/testing with macOS ZFS ranges from SSDs, PCI Flash Storage, Apple Xraids and up to 12TB sized disks.

Any combo I tried to get around the poor write speed didn't help me.

I've moved to FreeNAS for now (on the same hardware as above) and I'm now saturating my 10GB LAN.

It's a shame, cos Like you, I wanted to use it but it's just not happening for me.

I've posted in the past about macOS ZFS speeds with some interesting numbers.
viewtopic.php?f=26&t=3150&p=8763&hilit=tangles#p8763

I know Lundy is orsm and does an amazing job given it's primarily him all this time. Can't argue with that.
tangles
 
Posts: 195
Joined: Tue Jun 17, 2014 6:54 am

Re: Is anyone using 03X with usable performance, and if so,

Postby FadingIntoBlue » Thu Jan 30, 2020 11:09 pm

I've been using ZFS for several years; always found USB difficult from a consistency point of view, failed reads and writes, lock up etc. The two things which have helped me get decent performance are maximising ram and using Thunderbolt

95GB sparsebundle - zpool iostat NewPool 5
HW TB4 mirror <-> mini18 64GB
NewPool to Downloads Read - peak 94.1M
Downloads to NewPool Write - peak 517M
3.2GB Movie - zpool iostat NewPool 5
NewPool to Downloads Read - peak 257M
Downloads to NewPool Write - peak 372M

So that is a 2018 Mac mini 64GB, Thunderbolt 3 through adapter to Thunderbolt 2 OWC 4 drive case, 2x2 mirror 8:8 Tb and 6:8 Tb

No complaints!
My use of ZFS is driven by file preservation and integrity, rather than super fast performance. I haven't lost a file at all using ZFS.
FadingIntoBlue
 
Posts: 106
Joined: Tue May 27, 2014 12:25 am

Re: Is anyone using 03X with usable performance, and if so,

Postby FadingIntoBlue » Fri Jan 31, 2020 1:50 am

Just a couple of additions

1: it is the fastest mini 3.2 GHz 6-Core Intel Core i7

2: Comparison with 8GB ram

90GB sparsebundle - zpool iostat NewPool 5
HW TB4 mirror <-> mini18 8GB
NewPool to Downloads Read - peak 49.8M
2.2GB Movie - zpool iostat NewPool 5
NewPool to Downloads Read - peak 89.5M
Downloads to NewPool Write - peak 141M
FadingIntoBlue
 
Posts: 106
Joined: Tue May 27, 2014 12:25 am

Re: Is anyone using 03X with usable performance, and if so,

Postby DanielSmedegaardBuus » Sun Feb 02, 2020 11:21 pm

Okay, so I got around to reconfiguring the mini with some internal storage, made a pool on one SATA-attached spindle, and have been (trying to) use it since the day before yesterday. It's not as useless as with USB, but read speeds are in the 5-10-MB/s area, so still useless. I'm chugging this for good. I guess if I want a pool on a Mac in the future, I'll just create one in a VM and get 10x the performance by sharing it. Perhaps as an iSCSI target.

Sorry for the saltiness. Over and out.
DanielSmedegaardBuus
 
Posts: 38
Joined: Thu Aug 28, 2014 11:00 pm

Next

Return to General Help

Who is online

Users browsing this forum: No registered users and 31 guests

cron