ZFS on official RAID device (Promise Pegasus 4 drives)

Here you can discuss every aspect of OpenZFS on OS X. Note: not for support requests!

ZFS on official RAID device (Promise Pegasus 4 drives)

Postby storkinsj » Thu Jan 14, 2021 7:23 pm

Hi all-
I recently purchased a RAID to go in my new mac pro (not here yet). I want to use it from mac and windows on bootcamp, so I started to think about cross platform filesystems. ZFS came up.

I may just use some commercially supported NTFS solution because support for ZFS on Windows may not be as well supported. But for now I have this question: What is the best way to run ZFS on a RAID controller?

According to the manual, the controller supports RAID0, RAID1, RAID 5, RAID 6, RAID10 on the 4 disk array.
ZFS does have its own raidz support too. My guess is raiding ZFS on a RAID 5 array is going to needlessly hurt performance. I should probably choose one or the other for "redundancy" (other than the journaling redundancy).

Open to any and all comments.
storkinsj
 
Posts: 2
Joined: Thu Jan 14, 2021 7:09 pm

Re: ZFS on official RAID device (Promise Pegasus 4 drives)

Postby tangles » Sat Jan 16, 2021 2:47 pm

Ideally, you want to choose JBOD (Just a Bunch Of Disks).

If you don't have that option, then I suggest you choose RAID0, but have only the one disk in the RAID0 array, and create 4 of them.

This still may not be ideal as ZFS may not be talking directly to the disk. It all depends on the Pegasus software.

I did find the following at https://www.promise.com/Products/Pegasus/R4i-J2i/FAQ
Can the Pegasus R4i & J2i operate without using RAID? In other words, can each of the 2 or 4 drives operate independently?
Yes, Pegasus HDD can be setup in pass-thru mode for this purpose


So that looks promising for you.

Also found:
Image

So hopefully the current version you're using can still configure each disk to moot the RAID hardware.
tangles
 
Posts: 195
Joined: Tue Jun 17, 2014 6:54 am

Re: ZFS on official RAID device (Promise Pegasus 4 drives)

Postby storkinsj » Sun Jan 17, 2021 4:46 pm

Thank you for your incredibly awesome help.

It doesn't explain the why's , but the what's are exactly what I have heard (use JBOD) elsewhere.

It sounds like you are firmly convinced the OpenZFS redundancy system exceeds RAID 5 or 6 and is more performant.

-s
storkinsj
 
Posts: 2
Joined: Thu Jan 14, 2021 7:09 pm

Re: ZFS on official RAID device (Promise Pegasus 4 drives)

Postby tangles » Fri Jan 22, 2021 4:10 pm

mmm,

Well you don't use ZFS on macOS to get performance. Not having a go at the macOS devs, they are champions!

It's just that I'm not aware of too many code-optimisations focused around speed have occurred for macOS. (I could be wrong, so apologies if I'm mis-typing)

People tend to use ZFS filesystem on macOS because they care more about the integrity of their data than speed.

Remember that ZFS appeared on macOS when HFS was the default—a 40 year old codebase initially designed for floppy discs!

APFS isn't much better when it comes to data integrity because it too does not calculate checksums on datablocks. (it does for metadata datablocks only from memory)

Here's a fun exercise in frustration:
If you want to share data on a single machine via multiple OSes and have performance, I suspect running a TrueCORE VM to drive your ZFS pool via macOS or Windows would yield better performance.
Far more complex to do, but if it's integrity and speed you're after.
If you hosted the VM file on a exFAT partition using VirtualBox and gave it raw access to the ZFS disks, you might even be able to run/use the same TrueCORE VM via macOS and Windows or any other OS you choose, as long as VirtualBox has been compiled for the native OS. This would be neat!
Kind've like having your own locally resident NAS hiding inside your machine. vSphere comes to mind also here, it used to run natively on Apple Intel too a while back but am not sure if it does now considering Apple tried to kill off PCIe slots (and failed).

Clearly the easier option would be to use another old clunker PC, but with a 10GB ethernet card installed into one of its slots running TrueCORE and access it directly from your Mac with whatever OS you're running on your Mac at that time. This is what I do here at home with 10GB ethernet and the speed is mental. Plenty fast enough for my home needs for years to come.
There's a guy called Art of Server on ebay who I've purchased from multiple times in the past for storage focused items. He can help you attain ex-enterprise hardware (i.e. speed!) that are guaranteed to work with ZFS/TrueCORE. Intel or Chelsio 10GB ethernet adapters work well. You could go the cheaper SFP (fibre) option too if you were only wiring up your MacPro and the NAS box directly to each other for speed purposes, and then use gigabit ethernet (copper) back into your 1GB copper switch for your remaining/other computer needs.

Maybe it's time again to virtualise ZFS with the latest versions of macOS, Ubuntu, FreeBSD, TrueCORE and Win10, with multiple virtual disks and test the performance of mirror, stripe, raidz1 and raidz2 to see how the macOS compares.
This was done some time back and is still on the wiki performance page, although I suspect things have changed since then.
Image
I do remember ZFS on Mac around version 1.2 or 1.3 was very quick. I can't exactly remember the version number but I definitely still remember installing it onto a x6 drive pool and the I/O on the disks were going bezerk compared to the previous versions.
I don't know what changed in the code base after and around then, but for me, the drop in speed was especially noticeable when using ≥ 5 physical disks that I chose to move off macOS/ZFS. There are recent posts commenting on poor performance too by the looks.

Food for thought.
tangles
 
Posts: 195
Joined: Tue Jun 17, 2014 6:54 am

Re: ZFS on official RAID device (Promise Pegasus 4 drives)

Postby danielkyler » Wed Jan 10, 2024 1:13 am

tangles wrote:mmm,

Well you don't use ZFS on macOS to get performance. Not having a go at the macOS devs, they are champions!

It's just that I'm not aware of too many code-optimisations focused around speed have occurred for macOS. (I could be wrong, so apologies if I'm mis-typing)

People tend to use ZFS filesystem on macOS because they care more about the integrity of their data than speed.

Remember that ZFS appeared on macOS when HFS was the default—a 40 year old codebase initially designed for floppy discs!

APFS isn't much better when it comes to data integrity because it too does not calculate checksums on datablocks. (it does for metadata datablocks only from memory)

Here's a fun exercise in frustration:
If you want to share data on a single machine via multiple OSes and have performance, I suspect running a TrueCORE VM to drive your ZFS pool via macOS or Windows would yield better performance.
Far more complex to do, but if it's integrity and speed you're after.
If you hosted the VM file on a exFAT partition using VirtualBox and gave it raw access to the ZFS disks, you might even be able to run/use the same TrueCORE VM via macOS and Windows or any other OS you choose, as long as VirtualBox has been compiled for the native OS. This would be neat!
Kind've like having your own locally resident NAS hiding inside your machine. vSphere comes to mind also here, it used to run natively on Apple Intel too a while back but am not sure if it does now considering Apple tried to kill off PCIe slots (and failed).

Clearly the easier option would be to use another old clunker PC, but with a 10GB ethernet card installed into one of its slots running TrueCORE and access it directly from your Mac with whatever OS you're running on your Mac at that time. This is what I do here at home with 10GB ethernet and the speed is mental. Plenty fast enough for my home needs for years to come.
There's a guy called Art of Server on ebay who I've purchased from multiple times in the past for storage focused items. He can help you attain ex-enterprise hardware (i.e. speed!) that are guaranteed to work with ZFS/TrueCORE. Intel or Chelsio 10GB ethernet adapters work well. You could go the cheaper SFP (fibre) option too if you were only wiring up your MacPro and the NAS box directly to each other for speed purposes, and then use gigabit ethernet (copper) back into your 1GB copper switch for your remaining/other computer needs.

Maybe it's time again to virtualise ZFS with the latest versions of macOS, Ubuntu, FreeBSD, TrueCORE and Win10, with multiple virtual disks and test the performance of mirror, stripe, raidz1 and raidz2 to see how the macOS compares.
This was done some time back and is still on the wiki performance page, although I suspect things have changed since then.
Image
I do remember ZFS on Mac around version 1.2 or 1.3 was very quick. I can't exactly remember the version number but I definitely still remember installing it onto a x6 drive pool and the I/O on the disks were going bezerk compared to the previous versions.
I don't know what changed in the code base after and around then, but for me, the drop in speed was especially noticeable when using ≥ 5 physical disks that I chose to move off macOS/ZFS. There are recent posts commenting on poor performance too by the looks.

Food for thought.

I've encountered a similar situation, and I can attest to its consistent stability.
What is my IQ? I take iq test free.
danielkyler
 
Posts: 1
Joined: Wed Jan 10, 2024 1:04 am

Re: ZFS on official RAID device (Promise Pegasus 4 drives)

Postby gea » Tue Feb 13, 2024 6:13 am

The general rule is: never use a raid below ZFS

While ZFS works on such a raid as it simply sees a basic disk, you will loose Copy on Write protection with protected atomic writes (data+metadata or raid stripes over several disks) on the underlying disks in the raid. There is also no ZFS self healing possible as ZFS can only detect checksum errors but not repair and the underlying raid can also not repair as it lacks checksum and can't even detect the problems ZFS is aware of.

Regarding performance.
Not sure how OSX mounts a ZFS pool. On Windows, the pool is currently mounted as 512B physical/logical disk with around half the performance of an ntfs disk with the correct 4k setting. So unless this is fixed or verified not to be a problem, cross performance tests are too early.
gea
 
Posts: 8
Joined: Tue Jan 23, 2024 9:56 am


Return to General Discussions

Who is online

Users browsing this forum: No registered users and 12 guests