Advise to create a Mac ZFS server

[Jean-Claude2]

@Shargo,
thank you having answered to me rapidly, I saw it only today.
I don't care about boot drive, I care about my datas. Can I have an APFS boot drive, passthrough SATA/SAS contrôler and nvme or connect HDD directly to VM for datas on ZFS ?

[Sharko]

I think that would work fine, since you would be giving ZFS control of the hardware through the pass-through.

[tangles]

I strongly urge you to consider TrueNAS Scale (not Core) on those cMPros. Either that or see if you can Hypervisor them. I definitely would't run any VMs on top of macOS for server purposes.
As long as you can install TNS using native UEFI and not have the mac emulate X86 BIOS (CSM?) then it'll work okay and do the job for a very small team.
Using macOS for a server is going to bight you eventually. It's just not made for it anymore and Apple put any Server/NAS options to bed when they pivoted and focused on services using iCloud.
If Oracle hadn't have bought Sun just after Apple announced they'd adopt ZFS, then we'd have a very different Apple today when it comes to filesystems.

If you can't get TNS installed, you can always opt to use just Linux/BSD to get ZFS goodness and go down that path. (Or gut the cMPro and buy and old PC mobo that uses the Xeon CPU and build something inside the case as a fun project).

I currently use TNS and push iSCSI to the M2 Mini in the lounge via 10Gbps for music and photo libraries.
Apple does some funky stuff with these libraries on top of APFS that cannot be replicated easy with ZFS. ← I persevered for 6 months and finally gave up and moved the libraries off SMB to iSCSI.
Using TNS will solve all your network share permissions too. Install ZeroTier/TailScale on it also and the users will be able to access it anywhere in the world.

I have about 10 cheese grater cases that I just don't know what to do with. They're such beautiful engineering examples that I can't bear to eWaste them. I have used two cases in the paste to make a Hacintosh years ago. I put the front of the 2nd case onto the rear of the 1st case to avoid having to deal with the original cMPro motherboard port holes getting in the way. Doing a build this way provided a blank canvas for the PC motherboard's ports as you only have to deal with the power button and the optical drive slot on the rear.

Anyway, digressing now. I don't think the Xeon CPUs will enable you to run VMs inside TNS either as I believe they're missing some future extensions Intel released that TNS now requires. ← just so you know.

It's a shame because they're such well made machines.

[Jean-Claude2]

@Tangles,

Thank you for your answer, saw it only now.
On May, the 17th you told that Trueness scale (being linux deviated) will likely be more attractive from a Firmware-recognised booting point of view on Apple hardware than Truenas Core (being BSD dérivative). As Mac OS is BSD dérivated, it should be the opposite, shouldn't be ?

My idea was using a type 1 hypervisor, you don't agree either ?

Cheers

[tangles]

Hi Jean-Claude,

Your point is a valid one.

I've found that getting your desired OS to boot with the specified OS boot loader is the most challenging. Linux can use GRUB (but also has it's own now) and I can't even remember what BSD flavours uses now. ← was always more of an issue which is what my last post was primarily based on.

You'll definitely be able to natively install a flavour of Linux of BSD from around and up to 2018, but anything more modern than that I just don't know anymore. ← i.e. I've retired all my cheese-graters now and am about to do the same to the last trashcans I have in use. i.e. pull out the CPU/RAM and use in some generic mobo.

I can still remember having issues with Chelsio 10Gb Ethernet PCIe3 adapters being used with MPros given they're only PCIe2 slots and this WAS with macOS.

Unless you have some legacy macOS software you want/need to run, then this is really the only reason why I'd still have a MacPro on my VLANs running "whatever" macOS.

Take the plunge, grab some old PC that's suitable as a Server, learn as much as you can/need to for your environment (assuming your personality is of this nature) and have some fun on the way.
If your personality is not exactly thinking this is fun, then I guess it's time to look at something like a Mac Mini as a replacement and call it a a day.

Probably not what you wanted to hear, and I'm thinking you've found some solution by now anyway.

Cheers,

[Jean-Claude2]

@tangles @others,

Thank you for your answer, tangles. not already used my Chelsio 10G adapters with 2019 C MPRO., but I think they are older than yours. If 2019 CMPRO is costly and not up to date, can I buy a 280$ 2010 C MPRO for using 3 nvme SSD and O3X ? shall I be able to connect from recent MacBook Pro to 2010 C MAC PRO using remote desktop ? Should I use high Sierra on 2010 CMPRO and what release of 03X ?
With 2010 CPRO "only" 8 cores but only 160w needed by the CPU.
With 212 CPRO 12 Cores but 190w needed by the CPU.
Wich is best for using it as timemachine server ?

best regards.

[Bottacco]

Hi, I come late to the party but let me chime in.

If you really need performance and support for a heavy production environment, go the TrueNAS Scale route as @tangles suggests, but don't even bother with a cMP. There are many used Dell or Gigabyte servers you can buy that will work much better… but if you are doing it as home lab project or for a small workgroup/company, it is completely feasible to use cMPs.

My most beloved project is a cMP 5,1 12core with 96GBs or RAM with OpenCore Legacy Patcher to run Monterey, a Sonnet McFiver card for 10GB ethernet and two Samsung 990 Pro NVMe SSDs (I now, they are overkill, but the client always has what the client wants ) and an ATTO ExpressSAS PCIe HBA card connected to an external JBOD with 24x4TB HDDs configured with OpenZFS on OS X (4 RAIDZ2 vdevs), with L2ARC on the mirrored NVMe SSDs. It is working so well for a team of ten people that we are looking to upgrade the machine to 128GBs of RAM (the maximum) and try deduplication… I know, everybody tells me not to do it, but as I am going to have a duplicated setup like that one, I want to try the limits and even configure a metadata vdev and slog vdev on a 4 NVMe PCIe card (non bifurcation model) and see if it holds. Everything backs up to another two servers via send and receive and it is incredible to see backups (after the first one that we do locally and then take the backup servers to their final destination) happen in a matter of minutes across town.

At this point this project is just for the sake of trying, because if you take into account the price of the PCIe cards, NVMe SSDs, etc… you better go an buy a modern Intel/AMD rack mounted server with everything enclosed (right now we are using decommissioned SAS 6Gb JBOD boxes, so that part is really cheap). If the client lets me I would like to post some videos in Youtube explaining the project and showing it step by step.

If we finally decide to change solution then the magic of ZFS comes into play again: take the SAS JBOD boxes, connect them to a TNS, Linux or BSD server, import the pools and keep going on like nothing happened.

There are some 2019 MPs appearing in the used market, but they are still 1500€ for basic models. If they come down to 1000€ the client has allowed me to buy a couple of them and see if the performance justifies changing all the cMPs. I will make my best to show the improvement

Once those cMPs die because they brake, or probably because there will be a solution that doesn't chew so much energy and it is not justifiable the power consumption versus the performance, we plan to use the cMP cases to create some benches to sit on, so we can see them everyday and still admire their great design and usefulness over such a long period of time. Long life to cMPs

[Bottacco]

Thank you for your answer, tangles. not already used my Chelsio 10G adapters with 2019 C MPRO., but I think they are older than yours. If 2019 CMPRO is costly and not up to date, can I buy a 280$ 2010 C MPRO for using 3 nvme SSD and O3X ? shall I be able to connect from recent MacBook Pro to 2010 C MAC PRO using remote desktop ? Should I use high Sierra on 2010 CMPRO and what release of 03X ?
With 2010 CPRO "only" 8 cores but only 160w needed by the CPU.
With 212 CPRO 12 Cores but 190w needed by the CPU.

If you don't want to spend money just use the 2010 cMP with High Sierra (this way you don't need to change the graphics card for a Metal compatible one) and use the latest stable version of OpenZFS that is available at that point. I am currently running 2.2.2 but migrating to 2.2.3 when possible.
If power consumption worries you a lot, go for the 8 core model, because if you are the only one that is going to use the cMP you will not see the difference in performance if you don't use encryption (LZ4 compression goes almost unnoticed) and it will be better to have more RAM than not a better CPU.

Wich is best for using it as timemachine server ?

Really, any of them. TimeMachine is a process that takes time but it is not resource intensive on the server. As I said, have more RAM. If you want to get fancy you can start spending money as we did in the project with NVMes, 10Gbit ethernet cards, etc… but for TimeMachine you won't notice the difference.