Advise to create a Mac ZFS server

New to OpenZFS on OS X (Or ZFS in general)? Ask your questions here!

Advise to create a Mac ZFS server

Postby Bottacco » Thu Mar 16, 2023 2:15 pm

Hi, I have two cMP 5.1 2010 with main disk SSD, 16GBs or RAM each and a couple of Areca SAS RAID cards (but can configure them as pure JBOD passthrough) connecting a couple of SAS 6G Stardom 8 bay JBOD enclosures. Both machines run Mac OS High Sierra with all the updates.

One machine has been the main server where 15 users connect via SMB to copy data (they don't work off the server via network) from modern Macs running a variety of Catalina, BigSur and Monterey. After some files that have reside in that server for years have appeared corrupted I have decided to go ZFS for this server, while the other is still configured with RAID6 with JHFS+ acting as a backup server where the main server and users make automatic backups with Carbon Copy Cloner.

On both machines the only things installed are the operating system, Carbon Copy Cloner (CCC), the Areca PCIe cards with their drives and nothing else. They will be used only to share files and no other software will be running besides CCC and ZFS.

My idea is to configure the main server with ZFS, share three datasets over SMB and backup those via CCC to the backup server with the RAID6 and JHFS+ volumes until I have a stable configuration in the main server so I can go all ZFS on both and backup vía SEND/ssh RECEIVE to the backup server. One of those will have BackBlaze software to make a second backup to their backup service offsite, but I still don't have decided which will make the BackBlaze backup to the cloud (it will depend if there are any incompatibilities with the ZFS implementation).

I have installed OpenZFS on OS X 2.1.6 and created the ZPOOL in the main machine with three datasets. I used the main options that appear in this site to create de ZPOOL (sudo zpool create -f -o ashift=12 -O compression=lz4 -O casesensitivity=insensitive -O normalization=formD bigdata raidz1 disk1 disk2 disk3 disk4) but when I receive a batch of 8 new disks for that enclosure I will create a raidz2 and move all the info to the new zpool.

I have this running, I have selected the three datasets in the Finder and changed privileges so everyone can read and write for the time being (although this small company have it shared that way so any employee enters as guest… they had problems with permissions on files and that was their solution). I will change that in the future but right now I don't want to mess with permissions. I am copying 14TBs of files to this new zpool and everything seems to go fine, divided in those three datasets that I created so I can snapshot each one individually.

… and I have tones of questions, but lets start on your suggestions: what would you suggest for this couple of servers? anything different to what I have chosen as options?

All the datasets are shared via the standard sharing dialog in the Sharing preference pane, only SMB and with guest access activated. My datasets are pure ZFS volumes in the Finder with no APFS or HFS emulation activated and files seem to be correct (mainly PDFs, Photoshop, Illustrator, InDesign, Word, Excel, PowerPoint, zip files and the like) when copied and as there won't be any Photos library or iTunes/Music libraries and haven't bothered to activate HFS+/APFS emulation. Can that be a problem?

Lets start with this and I will share my other questions. Thanks in advance to everybody for all the information here in the forums and for this project. I wish Apple would support ZFS but what has been achieved by Lundman and the rest of past and present contributors is incredible. Thank you.
Bottacco
 
Posts: 10
Joined: Thu Feb 09, 2023 1:27 pm

Re: Advise to create a Mac ZFS server

Postby Bottacco » Mon Mar 20, 2023 12:49 am

Ok, I see there is not much traffic in this forum. I went and started everything and after two days of making different tests I haver more questions than results.

Carbon Copy Cloner doesn't work. I select one of the datasets as the source of a task and when starting the cloning process it gives an error stating "Please attach the source" because the source could not be found. The dataset is mounted in the desktop and the choosing dialog in CCC sees it and I can choose it, but the cloning process aborts right at the beginning.

I think I am going to change my initial setup and leave the RAID6 as the online working set and create a ZFS pool in the second machine as a backup, so CCC can clone from the RAID to the ZFS volume.

Maybe I should have created a ZVOL o use the "mimic.hfs" trick, but I run out of time making tests. Apart from CCC not working everything else was going quite smoothly and I like what ZFS brings to the table… but I have so many questions!!!
Bottacco
 
Posts: 10
Joined: Thu Feb 09, 2023 1:27 pm

Re: Advise to create a Mac ZFS server

Postby Bottacco » Mon Mar 20, 2023 10:26 am

The Backblaze menubar utility doesn't work either. It shows the datasets in the list, but they are greyed out and not selectable. Maybe a ZVOL would have been a better option, but I have the doubt of the disadvantages of creating a ZVOL instead of native ZFS dataset.

Sorry @lundman, could you explain how the ZFS extension presents the mounts to the system and why they are not recognized by utilities like Carbon Copy Cloner or Backblaze backup? What would be the best option for making a backup of the datasets that is not the snapshot/send/receive mechanism? Rsync maybe?

Thanks a lot and sorry for the basic questions.
Bottacco
 
Posts: 10
Joined: Thu Feb 09, 2023 1:27 pm

Re: Advise to create a Mac ZFS server

Postby FadingIntoBlue » Mon Mar 20, 2023 4:23 pm

I have a couple of large pools which I backup to a Synology NAS using GoodSync, and also to Backblaze. They are both seen as a backup target, or at least the zfs filesystems are.

Both have the following settings, which are what I think enables the backups to work:

sudo zfs set com.apple.ignoreowner=on <poolname>
sudo zfs set com.apple.mimic=hfs <poolname>

You may want to experiment with not using the ignoreowner setting, if you have sensitivity around different users and file access. I am more concerned with accessibility and organisation in my single user setup.

Hope that helps
FadingIntoBlue
 
Posts: 106
Joined: Tue May 27, 2014 12:25 am

Re: Advise to create a Mac ZFS server

Postby Bottacco » Tue Mar 21, 2023 10:05 am

Thanks a lot for the advice @FadingIntoBlue. I just had the zpool and datasets created as:

/Volumes/bigdata/ ---> This is the zpool
/Volumes/bigdata/dataset1
/Volumes/bigdata/dataset2
/Volumes/bigdata/dataset3

They all appeared mounted in Finder as ejectable volumes and they appeared in the Backblaze menubar utility but I could not select them. I will create a new test machine in a couple of days and I will try what you said because I had to put in production the MacPros and I needed to have backups, so I could not wait to experiment anymore (14TBs of data that could not be lost)

I did know about the com.apple.mimic=hfs (or apfs) but not about the ignoreowner property, so I will try them and see what comes out.

Thanks a lot for your help and I hope that this forum gets more traction. Cheers from Spain
Bottacco
 
Posts: 10
Joined: Thu Feb 09, 2023 1:27 pm

Re: Advise to create a Mac ZFS server

Postby Sharko » Tue Mar 28, 2023 10:45 am

@Bottacco: I wonder if you might be better off to set up one or both of your Mac Pro computers with TrueNAS, rather than MacOS. Given your relatively narrow focus of acting primarily as a file server / network attached storage, I think that with TrueNAS you'll have an easier time administering the hardware, and gain access to the considerably larger user community over at https://www.truenas.com/community/ .
Sharko
 
Posts: 230
Joined: Thu May 12, 2016 12:19 pm

Re: Advise to create a Mac ZFS server

Postby tangles » Wed May 17, 2023 4:18 am

agreed...

TrueNAS Scale (being Linux derived versus TrueNAS Core being BSD) will likely be more attractive from a Firmware-recognised booting point of view on Apple Hardware.

I had all sorts of issues using Apple's Music and Photos apps from a client using SMB to access ZFS shares from a Mac Server... This was years ago and so moved away from AFP shares to using NFS from a Mac Server which worked for a good few years.
Then with APFS on the scene and Apple's apps seemingly more and more reliant on it and not working correctly without APFS underneath, I changed to using iSCSI shares via TrueNAS Scale instead of SMB hosted by macOS based Server, so that the ZFS datasets/volumes could be formatted as APFS.


I now have the best of both worlds with zero Mac client software issues and no SBoD happening anymore...


I used to use GlobalSAN on intel and now use ATTO on Apple Silicon clients to access the iSCSI shares. (both not free, but hey, how important is your data!?!) to connect the iSCSI shares for Music/Pictures datasets given that Apple's Music and Photos apps hate having to use anything else other than APFS.
I share everything else to the network via SMB. Am about to play with ElasticSearch and see if I can get Spotlight to work with my SMB shares and have them indexable by clients.
TimeMachine service on TrueNAS exists also for all the clients.

I find ZFS on macOS is perfectly fine when used in a single Mac environment/machine, although it's now been years since I had Apple Music and Photos app Libraruy files hosted on ZFS datasets, so perhaps others can chime in here...
The moment you want to expand and offer ZFS datasets as network shares (with a Mac serving the datasets) to other machines on the network is when you really gotta know what you're doing. Because Lundman is a bloody legend! I persevered, swapped protocols even and finally had to move off macOS as a sever platform. The Mac platform can do the job no probs, it's just that Apple's apps now want APFS, and no amount of config tweaks seemed to appease the situation for me... 8(((

Anyway, the above is a rough timeline from 2007-2023, and the key message to point out here is that I'm still using the the same (primary) pool, same datasets and same files all this time as the disks underneath (and redundancy setups!) have come and gone, while all my data has been verified-on-disk the whole time... You gotta luv ZFS for that!

Cheers,
tangles
 
Posts: 195
Joined: Tue Jun 17, 2014 6:54 am

Re: Advise to create a Mac ZFS server

Postby Sharko » Tue Feb 06, 2024 1:04 pm

I find ZFS on macOS is perfectly fine when used in a single Mac environment/machine, although it's now been years since I had Apple Music and Photos app Library files hosted on ZFS datasets, so perhaps others can chime in here...


I'm still doing Photos and Music on top of vanilla OpenZFSonOSX 2.1.6, but you're right, it is problematic. This is under Monterey, using OpenCore on a classic Mac Pro 5,1. Vanilla ZFS datasets are usable, but just barely: I don't seem to have problems with Music (but then I don't subscribe to Apple Music, which might bring in problems), instead relying strictly on music that I've ripped from my own media. Photos is much more problematic: for about a year I've put up with it mostly working, but not completely working. The People feature seems especially troublesome: it simply would never process the photos properly to group them into various People albums. Also, I could never get iCloud Photos to work when the library was hosted on a standard dataset.

Recently I finally gave up and created a ZVOL on a separate internal disk, and then formatted it as APFS. Now Photos (and even iCloud Photos) seems to work fine, and I still retain some of the advantages of ZFS: I can replicate the Photos library off to other storage, for instance.

Thanks for the mention of Goodsync, I'm going to look that up. I have a TrueNAS core box, too, but I have just been doing manual backups to it on a monthly/when I remember schedule.
Sharko
 
Posts: 230
Joined: Thu May 12, 2016 12:19 pm

Re: Advise to create a Mac ZFS server

Postby Jean-Claude2 » Thu Feb 29, 2024 10:12 am

tangles wrote:agreed...

TrueNAS Scale (being Linux derived versus TrueNAS Core being BSD) will likely be more attractive from a Firmware-recognised booting point of view on Apple Hardware.

I had all sorts of issues using Apple's Music and Photos apps from a client using SMB to access ZFS shares from a Mac Server... This was years ago and so moved away from AFP shares to using NFS from a Mac Server which worked for a good few years.
Then with APFS on the scene and Apple's apps seemingly more and more reliant on it and not working correctly without APFS underneath, I changed to using iSCSI shares via TrueNAS Scale instead of SMB hosted by macOS based Server, so that the ZFS datasets/volumes could be formatted as APFS.


I now have the best of both worlds with zero Mac client software issues and no SBoD happening anymore...


I used to use GlobalSAN on intel and now use ATTO on Apple Silicon clients to access the iSCSI shares. (both not free, but hey, how important is your data!?!) to connect the iSCSI shares for Music/Pictures datasets given that Apple's Music and Photos apps hate having to use anything else other than APFS.
I share everything else to the network via SMB. Am about to play with ElasticSearch and see if I can get Spotlight to work with my SMB shares and have them indexable by clients.
TimeMachine service on TrueNAS exists also for all the clients.

I find ZFS on macOS is perfectly fine when used in a single Mac environment/machine, although it's now been years since I had Apple Music and Photos app Libraruy files hosted on ZFS datasets, so perhaps others can chime in here...
The moment you want to expand and offer ZFS datasets as network shares (with a Mac serving the datasets) to other machines on the network is when you really gotta know what you're doing. Because Lundman is a bloody legend! I persevered, swapped protocols even and finally had to move off macOS as a sever platform. The Mac platform can do the job no probs, it's just that Apple's apps now want APFS, and no amount of config tweaks seemed to appease the situation for me... 8(((

Anyway, the above is a rough timeline from 2007-2023, and the key message to point out here is that I'm still using the the same (primary) pool, same datasets and same files all this time as the disks underneath (and redundancy setups!) have come and gone, while all my data has been verified-on-disk the whole time... You gotta luv ZFS for that!

Cheers,

Hello,
what do you think about using apple Remote Desktop to connect to a virtualized macos O3X ? doesn't solve your problem without using linux that I don't know ?
What do you think about using iSCSI san for Mac OS X inside a virtualized Mac O3X ?
Cheers
Jean-Claude2
 
Posts: 3
Joined: Thu Feb 29, 2024 10:00 am

Re: Advise to create a Mac ZFS server

Postby Sharko » Thu Feb 29, 2024 9:54 pm

what do you think about using apple Remote Desktop to connect to a virtualized macos O3X ? doesn't solve your problem without using linux that I don't know ?
What do you think about using iSCSI san for Mac OS X inside a virtualized Mac O3X ?


It's generally not a good idea to run ZFS on top of some other software (like a virtualized environment). This is because when ZFS isn't directly on the hardware it can't ensure that redundant data is stored on different physical devices. ZFS can live in a virtualized MacOS container, but you might as well go with a non-redundant data set up (no ZRAID, no mirroring), and that loses one of the big advantages of ZFS.
Sharko
 
Posts: 230
Joined: Thu May 12, 2016 12:19 pm

Next

Return to Absolute Beginners Section

Who is online

Users browsing this forum: No registered users and 7 guests