Page 1 of 3

Back up datasets to non-ZFS volume

PostPosted: Tue Jul 24, 2018 7:38 pm
by tim.rohrer
I've created a couple of pools and datasets (Groups data and VMs) which I'm trying to incorporate into my current backup strategy. In the past, this strategy uses Time Machine and rsync to copy data to a 9T online storage set; this is formatted as HFS+. I also currently use rsync to copy files from various computers and volumes to this same online dataset. Additionally, I use rsync to copy all the non-Time Machine backups to external disks.

I discovered that Time Machine will not back up data from ZFS file systems. From my research, I understand I can create snapshots and use `zfs send/recv` to create incremental backups, but the target file systems must be ZFS. I wish to use my 9T online backup storage set as the target, so I'm trying to find a way to use this.

I've also read I create a pool in a file on a non-ZFS to receive snapshots, but something I read indicates this is not a great approach (nothing further provided). I wish to confirm, please, Is this or is it not a good approach?

If I wish to continue to use the 9T disk set, am I limited to using rsync to create copies of the pools/dataset? If so, will this include snapshots?

Thanks!

Tim

Re: Back up datasets to non-ZFS volume

PostPosted: Tue Jul 24, 2018 8:59 pm
by tim.rohrer
In some additional testing, I'm seeing now that Time Machine probably didn't work because I had mount points under /Volumes. I've created a test dataset, and I mounted it under my home directory. Adding some data, Time Machine backed it up.

So, now I'll investigate how to move a mount point.

I continue to welcome replies, especially any discussing if I'm on a bad track.

Thanks.

Re: Back up datasets to non-ZFS volume

PostPosted: Wed Jul 25, 2018 12:29 am
by Brendon
I once had a recent version of the code backing up a zfs dataset or datasets to a osx server timemachine share itself residing on zfs on the server. Got rid of the configuration as at least weekly it decided to redo the entire 100GB backup rather than just doing the deltas as it normally would. It may have been related to when I snapshotted the server side filesystem, but could never really prove it.

In the past though I had hfs being backed up with TM to a similar share on zfs without too many issues.

Re: Back up datasets to non-ZFS volume

PostPosted: Wed Jul 25, 2018 6:30 pm
by tim.rohrer
In the past though I had hfs being backed up with TM to a similar share on zfs without too many issues.


As I understand my research, you're talking about a system whereby you use a sparsebundle in the ZFS dataset to receive the TimeMachine backup. Sounds like it can work, and I can put quota limits, etc., on that dataset.

Did you also use that for network TMBs?

And, what are you doing today?

Re: Back up datasets to non-ZFS volume

PostPosted: Thu Jul 26, 2018 1:20 am
by Brendon
In both cases I used OSX Server timemachine shares.

Re: Back up datasets to non-ZFS volume

PostPosted: Thu Jul 26, 2018 9:12 am
by tim.rohrer
Thank you.

I'm going to spend some time getting my head wrapped around this. My current backup disk is a RAID5 (hardware) and I thought I read somewhere that I should not mix hardware RAID and ZFS, but I need to research that again. If I reformat the backup disk set as ZFS, I'll either have to use a RAIDZ configuration or mirrored vdevs (and lose about 3TB of storage).

Anyway, still learning what would be the best approach. I had hoped to keep the backup disk as HFS+, but since the zfs send/recv only works with zfs, and that could be a good part of my backup strategy, it might make more sense to switch it.

Re: Back up datasets to non-ZFS volume

PostPosted: Thu Jul 26, 2018 5:42 pm
by Brendon
I think there are any number of solutions that may work for you, no right or wrong answer.

Personally I was attracted the TM on ZFS because I could snapshot it, and when/if the sparsebundle got corrupted, revert to the last known good backup via snapshot recovery. In my mind it would be equally valid to have a local snapshot and use the TM backup as a mechanism of last resort in terms of recovery. Unfortunately I have some suspicion that snapshotting one of either the server or local was causing excess full backups, which in my case were very time consuming (and power consuming).

Its apparent that you are putting quite a bit of thought into this, and I see no harm in experimenting with a couple of approaches to see what works for you.

For me I was also quite happy zfs send -i the data using znapzend as well.

Sadly I no longer use a mac due to some hardware failures, and the cost of ownership exceeding my pain levels. So my current solution (which I'm reluctant to mention here), is an 6 (or 8 I forget which) bay NAS, with a 4 disk mirror, 1 disk hot spare,1 disk internal backup. It internally rsyncs (and can rsync other machines if necessary) and snapshots, and is our total data repository for the family. Next year when I get half decent broadband I will probably do some form of offsite backup. The NAS cost less than the thunderbolt DAS I had before it died.

Re: Back up datasets to non-ZFS volume

PostPosted: Fri Jul 27, 2018 8:24 pm
by tim.rohrer
Your secret is safe with me...and the 2 or 3 people reading this thread :-)

Part of my long-term strategy is to move my file server to Linux, which is one of the reasons ZFS appealed to me.

I've created a test ZVOL and formatted it to HFS (Journaled, Case-sensitive). Interestingly, it is not showing up yet as a disk for Time Machine. I'm now researching trying to figure out why.

Re: Back up datasets to non-ZFS volume

PostPosted: Sat Jul 28, 2018 11:43 am
by tiennou7
Heh, that's fun, I did exactly just that a few minutes ago, trying to find a way to have a network-served Time Machine destination backed by a ZFS volume.

Reproduction steps (this is from memory though) :
- have a pool handy
-
Code: Select all
zfs create -V $size pool/backup
I used
Code: Select all
($volume_size_in_mb * 1024 ** 3) % ($block_size_in_kb * 1024)
(the modulo ensures that the first part respects the blocksize criteria.
- formatted the new volume using Disk Utility as "HFS+ (journaled)".
- Time Machine picks up the volume on mount and asks to use it. Say "yes"
- profit!

I then created a network share on that, and allowed it as a Time Machine destination. I can confirm that it was showing up on my client machine, but I didn't try to backup to it. I did start the server's backup though, so I'll report after it's done and I made some more testing (snapshotting causing a Time Machine "reset" looks concerning, so I'll definitely test that), as well as clients backups.

HTH

Re: Back up datasets to non-ZFS volume

PostPosted: Sat Jul 28, 2018 1:03 pm
by Brendon
Server side. OSX server will host a TM backup on raw ZFS. No ZVOL needed.

Not sure if it needed mimic on or not.