crashplan backups on ZFS volumes

All your general support questions for OpenZFS on OS X.

crashplan backups on ZFS volumes

Postby calbear88 » Sun Nov 26, 2017 8:29 pm

Hi, I am using a backup program called Crashplan to backup my data.

Crashplan relies on spotlight to detect new or modified files. If I create a new file on my boot drive (formatted in APFS), and I run crashplan, it will detect the new file and back it up.

If a create a new file on my ZFS formatted volume, crashplan doesn't seem to detect the new file. I have to run a "verify selection" option in crashplan in order for it detect the file and back it up. During this "verify selection" process, crashplan scans the hard drive to look for any file changes (takes about 2 hours for 2TB).

I confirmed that spotlight is set to index to ZFS volume

There seems to a known issue with spotlight indexes on zfs volume that are deleted at reboot reported at viewtopic.php?f=26&t=2826

Perhaps this is a similar issue with spotlight having problems indexing ZFS volumes?

here's some info on how crashplan and spotlight work

https://support.code42.com/CrashPlan/4/ ... _Spotlight

"The CrashPlan app real-time backup feature cannot detect any new or changed files in your file selection if Spotlight does not report the changes to the files to CrashPlan."
calbear88
 
Posts: 14
Joined: Sun Mar 08, 2015 3:01 pm

Re: crashplan backups on ZFS volumes

Postby lundman » Mon Nov 27, 2017 2:51 am

I wonder how it queries spotlight to get the items changed? Perhaps we are not setting the correct date/time somewhere
User avatar
lundman
 
Posts: 316
Joined: Thu Mar 06, 2014 2:05 pm
Location: Tokyo, Japan

Re: crashplan backups on ZFS volumes

Postby calbear88 » Fri Dec 01, 2017 8:00 pm

HI, I asked Crashplan support about how their software detects real time file changes using spotlight, and why it doesn't work on ZFS volumes. Here's the reply I received

"As configured, Spotlight generates indexes of the files on your computer. CrashPlan reads these indexes for reference when determining what has changed and what has not. While we are not able to go into technical detail as to how this indexed information is organized by Spotlight, this is what the real-time scanning of CrashPlan relies on when seeing files and changes.

In regards to the drive in question with ZFS as its file system, we would not be able to determine how your Spotlight service works with ZFS. Further research is stating that Spotlight has traditionally not worked on ZFS, although we are uncertain if this has changed. We would otherwise not be able to assist with troubleshooting Spotlight on ZFS, or clarify why CrashPlan would not see changes if Spotlight was in fact working.

It is otherwise important to mention that if the drive in question is a network shared location, not local to the device, then real-time scanning would certainly not work.

In short, there is not any more-specific method that CrashPlan takes when using Spotlight, versus other applications that reference it. If you are noticing that this real-time scanning does not work, your best area of investigation is Spotlight itself."
calbear88
 
Posts: 14
Joined: Sun Mar 08, 2015 3:01 pm

Re: crashplan backups on ZFS volumes

Postby lundman » Sun Dec 03, 2017 3:45 pm

Yeah nobody really knows how Spotlight works, its got some deep dark secrets inside it. So it sounds like CrashPlan reads the Spotlight index databases directly, which is quite interesting. We do have a bunch of mmap/pagein/out fixes coming, which is cleaning up some issues in that general area - so it would be interesting to see how it runs with those.
User avatar
lundman
 
Posts: 316
Joined: Thu Mar 06, 2014 2:05 pm
Location: Tokyo, Japan


Return to General Help

Who is online

Users browsing this forum: No registered users and 1 guest