Need Help with a Backup Solution

Moderators: jhartley, MSR734, nola

Re: Need Help with a Backup Solution

Post by /dev/null » Sun Nov 11, 2012 8:35 am

I only clone the filesystem (excluded .zfs folder in ccc). Wouldn’t make any sense (without dedup), cauz i hold the latest 14 days, 6 weeks … of snapshots on my source, so at the moment that would make 2,4 TB * 14 even for the 2 weeks of snapshots of backup space, not counting the weekly, monthly, quarterly and yearly snapshots....
/dev/null Offline


 
Posts: 16
Joined: Sat Sep 15, 2012 7:13 am

Backblaze

Post by grahamperrin » Wed Nov 14, 2012 10:00 pm

Backblaze

grahamperrin wrote:I haven't decided whether to go with Amazon Glacier or Backblaze. Decision is long overdue. I might begin a free trial of Backblaze soon.


I haven't begun my own trial, but earlier this week I encouraged a friend (not yet a user of ZFS) to begin a trial.

In a shortlist, alexwasserman wrote:

BackBlaze … didn't like MacZFS.


Thanks. http://mac-os-x-zfs-discuss.19757.n3.na ... =Backblaze finds nothing and https://groups.google.com/forum/?fromgr ... /Backblaze finds only one topic but it's known that Google Groups often fails to find content in Google Groups.

Alex, please: was your problem with MacZFS as described by gwynnebaer on 2012-08-10?
grahamperrin Offline

User avatar
 
Posts: 1596
Joined: Fri Sep 14, 2012 10:21 pm
Location: Brighton and Hove, United Kingdom

Re: Backblaze

Post by amteza » Thu Nov 15, 2012 3:17 am

grahamperrin wrote:Backblaze

grahamperrin wrote:I haven't decided whether to go with Amazon Glacier or Backblaze. Decision is long overdue. I might begin a free trial of Backblaze soon.


I haven't begun my own trial, but earlier this week I encouraged a friend (not yet a user of ZFS) to begin a trial.

In a shortlist, alexwasserman wrote:

BackBlaze … didn't like MacZFS.


Thanks. http://mac-os-x-zfs-discuss.19757.n3.na ... =Backblaze finds nothing and https://groups.google.com/forum/?fromgr ... /Backblaze finds only one topic but it's known that Google Groups often fails to find content in Google Groups.

Alex, please: was your problem with MacZFS as described by gwynnebaer on 2012-08-10?

I'm using Backblaze in my home computer and I'm backing up several zfs volumes, no problems whatsoever.
amteza Offline


 
Posts: 22
Joined: Wed Oct 17, 2012 4:40 am
Location: Spain

Re: Need Help with a Backup Solution

Post by shuman » Thu Nov 15, 2012 9:37 am

Here's what I'm doing for now:

Manually, a couple of times a day:
Code: Select all
zfs snapshot -r zdata@`date +"%F-%H%M%S"`; zfs snapshot -r zTM@`date +"%F-%H%M%S"`; zfs snapshot -r iMovies@`date +"%F-%H%M%S"`


then immediately:
Code: Select all
/usr/local/bin/rsync -aEv --delete --exclude-from '/usr/local/bin/rsyncexclude' /Volumes/zdata/ /Volumes/zTM/zdata/


Whole process takes about 10 minutes depending on the amount of data being backed up.

I use rsync because it has saved my bacon in the past. Still open to suggestions.
- Mac Mini (Late 2012), 10.8.5, 16GB memory, pool - 2 Mirrored 3TB USB 3.0 External Drives
shuman Offline

User avatar
 
Posts: 96
Joined: Mon Sep 17, 2012 8:15 am

link

Post by grahamperrin » Thu Nov 15, 2012 4:09 pm

amteza wrote:I'm using Backblaze in my home computer and I'm backing up several zfs volumes, no problems whatsoever.


Only some of my ZFS volumes are allowed. The most important file system, my encrypted ZFS home, is excluded by Backblaze. For reference or discussion:

(35009) ZFS home misinterpreted as Boot Camp by Backblaze
grahamperrin Offline

User avatar
 
Posts: 1596
Joined: Fri Sep 14, 2012 10:21 pm
Location: Brighton and Hove, United Kingdom

Backblaze .bzvol at file system roots may be problematic

Post by grahamperrin » Wed Nov 21, 2012 3:40 am

Backblaze 2.1.0.572 reports a problematic hard drive configuration:

2012-11-21 08-23-53 screenshot.png
2012-11-21 08-23-53 screenshot.png (87.29 KiB) Viewed 179 times


This is not a drive configuration problem. It is more accurately a problem with the Backblaze interface to its own use of
.bzvol
as described at http://www.backblaze.com/bzvol.html

Backblaze can not show which volume is excluded until after you disconnect a volume. Example:

2012-11-21 08-34-43 screenshot.png
2012-11-21 08-34-43 screenshot.png (136.61 KiB) Viewed 179 times
grahamperrin Offline

User avatar
 
Posts: 1596
Joined: Fri Sep 14, 2012 10:21 pm
Location: Brighton and Hove, United Kingdom

Re: Need Help with a Backup Solution

Post by shuman » Mon Dec 17, 2012 9:28 pm

I have a backup strategy down. It is as follows:

Local Filesystems:
Code: Select all
zdata                1.95Ti   752Gi  16.9Mi  /Volumes/zdata
zdata/Backups         569Gi   752Gi   820Ki  /Volumes/zdata/Backups
zdata/Backups/Lion   92.5Gi   752Gi  29.6Gi  /Volumes/zdata/Backups/Lion
zdata/Backups/OldTM   477Gi   752Gi   477Gi  /Volumes/zdata/Backups/OldTM
zdata/Storage        15.7Gi   752Gi  9.88Gi  /Volumes/zdata/Storage
zdata/Users           478Gi   752Gi   596Ki  /Volumes/zdata/Users
zdata/Users/anika    3.18Gi   752Gi   764Mi  /Volumes/zdata/Users/anika
zdata/Users/chris     471Gi   752Gi   358Gi  /Volumes/zdata/Users/chris
zdata/Users/jaren    1.64Gi   752Gi   806Mi  /Volumes/zdata/Users/jaren
zdata/Users/severin  1.55Gi   752Gi   577Mi  /Volumes/zdata/Users/severin
zdata/Videos          935Gi   752Gi   910Gi  /Volumes/zdata/Videos


are sent to a 2 drive stripe called 'zTM' using jollyjinx' script located here https://github.com/jollyjinx/ZFS-TimeMachine and discussed here http://zevo.getgreenbytes.com/forum/viewtopic.php?f=4&t=1828

I take those 2 drives to work for safe off-site backup. I then take another 1 drive pool called 'zTM2' home and send recursively zdata/Users to it:
Code: Select all
zdata/Users            326Gi   590Gi   104Ki  /Volumes/zTM2/Users
This ensures my most important data is adequately backed up and always offsite.

The above takes care of all the zfs stuff. For my system files, I just rsync to zfs zdata/Backups/Lion which in return makes sure it is also offsite as well as providing local online backups should I need to quickly retrieve something.

This is working pretty well right now, but it is still me manually issuing the commands. Does anyone have a good resource for creating .plist files so launchd will automatically run the scripts? I've looked online and most of them look way to complex for what I need. :arrow: "Launchd, run the following script every 5 minutes." That's what I need. :P

shuman
- Mac Mini (Late 2012), 10.8.5, 16GB memory, pool - 2 Mirrored 3TB USB 3.0 External Drives
shuman Offline

User avatar
 
Posts: 96
Joined: Mon Sep 17, 2012 8:15 am

Lingon to help with snapshots and backups: part one

Post by grahamperrin » Tue Dec 18, 2012 2:40 am

shuman wrote:… a good resource for creating .plist files so launchd will automatically run the scripts? …


  1. versions of Lingon and Lingon 3
  2. choose a version that suits your requirements
  3. use the app to create the required files.
grahamperrin Offline

User avatar
 
Posts: 1596
Joined: Fri Sep 14, 2012 10:21 pm
Location: Brighton and Hove, United Kingdom

ZEVO Community Edition 1.1.1 launchd files for snapshots

Post by grahamperrin » Tue Dec 18, 2012 2:55 am

To complement Patrick's scripts, I use a feature of ZEVO that is present but rarely realised – maybe not suitable for all use cases.

The files

  1. Finder
  2. Go to Folder…
  3. /System/Library/Filesystems/zfs.fs/Contents/Resources

You'll find:
  • three files for snapshots
  • files for other purposes – all those are off-topic from backup.

Please do not edit any file in that area.

Instead, use launchctl(1) to load the one file that you want.

An example of the end result

Code: Select all
macbookpro08-centrim:~ gjp22$ sudo launchctl list | grep zfs
-   2   com.getgreenbytes.zfs.autosnapshots
-   0   com.getgreenbytes.zfs.autopoolscrubs
-   0   com.getgreenbytes.zfs.loader
124   -   com.getgreenbytes.zfs.delegate
macbookpro08-centrim:~ gjp22$ sudo launchctl list com.getgreenbytes.zfs.autosnapshots
{
   "Label" = "com.getgreenbytes.zfs.autosnapshots";
   "LimitLoadToSessionType" = "System";
   "OnDemand" = true;
   "LastExitStatus" = 512;
   "TimeOut" = 30;
   "StandardOutPath" = "/dev/null";
   "ProgramArguments" = (
      "/System/Library/Filesystems/zfs.fs/Contents/Resources/bin/zfs";
      "snapshot";
      "-r";
      "automatic";
   );
};
grahamperrin Offline

User avatar
 
Posts: 1596
Joined: Fri Sep 14, 2012 10:21 pm
Location: Brighton and Hove, United Kingdom

Arq, apparently for credit card holders only

Post by grahamperrin » Thu Dec 27, 2012 5:26 am

Recommended by amteza:

Online Backup for Mac | Arq | Haystack Software

Arq requires Amazon Web Services Simple Storage Service, but I can't sign up – maybe because I have a debit card but by choice, no credit card.

So for me, Arq is a non-starter.
grahamperrin Offline

User avatar
 
Posts: 1596
Joined: Fri Sep 14, 2012 10:21 pm
Location: Brighton and Hove, United Kingdom

PreviousNext

Return to General Discussion

Who is online

Users browsing this forum: ilovezfs and 0 guests

cron