Automated zpool check and scrub

Moderators: jhartley, MSR734, nola

Automated zpool check and scrub

Post by satadru » Tue Oct 09, 2012 12:30 pm

So I'm using a modified zpadmin.pl script in my crontab. This checks the zpools hourly, and does a weekly scrub on Monday mornings at 1AM.

Code: Select all
# ZPOOL MONITOR ->
0 * * * * ~/bin/zpadmin_modified.pl
# ZPOOL WEEKLY SCRUB ->
0 1 * * 1 ~/bin/zpadmin_modified.pl -scrub



I put the script in ~/bin - and note that the script can be run as a non-root user, for the security inclined.

Also do remember to make the script executable!

Code: Select all
chmod +x zpadmin_modified.pl


and here's the diff from the standard bigadmin zpadmin script, which I found here, since Sun's bigadmin no longer seems to be hosted by Oracle: ftp://ftp.scn.rain.com/pub/solaris/zpadmin.txt

(This was originally released under this license: http://web.archive.org/web/20100715080220/http://www.sun.com/bigadmin/common/berkeley_license.jsp)

Code: Select all
--- zpadmin.pl   2012-09-26 15:26:59.000000000 -0400
+++ zpadmin_modified.pl   2012-10-09 13:17:03.000000000 -0400
@@ -18,7 +18,7 @@
 use Net::SMTP;
 use Sys::Hostname;
 
-my $log_path="/nfs/zpool_adm_log";
+my $log_path="/var/log/zpool_adm_log";
 my $hostname = hostname;
 
 sub log_event {
@@ -41,7 +41,7 @@
 
 sub get_zp_list {
 # zpool list -H -o name
-      my $pool_names = `zpool list -H -o name`;
+      my $pool_names = `/usr/sbin/zpool list -H -o name`;
       if ($pool_names eq "no pools available\n") {
             return $pool_names;
       }
@@ -52,7 +52,7 @@
 sub get_zp_health {
 # zpool list -H -o health $pool_name
       my $pool_name = shift;
-      my $result = `zpool list -H -o health $pool_name`;
+      my $result = `/usr/sbin/zpool list -H -o health $pool_name`;
       chomp $result;
       return $result;
 }
@@ -66,7 +66,7 @@
 sub scrub_pool {
 # zpool scrub $pool_name
       my $pool_name = shift;
-      system("zpool scrub $pool_name") == 0 or
+      system("/usr/sbin/zpool scrub $pool_name") == 0 or
             die "zpool scrub $pool_name failed: $?";
       my $result = $? >> 8;
 
@@ -108,7 +108,7 @@
 }
 
 #START PROCESS>>>
-if ( ! -f "/sbin/zpool") {
+if ( ! -f "/usr/sbin/zpool") {
       print "zpools not supported in this sun release.\n";
       exit 1;
 }
Last edited by satadru on Wed Oct 10, 2012 9:47 pm, edited 1 time in total.
satadru Offline


 
Posts: 20
Joined: Fri Sep 14, 2012 10:07 pm

Re: Automated zpool check and scrub

Post by alexwasserman » Wed Oct 10, 2012 12:00 am

I just created a script to run the scrub, but I use a launchdaemon in launchd running as root to actually start the scrub.

It's runs early Monday AM, and usually takes about 5-6 hours to complete.

I use GeekTool to update zpool status on my desktop to keep an eye on things.
alexwasserman Offline


 
Posts: 21
Joined: Mon Sep 17, 2012 4:04 pm

Re: Automated zpool check and scrub

Post by grahamperrin » Wed Oct 10, 2012 1:13 am

Likewise, I opted for launchd. And changed my copy of the script a little to suit my ways of working, for example:

/private/var/log/com.getgreenbytes.zevo.forum.satadru.zpadmin.log

(OK, so it's not a GreenBytes-provided script, but I like the name of the log to remind me of the origin.)

A Lingon 2.2.1 view of one of the two daemons:

2012-10-10 06-55-42 screenshot.png
2012-10-10 06-55-42 screenshot.png (130.82 KiB) Viewed 179 times


For the .pl at first I forgot to set the executable bit (d'uh). After correcting my mistake I wondered why no log entries were made when volumes are mounted. I guess that entries are made only if there's a problem.

The other script, for scrub, all pools at once, I haven't allowed time for the daemon to initiate this but a manual run of the .pl succeeded. A Quick Look view of the log:

2012-10-10 06-54-41 screenshot.png
2012-10-10 06-54-41 screenshot.png (54.6 KiB) Viewed 179 times


Disk activity:

2012-10-10 07-03-39 screenshot.png
2012-10-10 07-03-39 screenshot.png (53.45 KiB) Viewed 179 times


As expected with Community Edition 1.1.1, System Preferences recognises only one of multiple checkups. In my case the pane recognises either:

  1. the scrub that began last; or
  2. if that scrub is interrupted, the scrub that began before.

Postscript

Spun off from this topic: multiple concurrent scrubs
grahamperrin Offline

User avatar
 
Posts: 1596
Joined: Fri Sep 14, 2012 10:21 pm
Location: Brighton and Hove, United Kingdom

Re: Automated zpool check and scrub

Post by alexwasserman » Wed Oct 10, 2012 10:33 pm

Last I checked Lingon only works with user daemons.

I wrote my launchd script (copy it from Lingon if you want) and then moved it to /Library/LaunchDaemons.

then:

sudo launchctl load DataScrub.plist

and it's loaded up as a system script and will run as root regardless of who is logged in.

I have it log to /var/log/ZFS and put an entry in /etc/newsyslog.d/ so that it will keep the last 10 logs and logrotate every so often.

Instructions for similar stuff is here:

http://alexwasserman.com/code_backups.htm
alexwasserman Offline


 
Posts: 21
Joined: Mon Sep 17, 2012 4:04 pm

Re: Automated zpool check and scrub

Post by grahamperrin » Thu Oct 11, 2012 12:54 am

alexwasserman wrote:… I have it log to /var/log/ZFS and put an entry in /etc/newsyslog.d/ so that it will keep the last 10 logs and logrotate every so often.

Instructions for similar stuff is here:

http://alexwasserman.com/code_backups.htm


Thanks!

… Lingon only works with user daemons. …


For reference: versions of Lingon and Lingon 3 – my preferred version of Lingon works with files for all classes of agent and daemon.
grahamperrin Offline

User avatar
 
Posts: 1596
Joined: Fri Sep 14, 2012 10:21 pm
Location: Brighton and Hove, United Kingdom

Re: Automated zpool check and scrub

Post by alexwasserman » Wed Oct 17, 2012 8:07 pm

Graham,

I think you misunderstood. On the Lingon homepage you'll see the following: Please note that Lingon 3 can only create jobs as the current user.

Basically, it can't create and run jobs as root (LaunchDaemons in /Library/LaunchDaemons) - only user LaunchAgents that go into ~/Library/LaunchAgents.

Launchd will only run user jobs after the user is logged in and their agents have been loaded. If you want a job to run at a time regardless you need to create a system job as a LaunchDaemon.

Also, TBH, I find that for such system type jobs, I'd rather have them scheduled into the system, rather than personally through my own user account.
alexwasserman Offline


 
Posts: 21
Joined: Mon Sep 17, 2012 4:04 pm

Re: Automated zpool check and scrub

Post by grahamperrin » Wed Oct 17, 2012 9:29 pm

I understand that limitation of Lingon 3, thanks – it's one of three reasons why I prefer a lesser version of the app – the screenshot above is from 2.2.1. More recently (off topic from ZFS, ZEVO, check or scrub): for some things I prefer 1.2.

… create a system job as a LaunchDaemon. … 


To administer system-wide daemons, aim for
/Library/LaunchDaemons
– never
/System/Library/LaunchDaemons
grahamperrin Offline

User avatar
 
Posts: 1596
Joined: Fri Sep 14, 2012 10:21 pm
Location: Brighton and Hove, United Kingdom

Re: Automated zpool check and scrub

Post by jollyjinx » Thu Oct 18, 2012 3:11 am

@alexwassermann:

why did you not use or if it did not fit your needs update the autoscrub script thats part of my TimeMachine backup script I've put on GitHub ( http://github.com/jollyjinx/ZFS-TimeMachine ) ? That would not have needed a patch ;-)
jollyjinx Offline


 
Posts: 60
Joined: Sun Sep 16, 2012 12:40 pm
Location: Munich - Germany

Re: Automated zpool check and scrub

Post by alexwasserman » Thu Oct 18, 2012 11:37 pm

@Jolly,

I didn't know you had a script. I just wrote my own quickly.

I actually have a Time Machine I use for keeping snapshot type data, and my docs get backed up to DropBox.

I really use ZFS heavily for the big stuff - iPhoto (50Gb), iTunes (>1Tb), and stuff like that.
alexwasserman Offline


 
Posts: 21
Joined: Mon Sep 17, 2012 4:04 pm


Return to General Discussion

Who is online

Users browsing this forum: No registered users and 1 guest

cron