is scrub throttled

Developer discussions.

is scrub throttled

Postby haer22 » Sun Mar 23, 2014 2:39 am

Upgraded from MacZFS to OpenZFS1.2.0 and there is a lot of new neat stuff.

But scrub seems to be stuck at slightly above 10 MB/s. On MacZFS it was around 100MB/s, depending on where in the scrub-cycle it was.

I have googled some sys kernel variables but they seem to be Linux/FreeBSD specific.

Is there a throttle on scrub? If so, how do I tweak it?

EDIT:
I created a pool on an SSD and that one scrubbed at 100MB+ so no throttle.
Still, why so slow? Isn't scrubbing block based, i.e. size of files should not matter?

EDIT#2:
I messed around a bit more with my single-SSD pool:
When I had 72GB spread over 3500 files, the scrub-speed was 174MB/s. Mostly DiskImages
When I uploaded my iPhotolibrary of 72 GB but spread over 158000 files, i.e. a LOT of SMALL files, the scrub-speed was 3.6MB/s when I stopped it after 2 hrs.
Conclusion: scrub is file based (which I think is stated in docs as well). Many small files will make the MB/s go down as in most/all filesystem operations.
haer22
 
Posts: 123
Joined: Sun Mar 23, 2014 2:13 am

Re: is scrub throttled

Postby lundman » Sun Mar 23, 2014 2:39 pm

scrub does have a priority setting in the ZIO pipeline, so that with other ZFS data requests, the scrub runs last. If you are doing no other IO, then this does not kick in.

With scrubbing, ZFS "tends" to do metadata first (ie, meta data, small files headers) and therefor start quite slowly. Once it starts on the data blocks, it picks up considerably.

Having said that, I wouldn't be too surprised at the possibility it could be slower than other implementations, we've not looked at optimisation.
User avatar
lundman
 
Posts: 1335
Joined: Thu Mar 06, 2014 2:05 pm
Location: Tokyo, Japan

Re: is scrub throttled

Postby haer22 » Wed Mar 26, 2014 2:22 am

3.8T took 51 hours to scrub. And I saw that the current bandwidth went up and down over the course of scrubbing between 10MB/s and 45 MB/s. It ended with a total average of 26MB/s and the final hour was on 10 MB/s.

That is a 4 disk raidz. I had planned to run it every week but I will back of to every fortnight.

If this is the same code that do the resilvering, I think it could need some love. If scrubbing is slow, that is one thing, but slow resilvering, that could hurt.
haer22
 
Posts: 123
Joined: Sun Mar 23, 2014 2:13 am

Re: is scrub throttled

Postby ilovezfs » Mon Apr 14, 2014 10:51 pm

For this very reason, you should read: http://constantin.glez.de/blog/2010/01/ ... still-best
ilovezfs
 
Posts: 232
Joined: Thu Mar 06, 2014 7:58 am

Re: is scrub throttled

Postby AlexWasserman » Mon Apr 21, 2014 7:19 pm

My 3.5Tb pool normally takes 25-35 hours to scrub, depending on system load.

This week it took 5.5 hours. 25% of the usual runtime.

I still have 3.8TB used, so it's not like the data has vanished.

Did a recent patch significantly improve the scrub code, or should I wary of this? Right now I'm ecstatic.
AlexWasserman
 
Posts: 11
Joined: Sat Mar 29, 2014 3:40 pm

Re: is scrub throttled

Postby lundman » Tue Apr 22, 2014 12:44 am

We discussed this in irc, but yes - the current master has had lots of speed improvements. We are moving towards a new installer but want to fix some ZVOL issues first.
User avatar
lundman
 
Posts: 1335
Joined: Thu Mar 06, 2014 2:05 pm
Location: Tokyo, Japan


Return to OpenZFS on OS X Development

Who is online

Users browsing this forum: No registered users and 11 guests