Read speeds dropping to zero

All your general support questions for OpenZFS on OS X.

Read speeds dropping to zero

Postby RobRehnmark » Wed Oct 05, 2016 2:45 am

I have recently started having this VERY unwanted problem where read speed drops to zero and I simply have to wait for the requested data.
This leads to a complete stop in playback/work when watching a video or editing in Final Cut Pro for example... I can get a spinning beach ball and then it just picks up when data comes from the pool again.
I'm not sure if it started with the update to El Capitan 10.11.6 but that's what I'm running now.
It's a 6-core 4 GHz Xeon with 24 GB of RAM (not ECC) and to my knowledge I never had any performance problems that where caused by hardware.
The pool is a 6 drive Raidz2 made from 3 TB Seagate Barracudas.
At one point I had some jerky mouse/UI so I built from source to get the larger memory chunks allocated by ZFS, 512 instead of 128.
Arc max is at 8 GB and Arc meta limit is at 6 GB.
I will post some screen shots to illustrate my problem and the severity of it.

This is how it behave when there is no read stop, scrubbing the pool at speeds of almost 300 MB to almost 700 MB per second:
Image

This shows the very irregular read speeds:
Image

Coming to a complete stop:
Image

And then picking it up again after about 40 seconds:
(the stoppage isn't always this long)
Image

Any help or ideas would be greatly appreciated.
Super grateful to the people developing OpenZFS on OS X!
RaidZ2 working well for storing my data, OS X Server share for Time Machine, FCPX, iTunes, Photos, etc. on my Hackinstosh.
RobRehnmark
 
Posts: 59
Joined: Sat Oct 17, 2015 12:23 am

Re: Read speeds dropping to zero

Postby haer22 » Wed Oct 05, 2016 11:22 am

I have the same problem. Even if I put arc_max to 8 GB I get freezes from time to time. Most probably a memory allocator problem.

I am eagerly awaiting the next version where this hopefully is fixed.
haer22
 
Posts: 123
Joined: Sun Mar 23, 2014 2:13 am

Re: Read speeds dropping to zero

Postby Brendon » Wed Oct 05, 2016 3:44 pm

This will probably alleviate some of the pausing.

sudo syctl -w kstat.zfs.darwin.tuneable.zfs_dirty_data_max=$((128*1024*1024)).

Cheers
Brendon
Brendon
 
Posts: 286
Joined: Thu Mar 06, 2014 12:51 pm

Re: Read speeds dropping to zero

Postby RobRehnmark » Thu Oct 06, 2016 5:34 am

Brendon wrote:This will probably alleviate some of the pausing.

sudo sysctl -w kstat.zfs.darwin.tuneable.zfs_dirty_data_max=$((128*1024*1024)).

[BGH - edited syctl -> sysctl]

Cheers
Brendon


I will try this but what does it actually do?

[BGH - further limits an aspect of ZFS that drives the allocator hard]

And will it stick between reboots, export and import?

[BGH -> only if you edit into zsysctl.conf which should be under /etc/zfs/...]

EDIT:
syctl: command not found

EDIT2:
changed syctl to zsysctl but got this error..
"top level name kstat in kstat.zfs.darwin.tuneable.zfs_dirty_data_max is invalid"
Last edited by RobRehnmark on Thu Oct 06, 2016 6:38 am, edited 2 times in total.
Super grateful to the people developing OpenZFS on OS X!
RaidZ2 working well for storing my data, OS X Server share for Time Machine, FCPX, iTunes, Photos, etc. on my Hackinstosh.
RobRehnmark
 
Posts: 59
Joined: Sat Oct 17, 2015 12:23 am

Re: Read speeds dropping to zero

Postby RobRehnmark » Thu Oct 06, 2016 5:35 am

Also, I just noticed that I got 1 checksum error on one of the drives.
It was repaired and it never happened before so I'm not worried.

Image
Super grateful to the people developing OpenZFS on OS X!
RaidZ2 working well for storing my data, OS X Server share for Time Machine, FCPX, iTunes, Photos, etc. on my Hackinstosh.
RobRehnmark
 
Posts: 59
Joined: Sat Oct 17, 2015 12:23 am

Re: Read speeds dropping to zero

Postby lundman » Wed Oct 12, 2016 7:46 pm

"sysctl"
User avatar
lundman
 
Posts: 1337
Joined: Thu Mar 06, 2014 2:05 pm
Location: Tokyo, Japan

Re: Read speeds dropping to zero - Oh CRAP!

Postby RobRehnmark » Thu Oct 13, 2016 11:56 am

EDIT:
Deleted and started a new thread.
Super grateful to the people developing OpenZFS on OS X!
RaidZ2 working well for storing my data, OS X Server share for Time Machine, FCPX, iTunes, Photos, etc. on my Hackinstosh.
RobRehnmark
 
Posts: 59
Joined: Sat Oct 17, 2015 12:23 am

Re: Read speeds dropping to zero

Postby RobRehnmark » Thu Nov 03, 2016 12:39 am

The complete stop in read speed was due to faulty drives.
It seemed like the drive(s) halted all SATA traffic when their SMART status was queried.
After replacing the drives it's all working ok again.
Super grateful to the people developing OpenZFS on OS X!
RaidZ2 working well for storing my data, OS X Server share for Time Machine, FCPX, iTunes, Photos, etc. on my Hackinstosh.
RobRehnmark
 
Posts: 59
Joined: Sat Oct 17, 2015 12:23 am


Return to General Help

Who is online

Users browsing this forum: No registered users and 28 guests

cron