Frequent Beachballs on El Capitan?

All your general support questions for OpenZFS on OS X.

Frequent Beachballs on El Capitan?

Postby Haravikk » Mon Jun 13, 2016 9:44 am

Anyone else experiencing beachballs on El Capitan? I recently upgraded, and reinstalled OpenZFS 1.5.2 for the El Capitan version, and this is my first time running a back-up to my array, which I do once a month or so, but I seem to be getting the spinning beachball and lockups in apps fairly frequently (every 10 minutes or so).

My array isn't very optimal, I have two five-disk enclosures connected by USB3, mirroring each pair of drives between the two (i.e- the first drive of each array is a mirror, the second in each mirrored and so-on). The drives are mostly old spare drives of matching sizes (across each mirror), and I really just use it as an extra back-up run once a month, plus it lets me tinker with ZFS in advance of maybe someday using it for a main volume in future. Still, I've never experienced lock-ups like this under Yosemite, and there don't appear to be any major health issues with any of the drives.

CPU usage doesn't spike or anything when this happens (or if it does Activity Monitor/iStat Menus don't update in time to show it) and I have plenty of RAM free for ZFS to use if it wants to (with no restrictions in place other than whatever the defaults are).

At the current moment Time Machine isn't even copying anything, it just seems to be checking loads of files, which seems to be a side-effect of the update to El Capitan (I had to inherit the backup) so it's got over a million files to verify that may have changed since last month. I don't know what this stage involves exactly, probably lots of stat calls and comparison to local files?


Any ideas what might cause lock-ups? It's especially disruptive to anything with audio since I have an external USB sound card, not sure why it's being disrupted though since it has its own USB port all to itself, but my mouse and keyboard (also USB) aren't affected, except that apps like Safari are sometimes locking up briefly, delaying typing till they unfreeze.

If it's my own fault mainly for hurling a load of old drives into a shoddy enclosure then I can live with it, if this backup ever finishes the next one should be fast again, but I find it weird that this seems to happen more on El Capitan. It very occasionally happened on Yosemite, but definitely not as often as this.
Haravikk
 
Posts: 74
Joined: Tue Mar 17, 2015 4:52 am

Re: Frequent Beachballs on El Capitan?

Postby fustbariclation » Mon Jul 25, 2016 3:00 am

Yes - I'm getting beachballs, and applications freezing.

It seems to happen when a zfs drive is being heavily used - by a compression or copy.

I'm running:

El Capitan: 10.11.6
OpenZFS: zfs.kext_version: 1.5.2-1

I've tried giving extra-nice values to the commands, but it seems to be some sort of locking problem.

Any suggestions?
fustbariclation
 
Posts: 11
Joined: Thu May 12, 2016 2:32 am

Re: Frequent Beachballs on El Capitan?

Postby Brendon » Mon Jul 25, 2016 3:24 am

See thread ui lag and jerky mouse.

In short is most likely the memory allocator. Current running seems to be not good for some.

- Brendon
Brendon
 
Posts: 286
Joined: Thu Mar 06, 2014 12:51 pm

Re: Frequent Beachballs on El Capitan?

Postby fustbariclation » Mon Jul 25, 2016 5:44 am

That's useful - thank you!

I'm quite happy to increase memory allocation. What's the best way of doing this?

I see from:

zdb -mmm <pool>

that there are tons of fragments. Eg. (just pulling one out at random):

metaslab 26 offset 6800000000 spacemap 53 free 16.0G
segments 118 maxsize 14.9G freepct 99%
In-memory histogram:
12: 7 *******
13: 9 *********
14: 19 *******************
15: 6 ******
16: 7 *******
17: 0
18: 0
19: 0
20: 0
21: 22 **********************
22: 28 ****************************
23: 9 *********
24: 7 *******
25: 1 *
26: 0
27: 1 *
28: 1 *
29: 0
30: 0
31: 0
32: 0
33: 1 *
On-disk histogram: fragmentation 4
12: 25507 ****************************************
13: 4941 ********
14: 3949 *******
15: 2249 ****
16: 1207 **
17: 728 **
18: 348 *
19: 197 *
20: 84 *
21: 56 *
22: 5 *
23: 0
24: 0
25: 0
26: 0
27: 0
28: 0
29: 0
30: 0
31: 0
32: 0
33: 1 *

It seems to me that most of them have fairly low usage.

Is there something useful that I can do with this information?
fustbariclation
 
Posts: 11
Joined: Thu May 12, 2016 2:32 am

Re: Frequent Beachballs on El Capitan?

Postby Brendon » Mon Jul 25, 2016 1:09 pm

Its not about "increasing memory allocation". Rather the way there is a bit of a bug in the way our allocator works that has resulted in us having to claim large blocks of memory from the OS and get good performance, but be potentially space inefficient, or suffer glitching and the odd beachball, there's a bit of a spectrum there. We used to claim 512k chunks, and recently shrunk that to 128k based on some feedback from someone. This seems to be working less well for some.

The only thing you can do right now is, assuming you are capable of building the code yourself, increase KMEM_QUANTUM in SPL to 512k and let us know if that makes things better for you.

There is a prototypical fix in the works for the underlying issue, but its not ready to go.

- Brendon
Brendon
 
Posts: 286
Joined: Thu Mar 06, 2014 12:51 pm

Re: Frequent Beachballs on El Capitan?

Postby Brendon » Wed Jul 27, 2016 12:01 am

I've patched master, see https://github.com/openzfsonosx/zfs/issues/524.

If you can recompile, it would be good to make sure this improves things for you.

- Brendon
Brendon
 
Posts: 286
Joined: Thu Mar 06, 2014 12:51 pm

Re: Frequent Beachballs on El Capitan?

Postby Haravikk » Wed Jul 27, 2016 2:39 am

Brendon wrote:Its not about "increasing memory allocation". Rather the way there is a bit of a bug in the way our allocator works that has resulted in us having to claim large blocks of memory from the OS and get good performance, but be potentially space inefficient, or suffer glitching and the odd beachball, there's a bit of a spectrum there. We used to claim 512k chunks, and recently shrunk that to 128k based on some feedback from someone. This seems to be working less well for some.

This is interesting; can we get some information about what the advantages of each are? 512k doesn't seem like it should really place any burden on modern machines. Would it be possible for this value to be made configurable in future or is its range of values too limited? For example, if 128k has some advantage, but 512k results in fewer deadlocks, then what would 256k be like?
Haravikk
 
Posts: 74
Joined: Tue Mar 17, 2015 4:52 am

Re: Frequent Beachballs on El Capitan?

Postby Brendon » Wed Jul 27, 2016 12:58 pm

@Haravikk

There is some commentary on our memory architecture here https://openzfsonosx.org/wiki/Developme ... chitecture.

- Brendon
Brendon
 
Posts: 286
Joined: Thu Mar 06, 2014 12:51 pm

Re: Frequent Beachballs on El Capitan?

Postby stumble » Wed Jul 27, 2016 2:13 pm

@Brendon: That's interesting. Thanks.
stumble
 
Posts: 37
Joined: Thu May 15, 2014 7:05 pm


Return to General Help

Who is online

Users browsing this forum: No registered users and 28 guests