ZFS memory usage is MUCH higher than zfs_arc_max

All your general support questions for OpenZFS on OS X.

ZFS memory usage is MUCH higher than zfs_arc_max

Postby Wowfunhappy » Fri Oct 16, 2020 12:06 pm

Hello! Before I go any further, I'd like to note that I'm using OS X 10.9, and so I'm on version 1.9.2 of OpenZFS on OS X. If this means I can't get help here, I totally understand, but I figured I'd still ask!

My computer has 32 GB of memory, but for a variety of reasons, I'd like to limit ZFS to around 4 GB. So a few months ago, I created the file /etc/zfs/zsysctl.conf with the contents:
Code: Select all
kstat.zfs.darwin.tunable.zfs_arc_max=4294967296

as per the instructions in the wiki.

Unfortunately, I regularly see ZFS using ~ 16 GB of memory. For instance, as of when I'm writing this, if I run in the Terminal:

Code: Select all
sysctl kstat.spl.misc.spl_misc.os_mem_alloc

I get

Code: Select all
kstat.spl.misc.spl_misc.os_mem_alloc: 14750056448

That comes out to around 14 GB! I know actual usage can be somewhat greater than the limit, but that's way higher! Is the wiki outdated? Am I doing something stupid?
Wowfunhappy
 
Posts: 17
Joined: Sat Jul 21, 2018 11:58 am

Re: ZFS memory usage is MUCH higher than zfs_arc_max

Postby leeb » Sat Oct 17, 2020 12:44 pm

Yeah, with overhead actual will usually be higher than max, but that does seem excessive. Do you see the same behavior if you set it temporarily (sysctl -w)?
leeb
 
Posts: 41
Joined: Thu May 15, 2014 12:10 pm

Re: ZFS memory usage is MUCH higher than zfs_arc_max

Postby mkush » Fri Oct 23, 2020 9:58 am

Perfect timing! I just came here looking to ask about the EXACT same thing, only a more extreme case.

My ARC Max is set to 1000000000 (1 billion). This has worked well for me in the past to get good performance on other systems. I don't think it was ever solved as to why setting a low number like this helped, but it did. See the old thread started by me on the subject.

Anyway, this is a new system, and a new disk array:
Mac Pro 2019
192GB memory
Version 1.9.4
MacOS 10.15.7
Samsung 860 EVO 4TB x16 attached by...
ATTO ExpressSAS H12F0
Configured as RAIDZ2

The ATTO is a PCIe host bus adapter (not RAID) with 16 external channels (each drive has its own channel).

Performance seems good enough... 488MB/s write, read not yet tested. I'd like more but I seem to recall that RAIDZ/2 performance ends up about the speed of a single drive, which is about what this is.

For fun, I pulled up the Activity Monitor and watched memory as I copied my first data to the volume... about 128GB of pictures from a CFexpress card. Memory just climbed and climbed... and now sits at over 121GB used. I'm not running anything else to speak of.

Running the command above shows:

Code: Select all
sysctl kstat.spl.misc.spl_misc.os_mem_alloc
kstat.spl.misc.spl_misc.os_mem_alloc: 117805416448

Which is nearly the entire amount of memory used. So what is going on?
mkush
 
Posts: 52
Joined: Tue Sep 30, 2014 1:17 pm

Re: ZFS memory usage is MUCH higher than zfs_arc_max

Postby mkush » Fri Oct 23, 2020 10:36 am

Setting interactively with:
Code: Select all
sudo sysctl -w kstat.zfs.darwin.tunable.zfs_arc_max=1000000000

works perfectly! Perhaps something in how I did the config file is wrong. I will check later to see if I see anything obvious.
mkush
 
Posts: 52
Joined: Tue Sep 30, 2014 1:17 pm

Re: ZFS memory usage is MUCH higher than zfs_arc_max

Postby mkush » Fri Oct 23, 2020 3:35 pm

There is nothing wrong with my zsysctl.conf file. It seems that it is simply not being read. No idea why. The other thing is that my pools NEVER import automatically. So, is some startup process simply not working for me? In the end it's not a huge deal... I'll just make a script that sets the ARC and imports my pools (by serial, as I like it). Speaking of importing by serial... that's really hit or miss for some reason. I'll make another post about that...
mkush
 
Posts: 52
Joined: Tue Sep 30, 2014 1:17 pm

Re: ZFS memory usage is MUCH higher than zfs_arc_max

Postby jawbroken » Sat Oct 24, 2020 1:26 am

If you're on Catalina then you might need to give “Full Disk Access” permission to /bin/bash to get automatic import working. Not sure if that will solve your other problem, though.
jawbroken
 
Posts: 10
Joined: Wed Apr 01, 2015 4:46 am

Re: ZFS memory usage is MUCH higher than zfs_arc_max

Postby mkush » Wed Oct 28, 2020 11:36 am

Sure enough, giving Full Disk Access to /bin/bash did fix the auto-import -- but not the reading of the conf file.

HOWEVER, two drives of my 8-HDD Backup pool (RAIDZ2) are now shown as having corrupt data and have been removed from the pool. See separate post on this problem.
mkush
 
Posts: 52
Joined: Tue Sep 30, 2014 1:17 pm

Re: ZFS memory usage is MUCH higher than zfs_arc_max

Postby Wowfunhappy » Mon Nov 16, 2020 9:47 pm

I fixed the problem by running
Code: Select all
sudo /usr/sbin/zsysctl -f /etc/zfs/zsysctl.conf


Just doing this one time appears to have fixed the problem permanently, across reboots and everything.
Wowfunhappy
 
Posts: 17
Joined: Sat Jul 21, 2018 11:58 am


Return to General Help

Who is online

Users browsing this forum: No registered users and 2 guests

cron