ZFS not observing desired memory limits?
Posted: Tue Jul 05, 2016 8:52 pm
Hi, I followed the instructions on this page:
https://openzfsonosx.org/wiki/Memory_utilization
to create a file "zsysctl.conf" that I had hoped would limit the RAM usage by ZFS; my intent was to reserve 8GB (out of 24GB total RAM) for the operating system and applications at all times. However, I'm not seeing that behavior, I don't think; if I do some actions that cause ZFS to cache a bunch of file data (like ditto an HFS directory onto a ZFS dataset, or run Carbon Copy Cloner to populate a ZFS dataset from a backup) the wired memory crawls up until wired+app memory consumes the whole 24GB. And then if I try to launch a large app like LibreOffice it takes 30+ seconds instead of 1 second.
Here is my zsysctl.conf file:
# Uses the standard SYSCTL.CONF(5) format.
# Comments are denoted by a "#" at the beginning of a line.
# It is highly recommended to put a date and justification as comments
# alongside each tuning.
# The zfs_arc_max parameter is in bytes and accepts decimal or
# hexadecimal values. The following text shows how to set this parameter
# to 16 GB, as an example:
# 27 May 2016: adminkurt
# Cap the ARC to 16 GB reserving 8 GB for applications (24 GB RAM in box)
kstat.zfs.darwin.tunable.zfs_arc_max=17179869184
# 27 May 2016: adminkurt
# Raise zfs_arc_meta_limit to 10 GB (instead of 1/4 of zfs_arc_max).
kstat.zfs.darwin.tunable.zfs_arc_meta_limit=10737418240
# end of file
Do you think ZFS is or isn't observing the limit I've tried to put in place? Or are the actions I'm doing (copying a large group of files in the Terminal, say) causing OS X to also use wired memory, and together they are grabbing all the available memory? How might I distinguish between those two possibilities?
Some notes on my setup:
I have four 2TB drives in the Mac Pro, set up as a zpool with two 2TB mirrored VDEVs. No deduplication is in use on this pool; I have a backup drive that I don't keep plugged that does have dedup turned on, just for testing, but in the cases I'm talking about the backup drive has not been connected and is not a factor.
Thanks for your help and suggestions.
https://openzfsonosx.org/wiki/Memory_utilization
to create a file "zsysctl.conf" that I had hoped would limit the RAM usage by ZFS; my intent was to reserve 8GB (out of 24GB total RAM) for the operating system and applications at all times. However, I'm not seeing that behavior, I don't think; if I do some actions that cause ZFS to cache a bunch of file data (like ditto an HFS directory onto a ZFS dataset, or run Carbon Copy Cloner to populate a ZFS dataset from a backup) the wired memory crawls up until wired+app memory consumes the whole 24GB. And then if I try to launch a large app like LibreOffice it takes 30+ seconds instead of 1 second.
Here is my zsysctl.conf file:
# Uses the standard SYSCTL.CONF(5) format.
# Comments are denoted by a "#" at the beginning of a line.
# It is highly recommended to put a date and justification as comments
# alongside each tuning.
# The zfs_arc_max parameter is in bytes and accepts decimal or
# hexadecimal values. The following text shows how to set this parameter
# to 16 GB, as an example:
# 27 May 2016: adminkurt
# Cap the ARC to 16 GB reserving 8 GB for applications (24 GB RAM in box)
kstat.zfs.darwin.tunable.zfs_arc_max=17179869184
# 27 May 2016: adminkurt
# Raise zfs_arc_meta_limit to 10 GB (instead of 1/4 of zfs_arc_max).
kstat.zfs.darwin.tunable.zfs_arc_meta_limit=10737418240
# end of file
Do you think ZFS is or isn't observing the limit I've tried to put in place? Or are the actions I'm doing (copying a large group of files in the Terminal, say) causing OS X to also use wired memory, and together they are grabbing all the available memory? How might I distinguish between those two possibilities?
Some notes on my setup:
I have four 2TB drives in the Mac Pro, set up as a zpool with two 2TB mirrored VDEVs. No deduplication is in use on this pool; I have a backup drive that I don't keep plugged that does have dedup turned on, just for testing, but in the cases I'm talking about the backup drive has not been connected and is not a factor.
Thanks for your help and suggestions.