Is the following possible?

All your general support questions for OpenZFS on OS X.

Is the following possible?

Postby tangles » Thu Jul 11, 2019 5:37 am


Here's the setup:
I have 48GB of RAM in an X58 system with Xeon CPU.
2 pairs of mirrored 12TB disks. (4 total).
Server does nothing but serve up files via SMB over 10 Gigabit link.

Here's what I'm seeing:
Using the Finder to copy data via SMB (so async writes) writing from Mac Mini with 10GB Nic to server with 10GB Nic and I see 1.1GB/s for about 5 seconds, and then transfer drops down to about 170MB/sec.
So that means about ~6GB of data is written to RAM before ZFS decides to commit RAM to the disks? After this, the transfer drops down to the sum of the vdev spindle speeds for the remaining data until the transfer is complete.

Here's what I'm wanting:
Because I have 48GB RAM, I'm wanting the RAM to be filled to at least 20GB before ZFS starts writing to the disks from RAM.
The reason for this is because almost all my files I ever send to the server don't exceed 20GB. Plus it would be nice to use the RAM for async writes because there's no need for any cache devices with this setup.

I have played with all sorts of sysctl commands in an attempt to make this happen, but no matter what I try, it's always slowing down after ~5 seconds.

What is/are the sysctl settings to achieve the above? or is this 5 seconds (or the equivalent of 5 seconds of data being written) hard coded somewhere?

I've found heaps of weblinks such as describing tunables such as vfs.zfs.txg.timeout but I'm just not having any luck achieving a 15-20sec duration of writing to RAM or ~20GB ram fill equivalent.

Can anyone help with this?

Posts: 146
Joined: Tue Jun 17, 2014 6:54 am

Re: Is the following possible?

Postby tangles » Mon Jul 15, 2019 1:29 am

So I take it that it's not possible to configure ZFS to have a "Write ARC" of a desired amount?

I'm not finding anything with the Google Machine… :cry:
Posts: 146
Joined: Tue Jun 17, 2014 6:54 am

Re: Is the following possible?

Postby lundman » Mon Jul 15, 2019 3:45 pm

Many tunables that are involved. Of course the arc_max - but that should already be big enough, then there is the pool_sync that runs every now and then, then the txg sync that runs every 5s and so on. You would have to increase the in-flight data to be much bigger to force it to skip writing each txg - which almost sounds unsafe :)

The real solution is to solve why writes are occasionally so poor - but that has proven to be quite the challenge.
User avatar
Posts: 564
Joined: Thu Mar 06, 2014 2:05 pm
Location: Tokyo, Japan

Re: Is the following possible?

Postby tangles » Tue Jul 16, 2019 3:17 am

Hi Lundy,

Thank you for replying.

I had a go with sysctl using:
Code: Select all
vfs.zfs.txg.timeout   15

vfs.zfs.vdev.async_read_max_active   32
vfs.zfs.vdev.async_read_min_active   8

vfs.zfs.vdev.async_write_max_active   32
vfs.zfs.vdev.async_write_min_active   8

vfs.zfs.vdev.scrub_max_active   64
vfs.zfs.vdev.scrub_min_active   24

vfs.zfs.vdev.sync_read_max_active   32
vfs.zfs.vdev.sync_read_min_active   8

Made no change to push more than the default of ~5 seconds/~4GB of async writes into RAM before starting to write to disks.

I just wish there was a tunable to specify how much RAM could be allocated to async writes before it's flushed to disk. It could have a hardcoded limit of 0.75xtotal_ram to ensure things remain safe.

Posts: 146
Joined: Tue Jun 17, 2014 6:54 am

Return to General Help

Who is online

Users browsing this forum: No registered users and 3 guests