Slow Write Performance

[mstan]

I am getting unacceptably slow write performance. For example, backing up a 4TB media dataset on my main internal pool to an external pool and restoring that data back to the internal pool is running at only 33MB/s avg with many brief spikes above and below that.

Hardware setup is a '09 MacPro, 32GB Ram 4 x 4TB HDDs in a 2 mirrored pair striped pool. External pool is an 8-bay SanDigital Sata II Tower with separate eSata ports for each 4 bay group feeding a Sata II Firmtek pcie card; 8 HDDs in 4 mirrored pairs mixing pairs of 2TBand 3TB HDDs.

Running Yosemite and El Cap on SSDs. Just upgraded to O3x 1.52 from 1.31rc. Performance was similar across Mac OS and O3x versions.

Needlees to say that I can't move terabytes of data at these speeds. What can I do?

[mstan]

Forgot to mention that neither pool is close to full running at 29% and 59% full.

[Brendon]

@mstan,

Yep thats very slow. How fast is your internal pool running? I'd suspect hardware issues of some sort. Not easy to track down, except by pulling things apart.

Maybe you can post some output of zpool iostat 1 here?

- Brendon

[mstan]

Ok. Here are some results from iostat; copying Apple Loops from the OS X system on a Samsung SSD to the ZFS internal pool:
zpool iostat 1 10
capacity operations bandwidth
pool alloc free read write read write
------------- ----- ----- ----- ----- ----- -----
MacProZFSPool 4.34T 2.91T 60 8 6.23M 344K
MacProZFSPool 4.34T 2.91T 88 52 386K 3.05M
MacProZFSPool 4.34T 2.91T 18 1.50K 75.7K 59.3M
MacProZFSPool 4.34T 2.91T 108 136 435K 6.43M
MacProZFSPool 4.34T 2.91T 35 1.44K 147K 56.1M
MacProZFSPool 4.34T 2.91T 87 32 335K 1.64M
MacProZFSPool 4.34T 2.91T 51 981 206K 43.2M
MacProZFSPool 4.34T 2.91T 82 580 332K 17.3M
MacProZFSPool 4.34T 2.91T 76 8 318K 270K
MacProZFSPool 4.34T 2.91T 33 1.04K 139K 45.3M

[Brendon]

What are "Apple Loops"? If you're specific, I'll do the same on my system when I get home, at least that will give you some comparison.

B.

[mstan]

What are "Apple Loops"? If you're specific, I'll do the same on my system when I get home, at least that will give you some comparison.

B.

They are musical effects installed with Apple's Logic DAW program.

[Brendon]

Ok, I dont have that/those.

- Brendon

[mstan]

Ok, I dont have that/those.

- Brendon

What kind audio files do you have? .wav .flac .aif?

[mstan]

Here is a copy of an 5GB 720p .mpg video file from system SSD to ZFS (it took a few minutes compared to 10 seconds copying from the ZFS pool to the SSD):


zpool iostat 1 10
capacity operations bandwidth
pool alloc free read write read write
------------- ----- ----- ----- ----- ----- -----
MacProZFSPool 4.34T 2.91T 30 56 2.66M 1.87M
MacProZFSPool 4.34T 2.91T 0 220 0 9.42M
MacProZFSPool 4.34T 2.91T 0 183 0 22.8M
MacProZFSPool 4.34T 2.91T 0 187 0 23.2M
MacProZFSPool 4.34T 2.91T 0 169 0 21.1M
MacProZFSPool 4.34T 2.91T 0 199 0 24.8M
MacProZFSPool 4.34T 2.91T 0 1.04K 0 115M
MacProZFSPool 4.34T 2.91T 0 524 0 64.8M
MacProZFSPool 4.34T 2.91T 0 225 0 9.03M
MacProZFSPool 4.34T 2.91T 0 0 0 0

[mstan]

All my posted tests were done under Yosemite 10.10.5; O3X 1.5.2 (note: I have not yet upgraded the pool for the new feature flags).