slow write speeds still :cry:

All your general support questions for OpenZFS on OS X.

Re: slow write speeds still :cry:

Postby lundman » Mon Apr 23, 2018 4:02 pm

Ah yep, disappointing.
User avatar
lundman
 
Posts: 1335
Joined: Thu Mar 06, 2014 2:05 pm
Location: Tokyo, Japan

Re: slow write speeds still :cry:

Postby e8wwv » Tue Apr 24, 2018 6:09 pm

lundman wrote:Ah yep, disappointing.



fwiw, I was getting ~90MB/s for the same operation under 1.7.0 (vs 59-61MB/s now).

The latest master following all the april updates seems to be less prone to choking on simultaneous activity at the expense of a lower 'top speed' vs 1.7.0. If that makes any sense. For example torrent activity doesnt seem to choke it as much but writing a tape is slower, was 4.5-6h in december now 11-11.5h for a full lto6.

is it possible or useful to 'ask for assistance' from other openzfs porting groups? seems like this port is not keeping up. given past performance it seems like something went wrong as far back as 2015. is it not reasonable to assume that every change since then needs to be reexamined? there must be a way to script the benchmarking of each commit since day 1. I was trying to suggest this earlier in the thread but that suggestion appears to have caused a problem for someone.
e8wwv
 
Posts: 14
Joined: Sat Apr 21, 2018 3:38 am

Re: slow write speeds still :cry:

Postby lundman » Wed Apr 25, 2018 6:51 pm

We are actually immediately behind illumos in terms of commits, so ahead of ZOL and FBSD. But this does appear to be something specific to this port, and it happened some time back. I can certainly, and have been, ask those devs for insights, but getting them to actually try the osx port has been impossible. I know Brendon tried all binary releases we had, in various combinations to get a feel for the progression. Ultimately, it may just been that we calmed it down a bit, as during heavy ZFS loads it would stutter the mouse, which many GUI users disliked, for some reason.. But it is ongoing still - we have also started looking at ZOL's assembler replacement code, and see if that can also help.
User avatar
lundman
 
Posts: 1335
Joined: Thu Mar 06, 2014 2:05 pm
Location: Tokyo, Japan

Re: slow write speeds still :cry:

Postby macz » Thu Apr 26, 2018 6:14 am

I run a backup server on OS X with 1.4.5 I think but also run my primary server as a esxi all-in-one with a omniosCE / napp-it storage VM providing 2 pools .. one for VMs on stripped S3500 SSD and a data/backup pool with 4x stripped 8TB spinners...

I too have noticed in ominos/illumos similar bad intrapool transfer speeds. a copy from the data pool to the ssd pool (no a sync write operation) as its done from the command line of ominos showed pretty pitiful throughput ...

so perhaps this is an inherited performance issue from the illumos master
macz
 
Posts: 53
Joined: Wed Feb 03, 2016 4:54 am

Re: slow write speeds still :cry:

Postby lundman » Thu Apr 26, 2018 4:08 pm

They did add the write-throttle. the idea is that a single stream can not starve out all IO. But in theory, you can start a second terminal, and get a second copy going and both will do "well".
User avatar
lundman
 
Posts: 1335
Joined: Thu Mar 06, 2014 2:05 pm
Location: Tokyo, Japan

Re: slow write speeds still :cry:

Postby e8wwv » Tue May 29, 2018 10:44 am

e8wwv wrote:
lundman wrote:Compile sources from git - instructions are in the wiki, but issue a "git checkout pr616" first. There might be a switch to zfsadm if you prefer to compile that way


apr 17 master 337f65a 20180421 16:43:01|7463|root|[L182] wrote 1219090432 blocks (2438180864 KBytes) on volume [1], 11:25:41, 59264 KB/sec
pr616 branch 5ae3d33 20180422 15:33:33|39793|root|[L182] wrote 1219090432 blocks (2438180864 KBytes) on volume [1], 11:09:13, 60722 KB/sec

No significant difference copying a large set.


Thought you would like to know, the May 22 master shows a significant improvement in speed. I had not updated since Apr 17 so I could not tell you which commit made the improvement.

may22 master dfbbb5a wrote 1220670464 blocks (2441340928 KBytes) on volume [1], 6:01:17, 112623 KB/sec

It should be noted that I can only attain this speed immediately after a reboot. If I run any intense disk operations without restarting, I get speeds about 80MB/sec. If a torrent app has been running, the machine needs to be restarted before I can get back to 110mb/sec again. Same thing if the machine has gone into hibernation at any point since the restart. Memory issue perhaps? I still have my arc set to 1GB. At no time was RAM more than 1/3 used (12/36gb)
e8wwv
 
Posts: 14
Joined: Sat Apr 21, 2018 3:38 am

Re: slow write speeds still :cry:

Postby lundman » Tue May 29, 2018 6:31 pm

How peculiar - don't see any specific commit that may have improved anything. Also, give the assembler commit a test, although it probably only affects encrypted datasets
User avatar
lundman
 
Posts: 1335
Joined: Thu Mar 06, 2014 2:05 pm
Location: Tokyo, Japan

Re: slow write speeds still :cry:

Postby e8wwv » Thu May 31, 2018 6:43 pm

lundman wrote:How peculiar - don't see any specific commit that may have improved anything. Also, give the assembler commit a test, although it probably only affects encrypted datasets


have there been any changes to zfs itself? im hesitant to try something that might reduce performance.

I tried creating a 'striped mirror' with the command:

Code: Select all
mirror disk6 disk7 mirror disk8 disk9


In testing I found it was no faster than a 2-disk mirror. Did I create the pool properly?

I also noticed that doing a brief copy between two other pools caused the read speed from the striped mirror to drop 50% from which it never recovered until the machine was restarted. Seems like a memory leak of some sort.

Image
e8wwv
 
Posts: 14
Joined: Sat Apr 21, 2018 3:38 am

Re: slow write speeds still :cry:

Postby lundman » Thu May 31, 2018 10:28 pm

We have no memory leaks (it reports on it on kextunload) but yes, it does seem the SPL glitched and never quite recovered. Like it enters low-memory mode and never recovers from it.
User avatar
lundman
 
Posts: 1335
Joined: Thu Mar 06, 2014 2:05 pm
Location: Tokyo, Japan

Re: slow write speeds still :cry:

Postby tangles » Tue Jul 10, 2018 9:58 pm

Just curious if anyone on the other side of the fence such as the Illumnios guys picked anything up in their builds to fix this by chance?

Any chatter about it even? cos it must be causing pain for others in prod. environments…
tangles
 
Posts: 195
Joined: Tue Jun 17, 2014 6:54 am

PreviousNext

Return to General Help

Who is online

Users browsing this forum: Google [Bot] and 25 guests