slow local send receive?

[Ikukuru]

hi,

I'm moving my data between two local pools and the speed is slow. around 200GB an hour. burst are 200MB/s but typically show 10-40MB/s

anything I can do?

32GB and 8-core xeon. 13TB of data on raidz2

[tangles]

I use mbuffer for local and over the network...

Code: Select all

* prepare to receive on destination host

mbuffer -s 128k -m 1G -I 9090 | zfs receive pool-name/zfs-dataset

* execute the following on source host

zfs send -i pool-name/zfs-dataset@n pool-name/zfs-dataset@(n+1) | mbuffer -s 128k -m 1G -O dest_IP:9090


the n & n+1 are just the snapshot names...

Keeps the i/o nice and consistent for me...

source:
http://everycity.co.uk/alasdair/2010/07 ... s-receive/

[tangles]

Using mbuffer locally is done on one line with two pipe commands

I don't have access to my local notes for the exact syntax

[rottegift]

1.6 includes compressed send/recv (the -c flag), which is likely to help in almost all cases and especially where the receiving dataset has the same compression as the sending one. (The zfs send -e flag is also useful).

The default (and old) behaviour is that the sender uncompresses each object to be sent and the receiver compresses it. With compressed send/recv, the sender transmits the objects as they are in the sender's pool, and the receiver stores the objects as they are in the receiver's pool if the compression algorithm is the same, or uncompresses-and-recompresses otherwise.

Where compression and checksum match on both sides and -c and -e are in use, both the sender and receiver do a lot less work, and a lot less data flows through the pipeline connecting the two.

The zfs man page seems up-to-date with respect to the -c flag.

Also useful for large send/recv is to get to know and love resumable send/recv (zfs send -ecL src/ds@a | zfs recv -vus dest/ds; *INTERRUPT* then receive_resume_token=`zfs get -H -o value receive_resume_token dest/ds` ; zfs send -e -t $receive_resume_token | zfs send -vus dest/ds).