System: Mac Pro 5,1 with 64GB of RAM, macOS 10.13.4, O3X 1.7.3 BETA. No tunables or such have been messed with. Source pool is a mirror of spinning platters (two HGST He6s), target is a brand new set of 2 pool mirrored SSDs. No L2ARC or ZIL or anything yet, all pretty vanilla. Reproduces consistently:
- Code: Select all
# zfs send -Rw source/foo@snapshot | zfs receive -suv target/bar
Incidentally I can't work around it doing ~25 gigs at a time since resume doesn't seem to work either (cutting out long token here for clarity):
- Code: Select all
System:~ root# zfs get receive_resume_token source/foo
NAME PROPERTY VALUE SOURCE
source/foo receive_resume_token [$TOKEN] -
System:~ root# zfs send -t [$TOKEN] | zfs receive -svu target/bar
receiving incremental stream of source/foo/Library@2018-04-11 into target/bar@2018-04-11
cannot receive incremental stream: kernel modules must be upgraded to receive this stream.
warning: cannot send 'source/foo/Library@2018-04-11': signal received
Finally I've tried to just send it over unencrypted then re-encrypt in the new pool, in this case that wouldn't have any security implications since it's all on one system, I had just wanted to do raw to try it out and for simplicity/speed. FWIW the man page is very confusing here, and I think I'm just kind of going to have to hack it here (do a bunch of sending/moving around). Man page for zfs receive states:
- Code: Select all
Unencrypted streams can be received as encrypted datasets,
either through inheritance or by specifying encryption parameters
with the -o options.
But it's not actually clear at how to do this since the example commands don't actually list any -o options except -o origin=snapshot, and a naive effort isn't successful:
- Code: Select all
System:~ root# zfs send -R source/foo@snapshot | zfs receive -v -o encryption=aes-256-ccm -o keyformat=raw -o keylocation=file:///user.zfskey32 target/bar
invalid optionusage:
receive [-vnsFu] <filesystem|volume|snapshot>
receive [-vnsFu] [-o origin=<snapshot>] [-d | -e] <filesystem>
receive -A <filesystem|volume>
For the property list, run: zfs set|get
For the delegated permission list, run: zfs allow|unallow
cannot send source/foo@snapshot: encrypted dataset source/foo may not be sent with properties without the raw flag
Eh. Edit: Looks like this is just my own misunderstanding, not related to using -o options. It happens the same way with send -R | receive and nothing else, despite the key being loaded (keystatus=available). So yeah, not sure how to send an encrypted dataset unencrypted period right now, I'll have to play with that. The man page didn't indicate anything special being needed, rather it seemed to indicate that sending unencrypted was the default if you don't use raw. The -R option is as it always has been and has no special notes on encryption.
I'm assuming I can just make a temporary parent directory with the right key, send to that, it'll be encrypted properly via inheritance, I can then zfs set its key to the same manually and move it up a level and delete the temporary parent and ultimately achieve the same effect. Still it seems like it'd be a lot better to just be able to replicate raw with a send -Rw | receive and call it a day. Any suggestions about what I might be getting wrong here would be great as well though. Still awesome that everything is mostly great in operation!