Kernel Panic doing raw encrypted send/receive, ZoL #7215?
Posted: Sun May 06, 2018 3:34 pm
Does O3X 1.7.3 beta implement patch #7236 yet? I'm excitedly in the process of finalizing a major update of a system (all the way from ZEVO!) and started by getting everything going on a large platter pool. That mostly is working right aside from known O3X bugs like no .DS_Store files or issues that I've managed to work around so far, so today I tried to then move a bunch of stuff from that onto an SSD pool (like user Library folders) for faster access. However, doing raw send receive triggers a kernel panic and system down after a few dozen gigabytes every time. I'm including two kernel panic reports that were generated, and see references to an invalid dnode block MAC. It might be entirely disconnected but it made me wonder if this might be related to ZoL #7215, which was initially noticed as something wrt split but seems similar to the problem I'm seeing too?
System: Mac Pro 5,1 with 64GB of RAM, macOS 10.13.4, O3X 1.7.3 BETA. No tunables or such have been messed with. Source pool is a mirror of spinning platters (two HGST He6s), target is a brand new set of 2 pool mirrored SSDs. No L2ARC or ZIL or anything yet, all pretty vanilla. Reproduces consistently:
Incidentally I can't work around it doing ~25 gigs at a time since resume doesn't seem to work either (cutting out long token here for clarity):
Finally I've tried to just send it over unencrypted then re-encrypt in the new pool, in this case that wouldn't have any security implications since it's all on one system, I had just wanted to do raw to try it out and for simplicity/speed. FWIW the man page is very confusing here, and I think I'm just kind of going to have to hack it here (do a bunch of sending/moving around). Man page for zfs receive states:
But it's not actually clear at how to do this since the example commands don't actually list any -o options except -o origin=snapshot, and a naive effort isn't successful:
Eh. Edit: Looks like this is just my own misunderstanding, not related to using -o options. It happens the same way with send -R | receive and nothing else, despite the key being loaded (keystatus=available). So yeah, not sure how to send an encrypted dataset unencrypted period right now, I'll have to play with that. The man page didn't indicate anything special being needed, rather it seemed to indicate that sending unencrypted was the default if you don't use raw. The -R option is as it always has been and has no special notes on encryption.
I'm assuming I can just make a temporary parent directory with the right key, send to that, it'll be encrypted properly via inheritance, I can then zfs set its key to the same manually and move it up a level and delete the temporary parent and ultimately achieve the same effect. Still it seems like it'd be a lot better to just be able to replicate raw with a send -Rw | receive and call it a day. Any suggestions about what I might be getting wrong here would be great as well though. Still awesome that everything is mostly great in operation!
System: Mac Pro 5,1 with 64GB of RAM, macOS 10.13.4, O3X 1.7.3 BETA. No tunables or such have been messed with. Source pool is a mirror of spinning platters (two HGST He6s), target is a brand new set of 2 pool mirrored SSDs. No L2ARC or ZIL or anything yet, all pretty vanilla. Reproduces consistently:
- Code: Select all
# zfs send -Rw source/foo@snapshot | zfs receive -suv target/bar
Incidentally I can't work around it doing ~25 gigs at a time since resume doesn't seem to work either (cutting out long token here for clarity):
- Code: Select all
System:~ root# zfs get receive_resume_token source/foo
NAME PROPERTY VALUE SOURCE
source/foo receive_resume_token [$TOKEN] -
System:~ root# zfs send -t [$TOKEN] | zfs receive -svu target/bar
receiving incremental stream of source/foo/Library@2018-04-11 into target/bar@2018-04-11
cannot receive incremental stream: kernel modules must be upgraded to receive this stream.
warning: cannot send 'source/foo/Library@2018-04-11': signal received
Finally I've tried to just send it over unencrypted then re-encrypt in the new pool, in this case that wouldn't have any security implications since it's all on one system, I had just wanted to do raw to try it out and for simplicity/speed. FWIW the man page is very confusing here, and I think I'm just kind of going to have to hack it here (do a bunch of sending/moving around). Man page for zfs receive states:
- Code: Select all
Unencrypted streams can be received as encrypted datasets,
either through inheritance or by specifying encryption parameters
with the -o options.
But it's not actually clear at how to do this since the example commands don't actually list any -o options except -o origin=snapshot, and a naive effort isn't successful:
- Code: Select all
System:~ root# zfs send -R source/foo@snapshot | zfs receive -v -o encryption=aes-256-ccm -o keyformat=raw -o keylocation=file:///user.zfskey32 target/bar
invalid optionusage:
receive [-vnsFu] <filesystem|volume|snapshot>
receive [-vnsFu] [-o origin=<snapshot>] [-d | -e] <filesystem>
receive -A <filesystem|volume>
For the property list, run: zfs set|get
For the delegated permission list, run: zfs allow|unallow
cannot send source/foo@snapshot: encrypted dataset source/foo may not be sent with properties without the raw flag
Eh. Edit: Looks like this is just my own misunderstanding, not related to using -o options. It happens the same way with send -R | receive and nothing else, despite the key being loaded (keystatus=available). So yeah, not sure how to send an encrypted dataset unencrypted period right now, I'll have to play with that. The man page didn't indicate anything special being needed, rather it seemed to indicate that sending unencrypted was the default if you don't use raw. The -R option is as it always has been and has no special notes on encryption.
I'm assuming I can just make a temporary parent directory with the right key, send to that, it'll be encrypted properly via inheritance, I can then zfs set its key to the same manually and move it up a level and delete the temporary parent and ultimately achieve the same effect. Still it seems like it'd be a lot better to just be able to replicate raw with a send -Rw | receive and call it a day. Any suggestions about what I might be getting wrong here would be great as well though. Still awesome that everything is mostly great in operation!