Encryption

Moderators: jhartley, MSR734, nola

encryption – GreenBytes, ZFS on Linux, illumos

Post by grahamperrin » Wed Feb 20, 2013 12:57 am

bking wrote:… if we want encryption built into zfs it will have to be coded by Don and company. that said I think under illumos …


GreenBytes

Considering the new range of solutions, I doubt that anyone from GreenBytes is working on encryption for #zevo alone.

ZFS on Linux

ZFS Crypto support · Issue #494 · zfsonlinux/zfs

illumos

Project Ideas - illumos - illumos wiki (2012-07-18)

Device drivers

> Niagra Crypto
> … Re-implement the crypto acceleration drivers for the SPARC sun4v cpus.

File systems

> ZFS encryption
> … Import and update the work started by Darren Moffat to provide cryptographic support for ZFS.

I'll align myself with the latter.
grahamperrin Offline

User avatar
 
Posts: 1596
Joined: Fri Sep 14, 2012 10:21 pm
Location: Brighton and Hove, United Kingdom

Re: Encryption

Post by ilovezfs » Fri Feb 22, 2013 2:31 am

So I decided to try out Graham's method of encrypting ZEVO using Core Storage:
http://apple.stackexchange.com/question ... on-on-os-x

One snag I ran into is that diskutil will NOT let you use the "diskutil cs decryptVolume" command on your volume to decrypt it in place. You also will NOT be able to delete the volume either. However, I figured out a way to get diskutil to agree to do its job and to decrypt the volume in place, while keeping your data intact. (The same procedure allows you to delete the volume, if that's your goal.)

The solution is to use gdisk (AKA gpt fdisk), which is a third party tool you can download here:
http://sourceforge.net/projects/gptfdisk/

In broad strokes, the steps are
1. Use gdisk to save a backup of the GPT of your ZEVO Core Storage volume's dev node to a backup file
2. Use gdisk's expert mode's zap command to zap the dev node's partition table.
3. Restart your computer.
4. Use "diskutil cs decryptVolume" to decrypt the volume in place.
5. Use gdisk to restore your ZEVO Core Storage volume's GPT from the backup file you saved in step 1.
6. Done. Your volume is decrypted and your zpool will import.

(You can use the same method to delete a ZEVO Core Storage volume, by replacing step 4 with the "diskutil cs deleteVolume" command.)

Summary of commands:
zpool status
zfs list vmzpool
ls -l /Volumes/vmzpool/
zpool export vmzpool
zpool status
diskutil cs info disk3
diskutil cs decryptVolume 32D42EFA-F0AE-471D-A408-D5359B30FC47 [note that diskutil refuses!]
sudo gdisk /dev/disk3
In gdisk's interactive mode:
command "b" to save a backup file
command "x" to enter expert mode
command "z" to zap the GPT, confirm "y", "y"
Restart the computer.
diskutil cs decryptVolume 32D42EFA-F0AE-471D-A408-D5359B30FC47 [note that diskutil complies!]
Wait for decryption to complete, using "diskutil cs list" to monitor the progress.
sudo gdisk /dev/disk3 <-- or whatever the dev node is called now
In gdisk's interactive mode:
command "r" to enter recovery mode
command "l" to load the backup file
command "w" to write the GPT, confirm "y"
zpool status
ls -l /Volumes/vmzpool


Here's a demonstration with my terminal output:
Josephs-Mac:~ joe$ zpool status
pool: vmzpool
state: ONLINE
scan: none requested
config:

NAME STATE READ WRITE CKSUM
vmzpool ONLINE 0 0 0
GPTE_D04CC44E-FD9A-40D3-890D-9CE9F606604A ONLINE 0 0 0 at disk3s2

errors: No known data errors
Josephs-Mac:~ joe$ zfs list vmzpool
NAME USED AVAIL REFER MOUNTPOINT
vmzpool 1.13Mi 8.80Gi 788Ki /Volumes/vmzpool
Josephs-Mac:~ joe$ ls -l /Volumes/vmzpool/
total 16
drwxr-xr-x 2 joe wheel 2 Feb 21 22:14 hello world
Josephs-Mac:~ joe$ zpool export vmzpool
Josephs-Mac:~ joe$ zpool status
no pools available
Josephs-Mac:~ joe$ diskutil cs info disk3
Core Storage Properties:
Role: Logical Volume (LV)
UUID: 32D42EFA-F0AE-471D-A408-D5359B30FC47
Parent LVF UUID: E356C0E5-D3EF-4A78-8B74-BB2E0646AF08
Parent LVG UUID: 933E7E97-FE12-4096-AF2E-66298238DC42
Device Identifier: disk3
LV Status: Online
Conversion Status: Complete
Content Hint: Apple_HFS
LV Name: myLV
LV Size: 10000003072 B
Josephs-Mac:~ joe$ diskutil cs decryptVolume 32D42EFA-F0AE-471D-A408-D5359B30FC47
Passphrase:
Started CoreStorage operation
Error: -69808: Some information was unavailable during an internal lookup
Josephs-Mac:~ joe$ sudo gdisk /dev/disk3
GPT fdisk (gdisk) version 0.8.6

Partition table scan:
MBR: protective
BSD: not present
APM: not present
GPT: present

Found valid GPT with protective MBR; using GPT.

Command (? for help): p
Disk /dev/disk3: 19531256 sectors, 9.3 GiB
Logical sector size: 512 bytes
Disk identifier (GUID): A3902C44-2699-4356-B361-05FA2CA72657
Partition table holds up to 128 entries
First usable sector is 34, last usable sector is 19531222
Partitions will be aligned on 8-sector boundaries
Total free space is 262157 sectors (128.0 MiB)

Number Start (sector) End (sector) Size Code Name
1 40 409639 200.0 MiB EF00 EFI System Partition
2 409640 19269071 9.0 GiB BF01 %noformat%

Command (? for help): b
Enter backup filename to save: backupOfVmzpoolGPT
The operation has completed successfully.

Command (? for help): x

Expert command (? for help): z
About to wipe out GPT on /dev/disk3. Proceed? (Y/N): y
Warning: The kernel may continue to use old or deleted partitions.
You should reboot or remove the drive.
GPT data structures destroyed! You may now partition the disk using fdisk or
other utilities.
Blank out MBR? (Y/N): y
Josephs-Mac:~ joe$ diskutil cs decryptVolume 32D42EFA-F0AE-471D-A408-D5359B30FC47
Passphrase:
Started CoreStorage operation
Error: -69808: Some information was unavailable during an internal lookup
Josephs-Mac:~ joe$ sudo shutdown -r now
Shutdown NOW!

*** FINAL System shutdown message from joe@Josephs-Mac.local ***
System going down IMMEDIATELY



System shutdown time has arrived
Josephs-Mac:~ joe$ Connection to 192.168.230.134 closed by remote host.
Connection to 192.168.230.134 closed.
Josephs-MacBook-Pro:~ joe$ ssh joe@192.168.230.134
Password:
Last login: Thu Feb 21 22:41:56 2013
Josephs-Mac:~ joe$ zpool status
no pools available
Josephs-Mac:~ joe$ diskutil cs decryptVolume 32D42EFA-F0AE-471D-A408-D5359B30FC47
Passphrase:
Started CoreStorage operation on disk3
Scheduling decryption of Core Storage Logical Volume
Core Storage LV UUID: 32D42EFA-F0AE-471D-A408-D5359B30FC47
Finished CoreStorage operation on disk3
Josephs-Mac:~ joe$ diskutil cs info 32D42EFA-F0AE-471D-A408-D5359B30FC47
Core Storage Properties:
Role: Logical Volume (LV)
UUID: 32D42EFA-F0AE-471D-A408-D5359B30FC47
Parent LVF UUID: E356C0E5-D3EF-4A78-8B74-BB2E0646AF08
Parent LVG UUID: 933E7E97-FE12-4096-AF2E-66298238DC42
Device Identifier: disk3
LV Status: Online
Conversion Status: Converting
LV Bytes Converted: 2348810240 B
Content Hint: Apple_HFS
LV Name: myLV
LV Size: 10000003072 B
Josephs-Mac:~ joe$ diskutil cs list
CoreStorage logical volume groups (1 found)
|
+-- Logical Volume Group 933E7E97-FE12-4096-AF2E-66298238DC42
=========================================================
Name: VM HD
Size: 42089914368 B (42.1 GB)
Free Space: 21771137024 B (21.8 GB)
|
+-< Physical Volume CCDD62F7-DD96-432E-8E5D-D7BAE8C04B05
| ----------------------------------------------------
| Index: 0
| Disk: disk1s2
| Status: Online
| Size: 42089914368 B (42.1 GB)
|
+-> Logical Volume Family D1CC8676-0A95-45C4-BD3E-16BF3D072011
| ----------------------------------------------------------
| Encryption Status: Unlocked
| Encryption Type: AES-XTS
| Conversion Status: Complete
| Conversion Direction: -none-
| Has Encrypted Extents: Yes
| Fully Secure: Yes
| Passphrase Required: Yes
| |
| +-> Logical Volume 4895582C-0DE6-4BD5-88B2-7DFE37272FEA
| ---------------------------------------------------
| Disk: disk2
| Status: Online
| Size (Total): 9999998976 B (10.0 GB)
| Size (Converted): -none-
| Revertible: No
| LV Name: VM HD
| Volume Name: VM HD
| Content Hint: Apple_HFS
|
+-> Logical Volume Family E356C0E5-D3EF-4A78-8B74-BB2E0646AF08
----------------------------------------------------------
Encryption Status: Unlocked
Encryption Type: AES-XTS
Conversion Status: Converting
Conversion Direction: backward
Has Encrypted Extents: Yes
Fully Secure: No
Passphrase Required: No
|
+-> Logical Volume 32D42EFA-F0AE-471D-A408-D5359B30FC47
---------------------------------------------------
Disk: disk3
Status: Online
Size (Total): 10000003072 B (10.0 GB)
Size (Converted): 9456058368 B (9.5 GB)
Revertible: No
LV Name: myLV
Content Hint: Apple_HFS
Josephs-Mac:~ joe$ diskutil cs info 32D42EFA-F0AE-471D-A408-D5359B30FC47
Core Storage Properties:
Role: Logical Volume (LV)
UUID: 32D42EFA-F0AE-471D-A408-D5359B30FC47
Parent LVF UUID: E356C0E5-D3EF-4A78-8B74-BB2E0646AF08
Parent LVG UUID: 933E7E97-FE12-4096-AF2E-66298238DC42
Device Identifier: disk3
LV Status: Online
Conversion Status: NoConversion
LV Bytes Converted: 10000003072 B
Content Hint: Apple_HFS
LV Name: myLV
LV Size: 10000003072 B
Josephs-Mac:~ joe$ sudo zpool import vmzpool
Password:
cannot import 'vmzpool': no such pool available
Josephs-Mac:~ joe$ sudo gdisk /dev/disk3
GPT fdisk (gdisk) version 0.8.6

Partition table scan:
MBR: not present
BSD: not present
APM: not present
GPT: not present

Creating new GPT entries.

Command (? for help): p
Disk /dev/disk3: 19531256 sectors, 9.3 GiB
Logical sector size: 512 bytes
Disk identifier (GUID): 380C3C42-FBFC-4238-8D3D-7A8F21F67486
Partition table holds up to 128 entries
First usable sector is 34, last usable sector is 19531222
Partitions will be aligned on 2048-sector boundaries
Total free space is 19531189 sectors (9.3 GiB)

Number Start (sector) End (sector) Size Code Name

Command (? for help): r

Recovery/transformation command (? for help): l
Enter backup filename to load: backupOfVmzpoolGPT

Recovery/transformation command (? for help): p
Disk /dev/disk3: 19531256 sectors, 9.3 GiB
Logical sector size: 512 bytes
Disk identifier (GUID): A3902C44-2699-4356-B361-05FA2CA72657
Partition table holds up to 128 entries
First usable sector is 34, last usable sector is 19531222
Partitions will be aligned on 2048-sector boundaries
Total free space is 262157 sectors (128.0 MiB)

Number Start (sector) End (sector) Size Code Name
1 40 409639 200.0 MiB EF00 EFI System Partition
2 409640 19269071 9.0 GiB BF01 %noformat%

Recovery/transformation command (? for help): w

Final checks complete. About to write GPT data. THIS WILL OVERWRITE EXISTING
PARTITIONS!!

Do you want to proceed? (Y/N): y
OK; writing new GUID partition table (GPT) to /dev/disk3.
Warning: The kernel may continue to use old or deleted partitions.
You should reboot or remove the drive.
The operation has completed successfully.
Josephs-Mac:~ joe$ sudo zpool import vmzpool
cannot import 'vmzpool': a pool with that name is already created/imported,
and no additional pools with that name were found
Josephs-Mac:~ joe$ zpool status
pool: vmzpool
state: ONLINE
scan: none requested
config:

NAME STATE READ WRITE CKSUM
vmzpool ONLINE 0 0 0
GPTE_D04CC44E-FD9A-40D3-890D-9CE9F606604A ONLINE 0 0 0 at disk3s2

errors: No known data errors
Josephs-Mac:~ joe$ zfs list vmzpool
NAME USED AVAIL REFER MOUNTPOINT
vmzpool 1.15Mi 8.80Gi 804Ki /Volumes/vmzpool
Josephs-Mac:~ joe$ ls -l /Volumes/vmzpool/
total 16
drwxr-xr-x 2 joe wheel 2 Feb 21 22:14 hello world
ilovezfs Online


 
Posts: 249
Joined: Sun Feb 10, 2013 9:02 am

Re: Encryption

Post by grahamperrin » Wed Feb 27, 2013 4:46 pm

ilovezfs wrote:… You … will NOT be able to delete the volume …


Please see the workaround under diskutil coreStorage deleteLVG fails where there's ZFS.

… diskutil will NOT let you use the "diskutil cs decryptVolume" command on your volume to decrypt it in place. …


The intention was encryption, not the opposite ;-)

If a decision is made to no longer encrypt data with Core Storage and ZFS, then I should recommend using just one pool technology: ZFS.

Aaron Toponce : ZFS Administration, Part VIII- Zpool Best Practices and Caveats advises:

… Do not use other volume management software beneath ZFS. ZFS will perform better, and ensure greater data integrity, if it has control of the whole block device stack. As such, avoid using dm-crypt, mdadm or LVM beneath ZFS. …
grahamperrin Offline

User avatar
 
Posts: 1596
Joined: Fri Sep 14, 2012 10:21 pm
Location: Brighton and Hove, United Kingdom

Re: Encryption

Post by mgrosman » Sun Sep 29, 2013 10:54 pm

For some reason the "zpool create" now causes this error:
Unable to begin erase operation: Placing a partition map on a Core Storage logical volume is not supported (-69671)

It doesn't matter if CS volume is encrypted or not. Even a simple "diskutil jhfs+ <vol_name> <dev_name>" causes the same error so it may not be related to ZFS.

The OS is 10.8.5. Before this upgrade I was able to create encrypted ZFS devs.
mgrosman Offline


 
Posts: 7
Joined: Sun Oct 07, 2012 3:53 pm

Re: Encryption

Post by ilovezfs » Mon Sep 30, 2013 12:25 am

mgrosman wrote:For some reason the "zpool create" now causes this error:
Unable to begin erase operation: Placing a partition map on a Core Storage logical volume is not supported (-69671)

It doesn't matter if CS volume is encrypted or not. Even a simple "diskutil jhfs+ <vol_name> <dev_name>" causes the same error so it may not be related to ZFS.

The OS is 10.8.5. Before this upgrade I was able to create encrypted ZFS devs.

Use /usr/sbin/gpt or gdisk http://sourceforge.net/projects/gptfdisk/ to create the partition table yourself.
ilovezfs Online


 
Posts: 249
Joined: Sun Feb 10, 2013 9:02 am

Cross reference

Post by grahamperrin » Mon Sep 30, 2013 12:13 pm

mgrosman: thanks. I have:

grahamperrin Offline

User avatar
 
Posts: 1596
Joined: Fri Sep 14, 2012 10:21 pm
Location: Brighton and Hove, United Kingdom

Re: Cross reference

Post by ilovezfs » Mon Sep 30, 2013 9:25 pm

grahamperrin wrote:mgrosman: thanks. I have:


I'm unclear why we think Apple is wrong not to support partition tables in the logical volumes. Why is this necessary at all?
ilovezfs Online


 
Posts: 249
Joined: Sun Feb 10, 2013 9:02 am

OpenZFS, ZFS on Linux and MacZFS approaches to encryption

Post by grahamperrin » Mon Sep 30, 2013 11:55 pm

For OpenZFS ZFS on Linux, around a month ago at https://github.com/zfsonlinux/zfs/issue ... t-23652335 :

> … some encryption code on github. It was determined on the mailing list that
> it includes code from the Solaris 11 leak and is therefore encumbered.
> We will not be using it.

In the shared and open context of OpenZFS, I think that it's important to note the avoidance of the leaked closed code.

From viewtopic.php?p=5366#p5366 :

ilovezfs wrote:… OpenZFS based new MacZFS … separate repository with filesystem-level encryption …


Please: is that separate repository for encryption for MacZFS free from the leaked code?
grahamperrin Offline

User avatar
 
Posts: 1596
Joined: Fri Sep 14, 2012 10:21 pm
Location: Brighton and Hove, United Kingdom

Partition maps on Core Storage logical volumes

Post by grahamperrin » Tue Oct 01, 2013 12:02 am

ilovezfs wrote:… why we think Apple is wrong …


I don't know whether Apple is right or wrong.

I guess that it has an undisclosed reason for restricting how Core Storage is to be used by Build 12F37 of OS X 10.8.5.

It so, and if that reason relates to development of OS X 10.9, it might be a few weeks before more is known to the public.
grahamperrin Offline

User avatar
 
Posts: 1596
Joined: Fri Sep 14, 2012 10:21 pm
Location: Brighton and Hove, United Kingdom

Re: Partition maps on Core Storage logical volumes

Post by ilovezfs » Tue Oct 01, 2013 12:06 am

grahamperrin wrote:
ilovezfs wrote:… why we think Apple is wrong …


I don't know whether Apple is right or wrong.

I guess that it has an undisclosed reason for restricting how Core Storage is to be used by Build 12F37 of OS X 10.8.5.

It so, and if that reason relates to development of OS X 10.9, it might be a few weeks before more is known to the public.

I suspect it's simply that they don't think it's necessary. As you see above, I had pointed out the troubles caused by a gpt when trying to decrypt. It would seem they noticed that issue, among probably others. My workaround was to obliterate the gpt while using their functions, but obviously that is a hack compared to just not using a gpt.
ilovezfs Online


 
Posts: 249
Joined: Sun Feb 10, 2013 9:02 am

Previous

Return to General Discussion

Who is online

Users browsing this forum: ilovezfs and 0 guests

cron