Utility bash script- create encrypted datasets(all ciphers)
Posted: Wed May 27, 2020 9:15 am--Available for testing: OSX bash script to create separate encrypted datasets for all currently-known / supported ZFS encryption formats.
Useful for testing I/O speed for each encryption cipher.
https://pastebin.com/GHAXLJW6
--OSX version 2 of the script: Tested on OSX 10.13 High Sierra and relies on (macports or brew) gnu "coreutils" and bash v5
-- Needs testing on Mojave / Catalina
--It is HIGHLY recommended to review/edit the script before running it, as there are a few EDITME's scattered in it that make some assumptions. (And don't forget to chmod +x it.)
--Features:
o Creates RAMdisk if you have at least 6GB RAM, copies ISO file to ramdisk
o Runs openssl benchmark tests and checks if VeraCrypt is installed
o Reads the man page for zfs and creates separate encrypted datasets for each known cipher (aes-128-ccm / gcm, etc)
( NOTE - this relies on a up-to-date man page )
o Generates a helpful log file
o Also cleans up after itself (needs to be called explicitly) by deleting the test datasets and ejecting the ramdisk.
--Known bug: logfile does not capture initial openssl tests:
#Doing aes-128 cbc for 3s on 16 size blocks: 14957278 aes-128 cbc's in 2.87s
--OSX version was adapted from the initial Linux version that I wrote. Once the encrypted datasets are created, you can e.g.
' time cp -v $isofile /Volumes/zpoolname/Test-aes-128-ccm ' and run fio and whatever other tests you like on each dataset.
--The script was a lot of fun to adapt for OSX after completing the Linux version. It's pretty heavily commented and contains a lot of tips for getting OSX information about the running environment that is normally provided by /proc in Linux.
--Log excerpt:
Useful for testing I/O speed for each encryption cipher.
https://pastebin.com/GHAXLJW6
--OSX version 2 of the script: Tested on OSX 10.13 High Sierra and relies on (macports or brew) gnu "coreutils" and bash v5
-- Needs testing on Mojave / Catalina
--It is HIGHLY recommended to review/edit the script before running it, as there are a few EDITME's scattered in it that make some assumptions. (And don't forget to chmod +x it.)
--Features:
o Creates RAMdisk if you have at least 6GB RAM, copies ISO file to ramdisk
o Runs openssl benchmark tests and checks if VeraCrypt is installed
o Reads the man page for zfs and creates separate encrypted datasets for each known cipher (aes-128-ccm / gcm, etc)
( NOTE - this relies on a up-to-date man page )
o Generates a helpful log file
o Also cleans up after itself (needs to be called explicitly) by deleting the test datasets and ejecting the ramdisk.
--Known bug: logfile does not capture initial openssl tests:
#Doing aes-128 cbc for 3s on 16 size blocks: 14957278 aes-128 cbc's in 2.87s
--OSX version was adapted from the initial Linux version that I wrote. Once the encrypted datasets are created, you can e.g.
' time cp -v $isofile /Volumes/zpoolname/Test-aes-128-ccm ' and run fio and whatever other tests you like on each dataset.
--The script was a lot of fun to adapt for OSX after completing the Linux version. It's pretty heavily commented and contains a lot of tips for getting OSX information about the running environment that is normally provided by /proc in Linux.
--Log excerpt:
- Code: Select all
# bash ./zfs-test-encryption-speeds--osx.sh
/var/root/zfs-test-encryption-speeds.log -> /var/root/zfs-test-encryption-speeds.log-old
Wed May 27 11:45:24 CDT 2020 - imac513.local - BEGIN encryption speed tests
o Kernel: 17.7.0
o Zpool version: zfs-1.9.4-0
zfs-kmod-1.9.4-0
ZFS kext module version: net.lundman.zfs(1.9.4)
o CPU detected: Processor Name: Intel Core i5
Processor Speed: 2.7 GHz
Number of Processors: 1
Total Number of Cores: 4
o CPU supports AES acceleration (blank=NO): hw.optional.aes: 1
machdep.cpu.features: FPU VME DE PSE TSC MSR PAE MCE CX8 APIC SEP MTRR PGE MCA CMOV PAT PSE36 CLFSH DS ACPI MMX FXSR SSE SSE2 SS HTT TM PBE SSE3 PCLMULQDQ DTES64 MON DSCPL VMX SMX EST TM2 SSSE3 CX16 TPR PDCM SSE4.1 SSE4.2 x2APIC POPCNT AES PCID XSAVE OSXSAVE TSCTMR AVX1.0
kstat.zfs.darwin.tunable.icp_aes_impl: cycle [fastest] generic x86_64 aesni
o Check for openssl / run benchmarks
/usr/bin/openssl
Doing aes-128 cbc for 3s on 16 size blocks: 15026856 aes-128 cbc's in 2.88s
Doing aes-128 cbc for 3s on 64 size blocks: 3939291 aes-128 cbc's in 2.84s
.
. ( Fast forward, fast forward )
.
LibreSSL 2.2.7
built on: date not available
options:bn(64,64) rc4(ptr,int) des(idx,cisc,16,int) aes(partial) blowfish(idx)
compiler: information not available
The 'numbers' are in 1000s of bytes per second processed.
type 16 bytes 64 bytes 256 bytes 1024 bytes 8192 bytes
aes-128 cbc 83482.53k 88772.75k 101248.00k 91318.16k 90517.10k
aes-192 cbc 80441.12k 77077.44k 78188.11k 84693.78k 79960.18k
.
. ( FF )
.
o PREP - Copy ~1GB iso file to ramdisk if not already there
/Volumes/ramdisk/bionic-desktop-amd64.iso not overwritten
real 0m0.005s
o Supported ZFS encryption types (per ' man zfs ')
aes-128-ccm,aes-192-ccm,aes-256-ccm,aes-128-gcm,aes-192-gcm,aes-256-gcm
o Creating zfilepool dataset for aes-128-ccm if it doesnt already exist
NAME PROPERTY VALUE SOURCE
zfilepool/Test-aes-128-ccm encryption aes-128-ccm -
zfilepool/Test-aes-128-ccm keylocation file:///var/root/zek-testencr-zfs.key local
o Creating zfilepool dataset for aes-192-ccm if it doesnt already exist
NAME PROPERTY VALUE SOURCE
zfilepool/Test-aes-192-ccm encryption aes-192-ccm -
zfilepool/Test-aes-192-ccm keylocation file:///var/root/zek-testencr-zfs.key local
o Creating zfilepool dataset for aes-256-ccm if it doesnt already exist
NAME PROPERTY VALUE SOURCE
zfilepool/Test-aes-256-ccm encryption aes-256-ccm -
zfilepool/Test-aes-256-ccm keylocation file:///var/root/zek-testencr-zfs.key local
o Creating zfilepool dataset for aes-128-gcm if it doesnt already exist
NAME PROPERTY VALUE SOURCE
zfilepool/Test-aes-128-gcm encryption aes-128-gcm -
zfilepool/Test-aes-128-gcm keylocation file:///var/root/zek-testencr-zfs.key local
o Creating zfilepool dataset for aes-192-gcm if it doesnt already exist
NAME PROPERTY VALUE SOURCE
zfilepool/Test-aes-192-gcm encryption aes-192-gcm -
zfilepool/Test-aes-192-gcm keylocation file:///var/root/zek-testencr-zfs.key local
o Creating zfilepool dataset for aes-256-gcm if it doesnt already exist
NAME PROPERTY VALUE SOURCE
zfilepool/Test-aes-256-gcm encryption aes-256-gcm -
zfilepool/Test-aes-256-gcm keylocation file:///var/root/zek-testencr-zfs.key local
Filesystem Type Size Used Avail Use% Mounted on
/dev/disk8s1 zfs 7.3G 460K 7.3G 1% /Volumes/zfilepool
/dev/disk9s1 hfs 1.1G 1.1G 30M 98% /Volumes/ramdisk
zfilepool/Test-aes-128-ccm zfs 7.3G 868K 7.3G 1% /Volumes/zfilepool/Test-aes-128-ccm
zfilepool/Test-aes-192-ccm zfs 7.3G 876K 7.3G 1% /Volumes/zfilepool/Test-aes-192-ccm
zfilepool/Test-aes-256-ccm zfs 7.3G 844K 7.3G 1% /Volumes/zfilepool/Test-aes-256-ccm
zfilepool/Test-aes-128-gcm zfs 7.3G 844K 7.3G 1% /Volumes/zfilepool/Test-aes-128-gcm
zfilepool/Test-aes-192-gcm zfs 7.3G 812K 7.3G 1% /Volumes/zfilepool/Test-aes-192-gcm
zfilepool/Test-aes-256-gcm zfs 7.3G 876K 7.3G 1% /Volumes/zfilepool/Test-aes-256-gcm
-rw-r-xr-- 1 _unknown _unknown 1.0G May 27 11:14 /Volumes/ramdisk/bionic-desktop-amd64.iso
Wed May 27 11:46:11 CDT 2020 ./zfs-test-encryption-speeds--osx.sh - imac513.local - Ready for testing
o ./zfs-test-encryption-speeds--osx.sh Logfile is: /var/root/zfs-test-encryption-speeds.log