1.5.2 usage impact

Developer discussions.

1.5.2 usage impact

Postby haer22 » Sun May 01, 2016 8:24 am

Guys,

could you expand a bit on what the changes mentioned in the changelog may impact on how we use (or should use) zfs.Some issues may be very internal but some may be visible to us users. The first two items seems to me very interesting.

cheers
/hans

OpenZFS_on_OS_X_1.5.2.dmg 2016-04-29[edit]
New upstream checksums (with new feature flags): SHA-512, Edonr, Skein (Matthew Ahrens)
Resumable send and receive (Matthew Ahrens)
Finder notification thread to refresh sizes across datasets (Jorgen Lundman)
SPL: enhanced kmem pressure system (rottegift)
SPL: Rewrite TSD using AVL tree (Jorgen Lundman)
Cache names in getattr (Jorgen Lundman)
InvariantDisks serial fixes (cbreak)
Hardlink LinkID fixes (Jorgen Lundman)
Trivial ACLs hidden (Jorgen Lundman)
Group ACL fixes (Jorgen Lundman)
IOkit deadlock on export fixes (Jorgen Lundman)
MAF and deadlocks in ZVOL fixes (Jorgen Lundman)
diskutil info on ZVOLs fix (Brendon Humphrey, ilovezfs)
ZED notification fixes (Andreas Weinlein)
Have automatic import always use by-id (Justin Scholz)
Fix handling of XATTR_CREATE and XATTR_REPLACE (ilovezfs)
Don't ignore VFS when it passes nosuid and nodev (ilovezfs)
Use a shared lock for online expansion (ilovezfs)
haer22
 
Posts: 123
Joined: Sun Mar 23, 2014 2:13 am

Re: 1.5.2 usage impact

Postby lundman » Sun May 01, 2016 6:52 pm

Google hits from the first two has

https://www.illumos.org/issues/4185

and

http://blog.delphix.com/matt/2015/03/25 ... ndreceive/

Complete with description, benchmarking, examples and slides. So those are the easiest for you to find more information on.

The O3X specific fixes would be harder to Google for, so let me know if you want to know more about any specific one.
User avatar
lundman
 
Posts: 1335
Joined: Thu Mar 06, 2014 2:05 pm
Location: Tokyo, Japan

Re: 1.5.2 usage impact

Postby haer22 » Sun May 01, 2016 11:48 pm

Thanks for the info (and "slap-to-the-head" :oops: )
haer22
 
Posts: 123
Joined: Sun Mar 23, 2014 2:13 am

Re: 1.5.2 usage impact

Postby haer22 » Mon May 02, 2016 12:41 am

Does this imply that one should change the checksum to Skein or Edon-R ?

The comment "lower security margin". Does that refer to the integrity of the data, i.e. the probability to notice a bit-error?

Code: Select all
This feature submission implements new hash algorithms into ZFS with improved performance:
SHA-512/256: 50% higher performance than SHA-256 on 64-bit hardware with minimum code changes.
Skein: 80% higher performance than SHA-256 with new and highly secure algorithm. Includes a KCF SW provider interface.
Edon-R: >350% higher performance than SHA-256. Lower security margin than Skein, but much higher throughput.
To address the potential security implications of Edon-R, this feature submission also implements salted checksums (aka MACs) in ZFS (both Skein and Edon-R utilize this).
Webrev at: http://cr.illumos.org/~webrev/skiselkov/new_hashes/
haer22
 
Posts: 123
Joined: Sun Mar 23, 2014 2:13 am

Re: 1.5.2 usage impact

Postby lundman » Mon May 02, 2016 12:48 am

I think with the new ciphers, I've heard them discussed with dedup in mind more than anything. Either way, they sound cool, so we have them :)
User avatar
lundman
 
Posts: 1335
Joined: Thu Mar 06, 2014 2:05 pm
Location: Tokyo, Japan

Re: 1.5.2 usage impact

Postby haer22 » Mon May 02, 2016 1:03 am

Yeah, I think I will run a "zdb -S gaia" and see how much I could save on dedups nowadays. Previously the cpu-usage went thru the roof.

lundman wrote:I think with the new ciphers, I've heard them discussed with dedup in mind more than anything. Either way, they sound cool, so we have them :)
haer22
 
Posts: 123
Joined: Sun Mar 23, 2014 2:13 am

Re: 1.5.2 usage impact

Postby haer22 » Mon May 02, 2016 1:16 am

The resume stuff just sounds WONDERFUL. I have a couple of 2-3 TB datasets that I shift around maybe once a year. And that is a painful period for me as the computers not always stay up for various reasons.

But, the man-page seems screwed up regarding the resume_token stuff. Seems like lines are gone for some reason. "man -c zfs" did not make any difference.

This is a copy of a portion of the output from "man zfs". Lines are obviously gone also around the explanations of "zfs send" and "zfs receive".

Does it look good on your systems?

Code: Select all
       mounted

           For file systems, indicates whether the file  system  is  currently
           mounted. This property can be either yes or no.


       origin

           For  cloned  file  systems  or volumes, the snapshot from which the
           clone was created. See also the clones property.
       For filesystems or volumes which have saved  partially-completed  state
       from this opaque token can be provided to to resume and complete the

       referenced

           The amount of data that is accessible by this dataset, which may or
           may not be shared with other datasets in the pool. When a  snapshot
           or  clone  is  created,  it initially references the same amount of
           space as the file system or snapshot it was created from, since its
           contents are identical.
haer22
 
Posts: 123
Joined: Sun Mar 23, 2014 2:13 am

Re: 1.5.2 usage impact

Postby lundman » Mon May 09, 2016 10:59 pm

We have a branch to lift in the original IllumOS man-pages again, without the ZOL changes, but they need to be cleaned up to add in ZOL changes :) For now, use the IllumOS man pages to read about resume
User avatar
lundman
 
Posts: 1335
Joined: Thu Mar 06, 2014 2:05 pm
Location: Tokyo, Japan

Re: 1.5.2 usage impact

Postby stumble » Wed May 11, 2016 2:15 am

stumble
 
Posts: 37
Joined: Thu May 15, 2014 7:05 pm

Re: 1.5.2 usage impact

Postby tangles » Wed Jun 08, 2016 1:30 am

so, should we suggest a "speed" config/option in the documentation of:
Code: Select all
 sudo zpool create -f -o ashift=12 -O compression=skein -O casesensitivity=insensitive -O atime=off -O normalization=formD tank mirror disk3 disk4


I saw that edon-R has some issues...
https://www.listbox.com/member/archive/ ... FD97918BF/

Mr Lundy? yr expertise please... :geek: :D
tangles
 
Posts: 195
Joined: Tue Jun 17, 2014 6:54 am

Next

Return to OpenZFS on OS X Development

Who is online

Users browsing this forum: No registered users and 13 guests