FYI: ZFS on Fusion Drive actually works.

Moderators: jhartley, MSR734, nola

FYI: ZFS on Fusion Drive actually works.

Post by jollyjinx » Wed Oct 31, 2012 12:02 pm

To spare someone the time to test:

I've been fiddling with Apples core storage layer (Fusion drive on old Macs - http://jollyjinx.tumblr.com) and now test ZFS on a fusion drive (FUD). I got ZFS using the FUD core storage and it is working by using first the SSD and then the HDD. Accessing data on the HDD will bring them eventually onto the SSD.

This might be a workaround of the limitation that a ZFS SSD cache disk is empty after reboot.
jollyjinx Offline


 
Posts: 60
Joined: Sun Sep 16, 2012 12:40 pm
Location: Munich - Germany

Re: FYI: ZFS on Fusion Drive actually works.

Post by markpaterson » Wed Oct 31, 2012 1:54 pm

cool. i'd love it if you could post a more detailed step by step how-to guide. I've read all your blog posts about this, and I'd like to try this out, but i'm not sure i follow everything you're doing.

my ZFS setup is currently 4 disk stripe of 2 mirrors (3TB & 3TB mirror, striped with a 3TB & 3TB mirror). I also have a 90GB SSD drive that is currently being used as a boot drive.

I'm prepared to move my boot drive to another drive so I can free up this SSD for use in a Fusion setup. Is there any way I could use my 4 disk striped mirror as a Fusion drive, with the SSD?
markpaterson Offline


 
Posts: 9
Joined: Fri Oct 05, 2012 1:48 pm

Re: FYI: ZFS on Fusion Drive actually works.

Post by grahamperrin » Wed Oct 31, 2012 5:06 pm

Nice work, thanks Patrick!

For the mix of ZFS with Apple CoreStorage, there's a closing line in one of my answers in Ask Different:

> Proceed with caution.

As far as I recall: whilst it's easy enough to give a CoreStorage logical volume to ZEVO, it was less easy to remove that type of thing from CoreStorage world. Something like that … in retrospect, the issue might have been the dynamic nature of ZEVO.
grahamperrin Offline

User avatar
 
Posts: 1596
Joined: Fri Sep 14, 2012 10:21 pm
Location: Brighton and Hove, United Kingdom

Re: FYI: ZFS on Fusion Drive actually works.

Post by grahamperrin » Wed Oct 31, 2012 5:46 pm

markpaterson wrote:… prepared to move my boot drive to another drive so I can free up this SSD for use in a Fusion setup. … 


You might get more milage from using part of the SSD as L2ARC or ZIL (not moving the startup volume away from that SSD).
grahamperrin Offline

User avatar
 
Posts: 1596
Joined: Fri Sep 14, 2012 10:21 pm
Location: Brighton and Hove, United Kingdom

Re: FYI: ZFS on Fusion Drive actually works.

Post by grahamperrin » Thu Nov 01, 2012 1:49 am

Patrick, please … when you next have the CoreStorage LV given to ZEVO, post to Tumblr (or here) the outputs from:

Code: Select all
diskutil coreStorage list


– and assuming that disk numbers are as they were in your third post,

Code: Select all
diskutil list disk1 && diskutil list disk7 && diskutil list disk8


Thanks.

In particular, the fuller tree view of your CoreStorage world might help to visualise what I'm trying to explain at viewtopic.php?p=1349#p1349

(Anything else to add?)
grahamperrin Offline

User avatar
 
Posts: 1596
Joined: Fri Sep 14, 2012 10:21 pm
Location: Brighton and Hove, United Kingdom

Re: FYI: ZFS on Fusion Drive actually works.

Post by jollyjinx » Thu Nov 01, 2012 10:17 am

I've rebooted and plugged in the disks on USB, now disk14 15 and 16 ( were 1 7 8 )
ZFS automatically mounted the zfs filesystem on them :-)

Code: Select all
   TIN>diskutil cs list
   CoreStorage logical volume groups (1 found)
   |
   +-- Logical Volume Group DE85044F-EADA-4F26-93B7-8CD0ADF006EC
      =========================================================
      Name:         bla
      Size:         869502550016 B (869.5 GB)
      Free Space:   365776592896 B (365.8 GB)
      |
      +-< Physical Volume 682DCC34-74A4-4290-80AE-EB127BA24746
      |   ----------------------------------------------------
      |   Index:    0
      |   Disk:     disk14s2
      |   Status:   Online
      |   Size:     119690149888 B (119.7 GB)
      |
      +-< Physical Volume 5FA828A9-3EDD-4CBC-8C93-27C0E07C2E8A
      |   ----------------------------------------------------
      |   Index:    1
      |   Disk:     disk15s2
      |   Status:   Online
      |   Size:     749812400128 B (749.8 GB)
      |
      +-> Logical Volume Family 3D387875-A26D-4F4A-AB8C-DEBC0CC2EFB8
         ----------------------------------------------------------
         Encryption Status:       Unlocked
         Encryption Type:         None
         Conversion Status:       NoConversion
         Conversion Direction:    -none-
         Has Encrypted Extents:   No
         Fully Secure:            No
         Passphrase Required:     No
         |
         +-> Logical Volume 6CBDCB9F-A08E-4E79-BDA7-69853BA24C1E
            ---------------------------------------------------
            Disk:               disk16
            Status:             Online
            Size (Total):       500000096256 B (500.0 GB)
            Size (Converted):   -none-
            Revertible:         No
            LV Name:            blub
            Content Hint:       Apple_HFS

   
   TIN>diskutil list disk14
   /dev/disk14
      #:                       TYPE NAME                    SIZE       IDENTIFIER
      0:      GUID_partition_scheme                        *120.0 GB   disk14
      1:                        EFI                         209.7 MB   disk14s1
      2:          Apple_CoreStorage                         119.7 GB   disk14s2
      3:                 Apple_Boot Boot OS X               134.2 MB   disk14s3
   TIN>diskutil list disk15
   /dev/disk15
      #:                       TYPE NAME                    SIZE       IDENTIFIER
      0:      GUID_partition_scheme                        *750.2 GB   disk15
      1:                        EFI                         209.7 MB   disk15s1
      2:          Apple_CoreStorage                         749.8 GB   disk15s2
      3:                 Apple_Boot Boot OS X               134.2 MB   disk15s3
   TIN>diskutil list disk16
   /dev/disk16
      #:                       TYPE NAME                    SIZE       IDENTIFIER
      0:      GUID_partition_scheme                        *500.0 GB   disk16
      1:                        EFI                         209.7 MB   disk16s1
      2:                        ZFS                         499.7 GB   disk16s2

jollyjinx Offline


 
Posts: 60
Joined: Sun Sep 16, 2012 12:40 pm
Location: Munich - Germany

Re: FYI: ZFS on Fusion Drive actually works.

Post by grahamperrin » Thu Nov 01, 2012 12:15 pm

Perfect, thanks. And I for the ZFS slice at disk16s2, the ZEVO zfs_pool_proxy might be disk17.

http://diigo.com/0u1wo for highlights from Apple Fusion Drive—wait, what? How does this work? | Ars Technica (2012-10-23) … whilst Lee Hutchinson seems almost certain that implementation is file-level, I am not certain of that.

As CoreStorage is for management of logical volumes – and as Patrick demonstrates a file system without HFS Plus working with CoreStorage in ways that are Fusion Drive-like – my assumption is:

  • block-level.

If only a few blocks of a colossal file are used frequently, it will make sense for CoreStorage to:

  • move only those few blocks (not the colossus) to the faster tier, to the faster physical volume(s).

Potential for confusion

Apple Fulfills Patent with the iMac's New Fusion Drive - Patently Apple (2012-10-23) – I don't think so. In a 2011 article about Apple's patent application, Patently Apple observed: "… Apple's twist to this drive is that it will handle different types of storage media based on an environmental state of the hybrid drive. If the drive undergoes temperature, vibration or acceleration variances that could cause the hard drive to temporarily fail, the system would save the volatile data to the Flash drive …" and "… Apple's invention generally relates to a hybrid storage device that includes the following: a hard-disk drive (HDD), a flash memory, an interface circuit, and control logic. This control logic receives a request … from an external device to write a block of data to the hybrid … includes a logical address for the block of data in a first portion of an address space which maps to the HDD. In response to the request, the control logic writes the block of data to the HDD. … at least a portion of the block of data to a logical address for the block of data in a second portion of the address space which maps to the flash memory. …" 

– I do not see such things in Apple's Fusion Drive application of CoreStorage.
grahamperrin Offline

User avatar
 
Posts: 1596
Joined: Fri Sep 14, 2012 10:21 pm
Location: Brighton and Hove, United Kingdom

Re: FYI: ZFS on Fusion Drive actually works.

Post by jollyjinx » Thu Nov 01, 2012 2:29 pm

@grahamperrin I've already tested and proven that it's blocklevel in one of my http://jollyjinx.tumblr.com blog posts ;-)
jollyjinx Offline


 
Posts: 60
Joined: Sun Sep 16, 2012 12:40 pm
Location: Munich - Germany

Re: FYI: ZFS on Fusion Drive actually works.

Post by grahamperrin » Thu Nov 01, 2012 2:45 pm

From Patrick's Fusion Drive - loose ends, with OS X 10.8.2:

Code: Select all
          disk1           disk7       cpu     load average
    KB/t tps  MB/s     KB/t tps  MB/s  us sy id   1m   5m   15m
  124.10 1738 210.58     0.00   0  0.00  15  6 78  1.15 0.98 0.99
  124.46 1731 210.41     0.00   0  0.00  16  6 78  1.15 0.98 0.99
  122.01 2006 239.05     0.00   0  0.00  17  8 75  1.21 1.00 0.99
  104.96 2119 217.15     0.00   0  0.00  16  7 77  1.21 1.00 0.99
  101.27 2105 208.22     0.00   0  0.00  17  8 75  1.21 1.00 0.99


I wonder whether similar I/O patterns will be noticeable with ZEVO ZFS with a CoreStorage LV comprising solid state and hard disk drives on Lion.
grahamperrin Offline

User avatar
 
Posts: 1596
Joined: Fri Sep 14, 2012 10:21 pm
Location: Brighton and Hove, United Kingdom

Re: FYI: ZFS on Fusion Drive actually works.

Post by grahamperrin » Fri Nov 02, 2012 1:25 am

>> Proceed with caution.

From a collection of files made this morning (2012-02-11), here's a CoreStorage logical volume group (LVG) apparently initialising long after removal of the physical device:

Image

Removal of the device was preceded by destruction of a ZFS pool. This morning's bug was probably limited to appearance. No significant disruption to workflow.

Yesterday, however, comparable tests led to something more concrete that caused the MacBookPro5,2 to not shut down; force was required.

Whatever the causes, this post should not raise alarm – for these test results, I worked with simple removable USB flash drives.

An approach that is more Fusion Drive-like – with a permanently connected SSD – should be less prone to peculiarities.
grahamperrin Offline

User avatar
 
Posts: 1596
Joined: Fri Sep 14, 2012 10:21 pm
Location: Brighton and Hove, United Kingdom

Next

Return to General Discussion

Who is online

Users browsing this forum: ilovezfs and 1 guest

cron