some actions take longer than expected to complete

This forum is to find answers to problems you may be having with ZEVO Community Edition.

Moderators: jhartley, MSR734, nola

some actions take longer than expected to complete

Post by grahamperrin » Fri Dec 28, 2012 2:49 am

Spun off from a general discussion, sparsebundles w/ JHFS+ on top of ZFS

An example of a command that seems to run for longer than expected

Code: Select all
ls -ahlR ~/Library


First run, 11:12 until ~17:46 = something over six hours.

Second run, 17:48 until ~23:45 = something over five hours.

Most noticeable:

  • pauses before listings for directories that have relatively high populations.

Screenshots from the first run

Timings above are approximate – I didn't watch Terminal non-stop. Maybe subtract half an hour or so from the second sum. Also, I used the Mac extensively during both runs. I could run a better structured command in a controlled environment to get exact timings but the gist is: too long.

For me, what's above is currently not a problem. For other use cases, the lack of performance is so problematic that HFS Plus is preferred, so this is troubleshooting. #zevo

Environment

MacBookPro5,2 with 8 GB memory and Seagate Momentus® XT ST750LX003-1AC154 solid state hybrid drive.

The pool for my home directory is part of a disk that is shared with HFS Plus. So there'll be latency, but I doubt that this alone explains the apparent periods of slowness.

Code: Select all
macbookpro08-centrim:~ gjp22$ date
Fri 28 Dec 2012 07:11:52 GMT
macbookpro08-centrim:~ gjp22$ sudo du -hs ~/Library
Password:
 60G   /Volumes/gjp22/Library


Code: Select all
macbookpro08-centrim:~ gjp22$ diskutil list
/dev/disk0
   #:                       TYPE NAME                    SIZE       IDENTIFIER
   0:      GUID_partition_scheme                        *750.2 GB   disk0
   1:                        EFI                         209.7 MB   disk0s1
   2:                  Apple_HFS swap                    32.0 GB    disk0s2
   3: FFFFFFFF-FFFF-FFFF-FFFF-FFFFFFFFFFFF               536.9 MB   disk0s3
   4:                  Apple_HFS spare                   671.1 MB   disk0s4
   5:          Apple_CoreStorage                         99.5 GB    disk0s5
   6:                 Apple_Boot Boot OS X               650.0 MB   disk0s6
   7:          Apple_CoreStorage                         616.3 GB   disk0s7
   8:                 Apple_Boot Boot OS X               134.2 MB   disk0s8
/dev/disk1
   #:                       TYPE NAME                    SIZE       IDENTIFIER
   0:                  Apple_HFS OS                     *99.2 GB    disk1
/dev/disk2
   #:                       TYPE NAME                    SIZE       IDENTIFIER
   0:      GUID_partition_scheme                        *616.0 GB   disk2
   1:                        EFI                         209.7 MB   disk2s1
   2:                        ZFS                         615.7 GB   disk2s2
/dev/disk3
   #:                       TYPE NAME                    SIZE       IDENTIFIER
   0:             zfs_pool_proxy gjp22                  *614.2 GB   disk3
   1:       zfs_filesystem_proxy casesensitive           163.6 GB   disk3s1
   2:       zfs_filesystem_proxy opt                     170.1 GB   disk3s2
/dev/disk4
   #:                       TYPE NAME                    SIZE       IDENTIFIER
   0:      GUID_partition_scheme                        *640.1 GB   disk4
   1:                        EFI                         209.7 MB   disk4s1
   2:                        ZFS                         639.8 GB   disk4s2
/dev/disk5
   #:                       TYPE NAME                    SIZE       IDENTIFIER
   0:             zfs_pool_proxy zhandy                 *635.7 GB   disk5
   1:       zfs_filesystem_proxy Pocket Time Machine     173.0 GB   disk5s1
/dev/disk6
   #:                       TYPE NAME                    SIZE       IDENTIFIER
   0:      GUID_partition_scheme                        *2.0 TB     disk6
   1:                        EFI                         209.7 MB   disk6s1
   2:                        ZFS                         2.0 TB     disk6s2
/dev/disk7
   #:                       TYPE NAME                    SIZE       IDENTIFIER
   0:             zfs_pool_proxy tall                   *2.0 TB     disk7
/dev/disk8
   #:                       TYPE NAME                    SIZE       IDENTIFIER
   0:     Apple_partition_scheme                        *295.3 GB   disk8
   1:        Apple_partition_map                         32.3 KB    disk8s1
   2:                  Apple_HFS AppleSeed               295.3 GB   disk8s2
/dev/disk9
   #:                       TYPE NAME                    SIZE       IDENTIFIER
   0:      GUID_partition_scheme                        *1.0 TB     disk9


Code: Select all
macbookpro08-centrim:~ gjp22$ diskutil coreStorage list
CoreStorage logical volume groups (2 found)
|
+-- Logical Volume Group 039C0D47-F3CF-44D3-A825-B48F01FCF334
|   =========================================================
|   Name:         OS
|   Status:       Online
|   Size:         99484213248 B (99.5 GB)
|   Free Space:   0 B (0 B)
|   |
|   +-< Physical Volume 86D2FA98-8C69-4828-B909-8312AE4A75E2
|   |   ----------------------------------------------------
|   |   Index:    0
|   |   Disk:     disk0s5
|   |   Status:   Online
|   |   Size:     99484213248 B (99.5 GB)
|   |
|   +-> Logical Volume Family 7AA96B8E-0E41-4C3F-9589-5FAE0C956372
|       ----------------------------------------------------------
|       Encryption Status:       Unlocked
|       Encryption Type:         AES-XTS
|       Conversion Status:       Complete
|       Conversion Direction:    -none-
|       Has Encrypted Extents:   Yes
|       Fully Secure:            Yes
|       Passphrase Required:     Yes
|       |
|       +-> Logical Volume B13EE5BF-5D08-49D3-94C2-DF58AFEA1D08
|           ---------------------------------------------------
|           Disk:               disk1
|           Status:             Online
|           Size (Total):       99165437952 B (99.2 GB)
|           Size (Converted):   -none-
|           Revertible:         No
|           LV Name:            OS
|           Volume Name:        OS
|           Content Hint:       Apple_HFS
|
+-- Logical Volume Group 902434C9-0131-4E3A-AE15-2B8B938087AD
    =========================================================
    Name:         gjp22-cs
    Status:       Online
    Size:         616336003072 B (616.3 GB)
    Free Space:   0 B (0 B)
    |
    +-< Physical Volume 179AADE6-34F1-404C-A994-9FD99C881BA6
    |   ----------------------------------------------------
    |   Index:    0
    |   Disk:     disk0s7
    |   Status:   Online
    |   Size:     616336003072 B (616.3 GB)
    |
    +-> Logical Volume Family FFCE2FAF-BE8E-4FEF-9F3E-E221C6CBCA11
        ----------------------------------------------------------
        Encryption Status:       Unlocked
        Encryption Type:         AES-XTS
        Conversion Status:       Complete
        Conversion Direction:    -none-
        Has Encrypted Extents:   Yes
        Fully Secure:            Yes
        Passphrase Required:     Yes
        |
        +-> Logical Volume 0CFAFD38-E79B-40AC-A4BE-63296E6B4331
            ---------------------------------------------------
            Disk:               disk2
            Status:             Online
            Size (Total):       616017227776 B (616.0 GB)
            Size (Converted):   -none-
            Revertible:         No
            LV Name:            gjp22-cs
            Content Hint:       Apple_HFS


Code: Select all
macbookpro08-centrim:~ gjp22$ zpool status -v
  pool: gjp22
 state: ONLINE
 scan: scrub repaired 0 in 27h51m with 0 errors on Tue Dec 25 13:52:01 2012
config:

   NAME                                         STATE     READ WRITE CKSUM
   gjp22                                        ONLINE       0     0     0
     GPTE_71B8BDA2-3EBA-4B91-9E1C-2AE2B1DAAD06  ONLINE       0     0     0  at disk2s2
   cache
     GPTE_64F61AFF-9EBC-4661-9520-7803CD1B8EE4  FAULTED     27   154     0  too many errors

errors: No known data errors

  pool: tall
 state: ONLINE
 scan: scrub repaired 0 in 57h12m with 0 errors on Wed Dec 19 21:00:26 2012
config:

   NAME                                         STATE     READ WRITE CKSUM
   tall                                         ONLINE       0     0     0
     GPTE_78301A52-4AFF-4D96-8DE9-E76ABC14909C  ONLINE       0     0     0  at disk6s2

errors: No known data errors

  pool: zhandy
 state: ONLINE
 scan: scrub canceled on Mon Dec 17 20:29:47 2012
config:

   NAME                                         STATE     READ WRITE CKSUM
   zhandy                                       ONLINE       0     0     0
     GPTE_A54431D5-B46F-44A9-83B4-76802A584C6E  ONLINE       0     0     0  at disk4s2

errors: No known data errors


Code: Select all
macbookpro08-centrim:~ gjp22$ zpool list
NAME      SIZE   ALLOC    FREE     CAP  HEALTH  ALTROOT
gjp22    572Gi   415Gi   157Gi     72%  ONLINE  -
tall    1.82Ti  1.78Ti  38.6Gi     97%  ONLINE  -
zhandy   592Gi   459Gi   133Gi     77%  ONLINE  -


Code: Select all
macbookpro08-centrim:~ gjp22$ zfs list
NAME                                       USED   AVAIL   REFER  MOUNTPOINT
gjp22                                     415Gi   148Gi   362Gi  /Volumes/gjp22
gjp22/casesensitive                       158Mi   148Gi   128Mi  /Volumes/gjp22/casesensitive
gjp22/opt                                7.41Gi   148Gi  6.21Gi  /opt
tall                                     1.78Ti  9.46Gi   440Gi  /Volumes/tall
tall/backups                              906Gi  9.46Gi  10.9Gi  /Volumes/tall/backups
tall/backups/LaCie d2 Extreme            2.96Mi  9.46Gi   688Ki  /Volumes/tall/backups/LaCie d2 Extreme
tall/backups/LaCie d2 Extreme/11G        1.16Mi  9.46Gi  1.16Mi  /Volumes/tall/backups/LaCie d2 Extreme/11G
tall/backups/LaCie d2 Extreme/12A        1.13Mi  9.46Gi  1.13Mi  /Volumes/tall/backups/LaCie d2 Extreme/12A
tall/backups/blocky                      3.31Gi  9.46Gi  3.31Gi  /Volumes/tall/backups/blocky
tall/backups/gjp22                        413Gi  9.46Gi   354Gi  /Volumes/tall/backups/gjp22
tall/backups/gjp22/casesensitive          157Mi  9.46Gi   128Mi  /Volumes/tall/backups/gjp22/casesensitive
tall/backups/gjp22/opt                   7.42Gi  9.46Gi  6.21Gi  /Volumes/tall/backups/gjp22/opt
tall/backups/zhandy                       479Gi  9.46Gi   433Gi  /Volumes/tall/backups/zhandy
tall/backups/zhandy/Pocket Time Machine  40.9Gi  9.46Gi  40.9Gi  /Volumes/tall/backups/zhandy/Pocket Time Machine
zhandy                                    459Gi   124Gi   415Gi  /Volumes/zhandy
zhandy/Pocket Time Machine               39.3Gi   124Gi  39.3Gi  /Volumes/zhandy/Pocket Time Machine



Code: Select all
macbookpro08-centrim:~ gjp22$ mount
/dev/disk1 on / (hfs, local, journaled)
devfs on /dev (devfs, local, nobrowse)
/dev/disk0s2 on /Volumes/swap (hfs, local, journaled)
/dev/disk0s4 on /Volumes/spare (hfs, local, journaled)
map -hosts on /net (autofs, nosuid, automounted, nobrowse)
map auto_home on /home (autofs, automounted, nobrowse)
localhost:/S2A_YAK5uFZEiqiCjA1paV on /Volumes/MobileBackups (mtmfs, nosuid, read-only, nobrowse)
/dev/disk3 on /Volumes/gjp22 (zfs, local, journaled, noatime)
/dev/disk3s1 on /Volumes/gjp22/casesensitive (zfs, local, journaled, noatime)
/dev/disk3s2 on /opt (zfs, local, journaled, noatime)
/dev/disk5 on /Volumes/zhandy (zfs, local, journaled, noatime)
/dev/disk5s1 on /Volumes/zhandy/Pocket Time Machine (zfs, local, journaled, noatime)
/dev/disk7 on /Volumes/tall (zfs, local, journaled, noatime)
/dev/disk8s2 on /Volumes/AppleSeed (hfs, local, nodev, nosuid, journaled, noowners, mounted by gjp22)
Wuala on /Volumes/WualaDrive (osxfusefs, local, nodev, nosuid, synchronous, mounted by gjp22)
grahamperrin Offline

User avatar
 
Posts: 1596
Joined: Fri Sep 14, 2012 10:21 pm
Location: Brighton and Hove, United Kingdom

Visualisations

Post by grahamperrin » Sat Dec 29, 2012 11:38 am

Disk Inventory X visualisations of the 60G ~/Library that featured in the example above
grahamperrin Offline

User avatar
 
Posts: 1596
Joined: Fri Sep 14, 2012 10:21 pm
Location: Brighton and Hove, United Kingdom

Thoughts

Post by grahamperrin » Sun Jan 13, 2013 5:41 pm

With HFS Plus, there's the catalog file and so on. Ready made, if you like.

As I understand it … with ZFS there's nothing like that catalogue. And so, more need to traverse things.

With enough L2ARC and with enough memory, I guess that a completed scrub of a pool would populate L2ARC in a way that should allow much faster listings.
grahamperrin Offline

User avatar
 
Posts: 1596
Joined: Fri Sep 14, 2012 10:21 pm
Location: Brighton and Hove, United Kingdom

Re: some actions take longer than expected to complete

Post by emory » Mon Jan 14, 2013 12:05 pm

In your opinion is it better to use a 120GB SSD (3Gbps) as an l2arc for a ZEVO storage pool, or a ZIL? I have a spare SSD after an upgrade and I used it for a DIY Fusion Drive, but I'm a little skittish of it.
emory Offline


 
Posts: 15
Joined: Mon Sep 17, 2012 7:47 pm

Re: some actions take longer than expected to complete

Post by mk01 » Mon Jan 14, 2013 6:32 pm

it would depend on amount of io caused by sync write.
mk01 Offline


 
Posts: 65
Joined: Mon Sep 17, 2012 1:16 am

Correction

Post by grahamperrin » Sun May 19, 2013 6:42 am

grahamperrin wrote:… I guess that a completed scrub of a pool would populate L2ARC in a way that should allow much faster listings.


That guess is probably wrong. viewtopic.php?p=4582#p4582 reminds me:

raattgift wrote:… Scrubbing and resilvering do not use the normal read pipeline …
grahamperrin Offline

User avatar
 
Posts: 1596
Joined: Fri Sep 14, 2012 10:21 pm
Location: Brighton and Hove, United Kingdom


Return to Troubleshooting

Who is online

Users browsing this forum: hlxpgxmum and 1 guest

cron