DiskImages framework, diskimages-helper and open files

This forum is to find answers to problems you may be having with ZEVO Community Edition.

Moderators: jhartley, MSR734, nola

DiskImages framework, diskimages-helper and open files

Post by grahamperrin » Thu Nov 07, 2013 4:03 pm

For reference only

If you use disk images that use Apple's private DiskImages framework:

  • beware of regular files and directory files remaining open longer than expected.

For example, after detaching a sparse bundle disk image:

Code: Select all
gpes3e-gjp4:~ gjp22$ sudo lsof /Volumes/tall/com.apple.backupd/
Password:
COMMAND    PID USER   FD   TYPE DEVICE SIZE/OFF   NODE NAME
mds         62 root  144r   DIR   1,24       10      2 /Volumes/tall/com.apple.backupd
diskimage 9903 root    4u   REG   1,24  8388608 128745 /Volumes/tall/com.apple.backupd/gpes3e-gjp4.sparsebundle/bands/347a4
diskimage 9903 root    5u   REG   1,24        0     23 /Volumes/tall/com.apple.backupd/gpes3e-gjp4.sparsebundle/token
diskimage 9903 root    6r   DIR   1,24        2     19 /Volumes/tall/com.apple.backupd/gpes3e-gjp4.sparsebundle
diskimage 9903 root    7r   DIR   1,24        2     22 /Volumes/tall/com.apple.backupd/gpes3e-gjp4.sparsebundle/bands
diskimage 9903 root    9u   REG   1,24  8388608 128420 /Volumes/tall/com.apple.backupd/gpes3e-gjp4.sparsebundle/bands/348eb
diskimage 9903 root   10u   REG   1,24  8388608 127380 /Volumes/tall/com.apple.backupd/gpes3e-gjp4.sparsebundle/bands/34a45
diskimage 9903 root   11u   REG   1,24  8388608 128644 /Volumes/tall/com.apple.backupd/gpes3e-gjp4.sparsebundle/bands/3480b
diskimage 9903 root   12u   REG   1,24  8388608 128409 /Volumes/tall/com.apple.backupd/gpes3e-gjp4.sparsebundle/bands/348f4
diskimage 9903 root   13u   REG   1,24  8388608 128639 /Volumes/tall/com.apple.backupd/gpes3e-gjp4.sparsebundle/bands/3480e
diskimage 9903 root   14u   REG   1,24  8388608 128889 /Volumes/tall/com.apple.backupd/gpes3e-gjp4.sparsebundle/bands/34714
diskimage 9903 root   15u   REG   1,24  8388608 129216 /Volumes/tall/com.apple.backupd/gpes3e-gjp4.sparsebundle/bands/345cf
diskimage 9903 root   16u   REG   1,24  8388608 128657 /Volumes/tall/com.apple.backupd/gpes3e-gjp4.sparsebundle/bands/347fc
diskimage 9903 root   17u   REG   1,24    53248     25 /Volumes/tall/com.apple.backupd/gpes3e-gjp4.sparsebundle/bands/19
diskimage 9903 root   18u   REG   1,24  8388608 128429 /Volumes/tall/com.apple.backupd/gpes3e-gjp4.sparsebundle/bands/348e0
diskimage 9903 root   19u   REG   1,24  8388608 128214 /Volumes/tall/com.apple.backupd/gpes3e-gjp4.sparsebundle/bands/349b9
diskimage 9903 root   20u   REG   1,24  8388608 128932 /Volumes/tall/com.apple.backupd/gpes3e-gjp4.sparsebundle/bands/346eb
diskimage 9903 root   21u   REG   1,24   692224 135263 /Volumes/tall/com.apple.backupd/gpes3e-gjp4.sparsebundle/bands/3c73e
diskimage 9903 root   22u   REG   1,24  8388608 128916 /Volumes/tall/com.apple.backupd/gpes3e-gjp4.sparsebundle/bands/346fb
diskimage 9903 root   23u   REG   1,24  8388608 129271 /Volumes/tall/com.apple.backupd/gpes3e-gjp4.sparsebundle/bands/34596
diskimage 9903 root   24u   REG   1,24  8388608 129213 /Volumes/tall/com.apple.backupd/gpes3e-gjp4.sparsebundle/bands/345d0
gpes3e-gjp4:~ gjp22$


Scope

When I fed a related bug to Apple, early this year, the disk image was stored on ZFS (ZEVO).

Notes from mid-2011 suggest that the same symptoms can occur with images stored on JHFS+.

Whilst diskimages-helper is prominent (abbreviated to diskimage in output from lsof), we should not assume a bug in DiskImages framework. The bug may be elsewhere.

Workaround

For the already detached image, use hdiutil(1):

  1. attach the image
  2. detach the image

– it may help to perform both steps in relatively quick succession. In most cases I find that the command line approach detachment leaves no files unexpectedly open.

Exceptionally

It's possible to remove (rm) a sparse bundle disk image without realising that files within the image are open.

A more detailed view of the example above:

Code: Select all
gpes3e-gjp4:~ gjp22$ zpool export tall
cannot unmount '/Volumes/tall/com.apple.backupd': Operation not permitted
gpes3e-gjp4:~ gjp22$ zpool export tall
cannot unmount '/Volumes/tall/com.apple.backupd': Operation not permitted
gpes3e-gjp4:~ gjp22$
gpes3e-gjp4:~ gjp22$ sudo lsof /Volumes/tall/com.apple.backupd/
Password:
COMMAND    PID USER   FD   TYPE DEVICE SIZE/OFF   NODE NAME
mds         62 root  144r   DIR   1,24       10      2 /Volumes/tall/com.apple.backupd
diskimage 9903 root    4u   REG   1,24  8388608 128745 /Volumes/tall/com.apple.backupd/gpes3e-gjp4.sparsebundle/bands/347a4
diskimage 9903 root    5u   REG   1,24        0     23 /Volumes/tall/com.apple.backupd/gpes3e-gjp4.sparsebundle/token
diskimage 9903 root    6r   DIR   1,24        2     19 /Volumes/tall/com.apple.backupd/gpes3e-gjp4.sparsebundle
diskimage 9903 root    7r   DIR   1,24        2     22 /Volumes/tall/com.apple.backupd/gpes3e-gjp4.sparsebundle/bands
diskimage 9903 root    9u   REG   1,24  8388608 128420 /Volumes/tall/com.apple.backupd/gpes3e-gjp4.sparsebundle/bands/348eb
diskimage 9903 root   10u   REG   1,24  8388608 127380 /Volumes/tall/com.apple.backupd/gpes3e-gjp4.sparsebundle/bands/34a45
diskimage 9903 root   11u   REG   1,24  8388608 128644 /Volumes/tall/com.apple.backupd/gpes3e-gjp4.sparsebundle/bands/3480b
diskimage 9903 root   12u   REG   1,24  8388608 128409 /Volumes/tall/com.apple.backupd/gpes3e-gjp4.sparsebundle/bands/348f4
diskimage 9903 root   13u   REG   1,24  8388608 128639 /Volumes/tall/com.apple.backupd/gpes3e-gjp4.sparsebundle/bands/3480e
diskimage 9903 root   14u   REG   1,24  8388608 128889 /Volumes/tall/com.apple.backupd/gpes3e-gjp4.sparsebundle/bands/34714
diskimage 9903 root   15u   REG   1,24  8388608 129216 /Volumes/tall/com.apple.backupd/gpes3e-gjp4.sparsebundle/bands/345cf
diskimage 9903 root   16u   REG   1,24  8388608 128657 /Volumes/tall/com.apple.backupd/gpes3e-gjp4.sparsebundle/bands/347fc
diskimage 9903 root   17u   REG   1,24    53248     25 /Volumes/tall/com.apple.backupd/gpes3e-gjp4.sparsebundle/bands/19
diskimage 9903 root   18u   REG   1,24  8388608 128429 /Volumes/tall/com.apple.backupd/gpes3e-gjp4.sparsebundle/bands/348e0
diskimage 9903 root   19u   REG   1,24  8388608 128214 /Volumes/tall/com.apple.backupd/gpes3e-gjp4.sparsebundle/bands/349b9
diskimage 9903 root   20u   REG   1,24  8388608 128932 /Volumes/tall/com.apple.backupd/gpes3e-gjp4.sparsebundle/bands/346eb
diskimage 9903 root   21u   REG   1,24   692224 135263 /Volumes/tall/com.apple.backupd/gpes3e-gjp4.sparsebundle/bands/3c73e
diskimage 9903 root   22u   REG   1,24  8388608 128916 /Volumes/tall/com.apple.backupd/gpes3e-gjp4.sparsebundle/bands/346fb
diskimage 9903 root   23u   REG   1,24  8388608 129271 /Volumes/tall/com.apple.backupd/gpes3e-gjp4.sparsebundle/bands/34596
diskimage 9903 root   24u   REG   1,24  8388608 129213 /Volumes/tall/com.apple.backupd/gpes3e-gjp4.sparsebundle/bands/345d0
gpes3e-gjp4:~ gjp22$ ls -l /Volumes/tall/com.apple.backupd/
total 16
drwx------@ 3 gjp22  staff  9 19 Oct 22:14 Mavericks-VirtualBoxVM.sparsebundle
gpes3e-gjp4:~ gjp22$


The temptation was to kill the PID (9903) for diskimages-helper but from past experience, I guessed that doing so might not allow me to export the pool.

I opted to restart the Mac.
grahamperrin Offline

User avatar
 
Posts: 1596
Joined: Fri Sep 14, 2012 10:21 pm
Location: Brighton and Hove, United Kingdom

Return to Troubleshooting

Who is online

Users browsing this forum: bileyqrkq, ilovezfs, toduefiwu and 0 guests

cron