recover half mirror

[lundman]

All writes happen to all vdevs in a mirror, at the same time, so it is hard to imagine this situation to happen. If it was a failing HDD maybe, but in this case, you just removed a mirror. Did you try the -T txg import? Since you can import txg-1, and therefore step one txg back into the past, maybe up to -16, and see if it was a recent write. You could also send the data, and turn on the zfs send ignore-errors tunable.

[patmuk]

Yes, I just removed the drive. Should have used

Code: Select all

split

, but didn't thought about that.

Thanks a bunch for your suggestions. They didn't work, but maybe I understood them wrong?

Code: Select all

sanpat@SanPat-Mac-mini ~> sudo zpool import -o readonly=on -T 16 datapool1 d16
cannot import 'datapool1' as 'd16': one or more devices is currently unavailable

sanpat@SanPat-Mac-mini ~> sudo zpool import -o readonly=on -T 16 -V datapool1 d16
sanpat@SanPat-Mac-mini ~> sudo zpool list
NAME   SIZE  ALLOC   FREE  CKPOINT  EXPANDSZ   FRAG    CAP  DEDUP  HEALTH  ALTROOT
d16       -      -      -        -         -      -      -      -  FAULTED  -
sanpat@SanPat-Mac-mini ~> sudo zfs mount d16
cannot open 'd16': I/O error


And send

Code: Select all

sanpat@SanPat-Mac-mini ~> sudo zfs send -ignore-errors d16 >> t.txt
cannot open 'd16': I/O error


Did I misunderstood you?

[lundman]

Ah so the rolling back of txg, you first need to lookup the current txg, which you can do with zdb $pool | grep txg. Say it is 123456. Then you try import -T 123455 - ie, one less, to go back one write, in case that last write was what made it wrong.
You can go back as many as there are uberblocks, so something like 16.

The send, you need to set
sysctl kstat.zfs.darwin.tunable.zfs_send_corrupt_data=1

then issue the send.

[patmuk]

First of all: Thanks a lot for your continuing support and not giving up yet )

I still could not make it work, here are the responese:

Code: Select all

sanpat@SanPat-Mac-mini ~> sudo zdb -e -d datapool1
zdb: can't open 'datapool1': Device not configured


It works for the pool "secure"

Code: Select all

sanpat@SanPat-Mac-mini ~> sudo sysctl kstat.zfs.darwin.tunable.zfs_send_corrupt_data=1
kstat.zfs.darwin.tunable.zfs_send_corrupt_data: 0 -> 1
sanpat@SanPat-Mac-mini ~>  sudo zfs send -ignore-errors datapool1 >> t.txt
cannot open 'datapool1': I/O error


I can´t export the pool as well

Code: Select all

sudo zpool export -f datapool1
cannot export 'datapool1': one or more devices is currently unavailable


Any other options coming to your mind? Can I force zdb in some way?

[lundman]

Yes, you can use "-l" with zdb and feed it a /dev/disk entry instead, and it reads the label off the disk.

[patmuk]

Thanks! Looks like I am getting closer - but I still get the I/O error.
It says that no trusted uberblck was found ...

Code: Select all

sanpat@SanPat-Mac-mini ~> sudo zdb -e datapool1 -l distk2s1

Configuration for import:
        vdev_children: 1
        version: 5000
        pool_guid: 3571308118281011652
        name: 'datapool1'
        txg: 18742113
        state: 0
        hostid: 2779865448
        hostname: 'Patricks-Mac-mini-2.local'
        vdev_tree:
            type: 'root'
            id: 0
            guid: 3571308118281011652
            children[0]:
                type: 'mirror'
                id: 0
                guid: 6299982245232111674
                whole_disk: 0
                metaslab_array: 34
                metaslab_shift: 35
                ashift: 12
                asize: 4000771997696
                is_log: 0
                create_txg: 4
                children[0]:
                    type: 'disk'
                    id: 0
                    guid: 6980676228928256395
                    whole_disk: 1
                    DTL: 229
                    create_txg: 4
                    path: '/private/var/run/disk/by-id/media-5DF040D0-4447-F94F-9F4D-C7C61EF19B7D'
                children[1]:
                    type: 'disk'
                    id: 1
                    guid: 1265025113965553803
                    path: '/private/var/run/disk/by-id/media-47DBA685-EEE4-7842-B437-1347665B384F'
                    whole_disk: 1
                    not_present: 1
                    DTL: 76
                    create_txg: 4
zdb: can't open 'datapool1': Device not configured

ZFS_DBGMSG(zdb):
spa_import: importing datapool1
spa_load(datapool1, config trusted): LOADING
spa_load(datapool1, config untrusted): FAILED: no valid uberblock found
spa_load(datapool1, config untrusted): UNLOADING


I tried importing it with txg 18742113 - 18742107, 18742100, 18742000 and 10742000, but always got

Code: Select all

sanpat@SanPat-Mac-mini ~> sudo zpool import -o readonly=on -T 18742109 -V datapool1 d1
sanpat@SanPat-Mac-mini ~> zfs mount d1
cannot open 'd1': I/O error
sanpat@SanPat-Mac-mini ~> zpool list
NAME   SIZE  ALLOC   FREE  CKPOINT  EXPANDSZ   FRAG    CAP  DEDUP  HEALTH  ALTROOT
d1        -      -      -        -         -      -      -      -  FAULTED  -


Admittedly, I imported it always using a new label, without exporting it, as the export (even with -f) does not work as it claims that one or more device is busy.

can I do something about the spa_laoding failure?

[lundman]

No uber blocks? that is peculiar. I honestly don't know what to suggest at this point - is it possible to see if it imports on other ZFS platforms?

[patmuk]

I can connect the disks to another Mac. I could install a linux platform and try it with another hfs implementation, if that is what you mean?
Or do you mean importing another dataset?

[lundman]

Right yeah, boot one of the other platform CDs, like Linux or Illumos - no need to necessarily install the OS, then try importing. I heard some people had success with Klennet ZFS Recovery as well, but I've not tried it.