where are disk errors logged in Sierra?

All your general support questions for OpenZFS on OS X.

where are disk errors logged in Sierra?

Postby haer22 » Thu Apr 20, 2017 6:20 am

I just got:
Code: Select all
[ihecc:~] root# zpool import -d /dev
   pool: gaia
     id: 827519048864586214
  state: FAULTED
 status: The pool metadata is corrupted.
 action: The pool cannot be imported due to damaged devices or data.
   The pool may be active on another system, but can be imported using
   the '-f' flag.
   see: http://zfsonlinux.org/msg/ZFS-8000-72
 config:

   gaia        FAULTED  corrupted data
     raidz2-0  ONLINE
       disk14  ONLINE
       disk11  ONLINE
       disk12  ONLINE
       disk13  ONLINE
       disk10  ONLINE
   logs
     disk3s2   ONLINE
[ihecc:~] root# zpool import -d /dev gaia
cannot import 'gaia': I/O error
   Destroy and re-create the pool from
   a backup source.
[ihecc:~] root#


A week ago I got the same thing on my other pool. After trying to export/import disks to see if I could find the failing disk and/or enclosure slot. No luck. So I recreated the pool as I had a backup.

Now this shit happens to the second pool. Luckily I had just copied everything back so I have not lost any data. Still, shipping 15+ TB back and forth is not fun and it takes a loooong time.

What I did was halting the machine. That is using the command "halt", not the power-switch-hard-halt. I had a scrub going on, otherwise everything was normal.

What is this "I/O error"? I cannot just be a disk that is messing up, or? All the disks are ONLINE.
So, did some über-blocks get messed up?
A nice halt, should be OK to do, right? Or do I have to export the disks? What if there is a power-outage...

And where can I find the disk error logs in Sierra. Seems that /var/logs/system is being depreciated.

I have been a happy camper of ZFS for quite some time now, but my confidence in ZFS just got a severe blow.
haer22
 
Posts: 123
Joined: Sun Mar 23, 2014 2:13 am

Re: where are disk errors logged in Sierra?

Postby lundman » Thu Apr 20, 2017 6:16 pm

Can't say I've played with halt, but it just does what it promises, and ZFS updates all the blocks from bottom up, and the uberblock is the final update, so if you are halfway through a txg update, it will only be in uber when it is complete, otherwise you will see the previous txg. Hence ZFS is always consistent.

I'd be more concerned about the disk order as I see you are still using /dev/disk names, which can be a bad idea. So if your HDDs have been reordered, you may indeed get the error you see. Please check /var/run/disk/ path as well..
User avatar
lundman
 
Posts: 1337
Joined: Thu Mar 06, 2014 2:05 pm
Location: Tokyo, Japan

Re: where are disk errors logged in Sierra?

Postby haer22 » Thu Apr 20, 2017 10:22 pm

lundman wrote:Can't say I've played with halt, but it just does what it promises, and ZFS updates all the blocks from bottom up, and the uberblock is the final update, so if you are halfway through a txg update, it will only be in uber when it is complete, otherwise you will see the previous txg. Hence ZFS is always consistent.

Yeah, that has been my impression as well. I have been "halting" many times instead of doing the "Finder shutdown". Halting is faster and (presumably) safe as well.
I'd be more concerned about the disk order as I see you are still using /dev/disk names, which can be a bad idea. So if your HDDs have been reordered, you may indeed get the error you see. Please check /var/run/disk/ path as well..

Oh, I always use the media-xxxxx... device names when I do the actual import.

If an re-order happens, how fast do zed recognise that and generate new links from media-xxxx to the new /dev/diskNN?
If the re-order happens and there is a disk under the media-xxxx device name, albeit a new wrong one, will zfs realise it is the wrong disk when it is doing the next txg-writing? I.e. no disk errors but data will be written in a very wrong place.
I was using a couple of USB disk-attachments, some of dubious quality, so a spontaneous re-attach of the USB-connection *may* have happened. Normally the disks sits in enclosures attached via eSATA. That could explain why something that has worked many times, now suddenly screws up twice within 10 days.

Code: Select all
[ihecc:~] root# zpool import
   pool: gaia
     id: 827519048864586214
  state: FAULTED
 status: The pool metadata is corrupted.
 action: The pool cannot be imported due to damaged devices or data.
   The pool may be active on another system, but can be imported using
   the '-f' flag.
   see: http://zfsonlinux.org/msg/ZFS-8000-72
 config:

   gaia                                            FAULTED  corrupted data
     raidz2-0                                      ONLINE
       media-0A89D98E-86A0-4242-AE2E-8ECDA2902B86  ONLINE
       media-E87BF3FD-6DEF-CA48-B2DA-D344633DCBDB  ONLINE
       media-1C2D5602-329E-7146-A8ED-F48EB8057223  ONLINE
       media-FDA72E74-3E71-4944-AFA4-A763D3A37360  ONLINE
       media-BFA83DC0-89CC-F343-87B5-C946A9C1365D  ONLINE
   logs
     media-931B9542-FFEE-4805-A21E-3C5E836730FC    ONLINE
[ihecc:~] root# zpool import -f gaia
cannot import 'gaia': I/O error
   Destroy and re-create the pool from
   a backup source.
[ihecc:~] root#
haer22
 
Posts: 123
Joined: Sun Mar 23, 2014 2:13 am

Re: where are disk errors logged in Sierra?

Postby JasonBelec » Wed Apr 26, 2017 6:12 am

In many instances, just restarting everything resolves this. Yes its a pain, but usually totally recoverable from.
JasonBelec
 
Posts: 32
Joined: Mon Oct 26, 2015 1:07 pm

Re: where are disk errors logged in Sierra?

Postby haer22 » Thu Apr 27, 2017 11:46 am

When things get "strange", I always export and then import the pool to see if the issue has resolved itself. Next step is reboot.

Neither helped in this case.
haer22
 
Posts: 123
Joined: Sun Mar 23, 2014 2:13 am


Return to General Help

Who is online

Users browsing this forum: No registered users and 29 guests