Page 1 of 1

ZFS lost device hung entire system

PostPosted: Thu Aug 10, 2017 8:44 pm
by afberendsen
I have ZFS installed on my iMac.
I am using ZFS mostly on my USB attached HDD.
One of disks disconnected (USB fault) and my entire machine is hung on I/O.
This means that I can still use the shell (where I am almost all the time)...unless is something that have to access the I/O subsystem, which makes the session to hung.
I already had few iTerm sessions open to helpmeet identify the culprit, and seems that it is the ZFS.
Facts I have at this stage:
- iMac Intel running Sierra (see below)
- ZFS installed three weeks ago
- zpool list, pool iostat, pool status is showing information, which I cannot copy/paste because I am using second computer. The computer experiencing the problem cannot be used for any activity that relies on disk access
- sudo pool clear hangs the session
- Disk affected is /dev/disk9 vol201. Turning it off does not make any difference. Turning it on back still any difference
- Only solution, at this point, is to reset the computer...again

Any ideas?

Code: Select all
+-------------------------------------------------------------------------------------------+
|                                Andreas Friedrich Berendsen                                |
|                                                                                           |
| Host name .......: H0000000                                                               |
| Hardware platform: Apple iMac "Core i7"/3.5 27-Inch (Late 2013)                           |
|   Model .........: iMac (ModelId=iMac14,2)                                                |
|   Apple name ....: iMac (27-inch, Late 2013)                                              |
|   EveryMac name .: iMac "Core i7" 3.5 27-Inch (Late 2013)                                 |
|   Serial Number .: C02LT0NJF8JC                                                           |
|   CPU ...........: 1 x Intel Core i7 @ 3.5 GHz                                            |
|   RAM ...........: 32 GB (8GB+8GB+8GB+8GB DDR3 @ 1600MHz)                                 |
|   Storage:                                                                                |
|      FDD ........: None                                                                   |
|      ZDD ........: None                                                                   |
|      ODD ........: None                                                                   |
|      HDD ........: 6 TB (~5 TiB)                                                          |
| Software platform: Mac OS X                                                               |
|   OS Version ....: 10.12.2 (Sierra)                                                       |
|   Build .........: 16C67                                                                  |
|-------------------------------------------------------------------------------------------|
| Data collect: sw_vers; system_profiler SPHardwareDataType                                 |
| My collection: https://sites.google.com/site/afberendsen/instances/h0000000               |
+-------------------------------------------------------------------------------------------+

Re: ZFS lost device hung entire system

PostPosted: Wed Aug 16, 2017 3:45 pm
by JasonBelec
Yup. USB bad. I don't know how much louder we can say this, over and over again to prevent this issue. I guess we are fighting the cheap enclosure battle. I can pull up 23 instances of clients using ZFS and USB, I cringe, everyone had issues. Just did some tests yesterday with two SSD's in a USB3 enclosure and both took down the respective pools. If you put those disks in real enclosures with power you should be able to recover. Shutdown everything first. swap. reboot, and see, run a scrub for sure.

Re: ZFS lost device hung entire system

PostPosted: Thu Aug 17, 2017 10:13 pm
by lundman
Yes, USB has been a stickler for us, and we've tried to make things better. At the very least make sure you use the /var/run/disk names instead of /dev/disk as that will handle re-attached disks a little better (import -d /var/run/disk/*).

If you do have a hang, at the very least we need the output from "spindump" to see what threads are going nowhere, fast.

Re: ZFS lost device hung entire system

PostPosted: Sun Aug 20, 2017 10:15 pm
by xenophon
Hi, and sorry for my noobness.

How exactly do you implement the /var/run convention on existing pools?

TIA

Xen

Re: ZFS lost device hung entire system

PostPosted: Sun Aug 20, 2017 10:46 pm
by FadingIntoBlue
No worries - a search for import -d on the forum will throw up this useful thread, which explains it all fairly well.

https://openzfsonosx.org/forum/viewtopic.php?f=26&t=3011

Cheers

Re: ZFS lost device hung entire system

PostPosted: Sun Sep 17, 2017 10:16 pm
by sgbirch
JasonBelec wrote:If you put those disks in real enclosures with power you should be able to recover. Shutdown everything first. swap. reboot, and see, run a scrub for sure.


What do you mean by this, can you give more information about which USB adapters are a problem and which are recommended? Is there an alternative to USB?

I am using powered adapters by Technet and am fighting a problem where the Mac grinds to a halt if I run two rsyncs from one ZFS mirror to another. Could this is be a problem with the USB adapter?

Re: ZFS lost device hung entire system

PostPosted: Mon Sep 18, 2017 1:51 pm
by xbytor
What do you mean by this, can you give more information about which USB adapters are a problem and which are recommended? Is there an alternative to USB?


eSata if your iMac has the port. Just get a decent case and your set.
Your other alternative is getting a Thunderbolt case if you have that port.

I don't know about USB3, but USB2 enclosures and ZFS typically don't play together. Been down this road myself. All my USB2 drives are for my Win7 boxes.