Big Sur 2.0 pkg - kernel panics after 2 weeks of uptime

All your general support questions for OpenZFS on OS X.

Re: Big Sur 2.0 pkg - kernel panics after 2 weeks of uptime

Postby xenophon » Fri Jul 23, 2021 5:32 am

Arghhh. Had high hopes with the rc2 release, which was stable for (almost) a week on an up-to-date 11.4 system.

Then suddenly the computer was not accessible by ssh, and keyboard input was disabled; the mouse moved around, albeit beachballing everywhere.

After this slow, stuttering freeze, watchdogd panicked:
Code: Select all
panic(cpu 2 caller 0xffffff8007bf9a25): userspace watchdog timeout: no successful checkins from com.apple.logd in 120 seconds
service returned not alive with context : unresponsive dispatch queue(s): com.apple.firehose.io-wl
service: com.apple.logd, total successful checkins since wake (6010 seconds ago): 590, last successful checkin: 120 seconds ago
service: com.apple.WindowServer, total successful checkins since wake (6010 seconds ago): 602, last successful checkin: 0 seconds ago

Backtrace (CPU 2), Frame : Return Address
0xffffffc1fdab7670 : 0xffffff800488e0dd mach_kernel : _handle_debugger_trap + 0x3fd
0xffffffc1fdab76c0 : 0xffffff80049d4f33 mach_kernel : _kdp_i386_trap + 0x143
0xffffffc1fdab7700 : 0xffffff80049c552a mach_kernel : _kernel_trap + 0x55a
0xffffffc1fdab7750 : 0xffffff8004832a2f mach_kernel : _return_from_trap + 0xff
0xffffffc1fdab7770 : 0xffffff800488d8fd mach_kernel : _DebuggerTrapWithState + 0xad
0xffffffc1fdab7890 : 0xffffff800488dbf3 mach_kernel : _panic_trap_to_debugger + 0x273
0xffffffc1fdab7900 : 0xffffff800509d884 mach_kernel : _panic_with_thread_context
0xffffffc1fdab7970 : 0xffffff8007bf9a25 com.apple.driver.watchdog : __ZN10IOWatchdog14userspacePanicEP8OSObjectPvP25IOExternalMethodArguments.cold.1 + 0x27
0xffffffc1fdab7980 : 0xffffff8007bf9660 com.apple.driver.watchdog : __ZN10IOWatchdog35userspaceDisableUserspaceMonitoringEP8OSObjectPvP25IOExternalMethodArguments
0xffffffc1fdab79a0 : 0xffffff800501e13e mach_kernel : __ZN12IOUserClient14externalMethodEjP25IOExternalMethodArgumentsP24IOExternalMethodDispatchP8OSObjectPv + 0x1de
0xffffffc1fdab79f0 : 0xffffff8007bf8a34 com.apple.driver.watchdog : __ZN20IOWatchdogUserClient14externalMethodEjP25IOExternalMethodArgumentsP24IOExternalMethodDispatchP8OSObjectPv + 0x7c
0xffffffc1fdab7b20 : 0xffffff800502837b mach_kernel : _is_io_connect_method + 0x35b
0xffffffc1fdab7c80 : 0xffffff8004980961 mach_kernel : _iokit_server_routine + 0x4dd1
0xffffffc1fdab7d90 : 0xffffff800489375d mach_kernel : _ipc_kobject_server + 0x2fd
0xffffffc1fdab7e00 : 0xffffff8004869cf5 mach_kernel : _ipc_kmsg_send + 0x115
0xffffffc1fdab7e60 : 0xffffff8004881312 mach_kernel : _mach_msg_overwrite_trap + 0x2d2
0xffffffc1fdab7ef0 : 0xffffff80049a982d mach_kernel : _mach_call_munger64 + 0x29d
0xffffffc1fdab7fa0 : 0xffffff8004833216 mach_kernel : _hndl_mach_scall64 + 0x16
      Kernel Extensions in backtrace:
         com.apple.driver.watchdog(1.0)[C29AFC04-7113-39AA-80DF-00F4DDAD0F46]@0xffffff8007bf7000->0xffffff8007bf9fff

Process name corresponding to current thread: watchdogd
Boot args: keepsyms=1 darkwake=3 shikigva=80 debug=0x100 igfxonln=1 igfxfw=2 forceRenderStandby=0 alcid=11 kernel_stack_pages=8

Mac OS version:
20F71

Kernel version:
Darwin Kernel Version 20.5.0: Sat May  8 05:10:33 PDT 2021; root:xnu-7195.121.3~9/RELEASE_X86_64
Kernel UUID: 52A1E876-863E-38E3-AC80-09BBAB13B752
KernelCache slide: 0x0000000004600000
KernelCache base:  0xffffff8004800000
Kernel slide:      0x0000000004610000
Kernel text base:  0xffffff8004810000
__HIB  text base: 0xffffff8004700000
System model name: iMac19,1 (Mac-AA95B1DDAB278B95)
System shutdown begun: NO
Panic diags file available: YES (0x0)
Hibernation exit count: 0

System uptime in nanoseconds: 326082019223097
Last Sleep:           absolute           base_tsc          base_nano
  Uptime  : 0x00012891e2ec6b2a
  Sleep   : 0x000123163cf9a9a8 0x000000012be3a81e 0x0001227af1c9a2b3
  Wake    : 0x00012316551df809 0x000000012c496f26 0x00012316463f4182


I've installed 11.5 and will report back.

In retrospect, my system has always had this long-uptime-leading-to-kp issue, not just with Big Sur and OOO 2.x, but even prior versions. In fact, it plagued me for years!

It may be an esoteric bug, specific to my circumstances (OSX86 machines old and new), but suspect it is not. Still, for practical purposes, I have started giving up on resolution of the issue.

I am very grateful for this project, the forum and all the work that goes in OpenZFSonOSX.

My work (life?) depends on it, and I will probably implement some sort of workaround, e.g. rebooting once weekly or something (ugh!).

Take care and, as always, thanks!

Xen
xenophon
 
Posts: 19
Joined: Tue Jul 28, 2015 11:58 pm

Re: Big Sur 2.0 pkg - kernel panics after 2 weeks of uptime

Postby lundman » Sun Jul 25, 2021 9:59 pm

"com.apple.firehose.io-wl"

That is not ZFS, but more interestingly, this comes from https://opensource.apple.com/source/lib ... /firehose/
where it creates the queue:

Code: Select all
```
        fs->fs_io_wl = dispatch_workloop_create_inactive("com.apple.firehose.io-wl");
        dispatch_set_qos_class_fallback(fs->fs_io_wl, QOS_CLASS_UTILITY);
        dispatch_activate(fs->fs_io_wl);
```
contained in `/usr/libexec/logd`. Be fun to try if you can just restart logd daily or something. But presumably some hardware stops talking to logd, so restarting
logs might not actually solve the problem.
User avatar
lundman
 
Posts: 1335
Joined: Thu Mar 06, 2014 2:05 pm
Location: Tokyo, Japan

Previous

Return to General Help

Who is online

Users browsing this forum: Google [Bot] and 36 guests