zfs panic (reboot) during zfs upgrade -a

Issues specific to non-OpenZFS pools (i.e., ZEVO & MacZFS).

zfs panic (reboot) during zfs upgrade -a

Postby kgreene » Sat Sep 09, 2017 1:20 pm

I performed a zpool upgrade -a successfully:

Code: Select all

$ sudo zpool upgrade -a
This system supports ZFS pool feature flags.

Successfully upgraded 'gallifrey' from version 8 to feature flags.
Enabled the following features on 'gallifrey':
  async_destroy
  empty_bpobj
  lz4_compress
  multi_vdev_crash_dump
  spacemap_histogram
  enabled_txg
  hole_birth
  extensible_dataset
  embedded_data
  bookmarks
  filesystem_limits
  large_blocks
  sha512
  skein
  edonr






Code: Select all
$ zfs upgrade -v
The following filesystem versions are supported:

VER  DESCRIPTION
---  --------------------------------------------------------
 1   Initial ZFS filesystem version
 2   Enhanced directory entries
 3   Case insensitive and filesystem user identifier (FUID)
 4   userquota, groupquota properties
 5   System attributes

For more information on a particular version, including supported releases,
see the ZFS Administration Guide.

$ zfs upgrade
This system is currently running ZFS filesystem version 5.

The following filesystems are out of date, and can be upgraded.  After being
upgraded, these filesystems (and any 'zfs send' streams generated from
subsequent snapshots) will no longer be accessible by older software versions.


VER  FILESYSTEM
---  ------------
 2   gallifrey


I then attempted a zfs upgrade -a. (from pool version 8 (originally, now feature flags), zfs version 2).

The machine rebooted.

Code: Select all
Anonymous UUID:       885A3338-EBAB-70E3-2B06-6FABB4B0B5C1

Sat Sep  9 14:04:20 2017

*** Panic Report ***
panic(cpu 2 caller 0xffffff7f9d48af19): "VERIFY3(" "0" " " "==" " " "dsl_dataset_own(dp, name, tag, &newds)" ") " "failed (" "0" " " "==" " " "16" ")\n"@dmu_objset.c:643
Backtrace (CPU 2), Frame : Return Address
0xffffff92dfa8b770 : 0xffffff801bae953c
0xffffff92dfa8b7f0 : 0xffffff7f9d48af19
0xffffff92dfa8b940 : 0xffffff7f9d4f9f12
0xffffff92dfa8b980 : 0xffffff7f9d4f8a73
0xffffff92dfa8ba00 : 0xffffff7f9d4f8024
0xffffff92dfa8ba80 : 0xffffff7f9d4fd4c5
0xffffff92dfa8bad0 : 0xffffff7f9d4ff77e
0xffffff92dfa8bb60 : 0xffffff801bd53cb3
0xffffff92dfa8bb90 : 0xffffff801bd4915e
0xffffff92dfa8bc10 : 0xffffff801bd3a8e4
0xffffff92dfa8be10 : 0xffffff801bf51c2b
0xffffff92dfa8be40 : 0xffffff801bf9c733
0xffffff92dfa8bf50 : 0xffffff801c0240f5
0xffffff92dfa8bfb0 : 0xffffff801ba9ad96
      Kernel Extensions in backtrace:
         net.lundman.zfs(1.6.1)[CD22A326-A364-3B9C-8169-581C8559F853]@0xffffff7f9d476000->0xffffff7f9d723fff
            dependency: com.apple.iokit.IOStorageFamily(2.1)[F908D7F5-4F54-3B89-8657-57F06350F4DB]@0xffffff7f9c250000
            dependency: net.lundman.spl(1.6.1)[52421600-30A3-3DD8-805C-4EA84362E240]@0xffffff7f9c280000

BSD process name corresponding to current thread: zfs

Mac OS version:
16G29

Kernel version:
Darwin Kernel Version 16.7.0: Thu Jun 15 17:36:27 PDT 2017; root:xnu-3789.70.16~2/RELEASE_X86_64
Kernel UUID: D3314D98-5D40-3CD8-98A4-F1DD46C20E03
Kernel slide:     0x000000001b800000
Kernel text base: 0xffffff801ba00000
__HIB  text base: 0xffffff801b900000
System model name: MacPro5,1 (Mac-F221BEC8)

System uptime in nanoseconds: 73000181406723
last loaded kext at 58674373586059: com.apple.driver.usb.cdc   5.0.0 (addr 0xffffff7fa06a8000, size 28672)
last unloaded kext at 58745887173702: com.apple.driver.usb.cdc   5.0.0 (addr 0xffffff7fa06a8000, size 28672)
loaded kexts:
org.virtualbox.kext.VBoxNetAdp   5.1.26
org.virtualbox.kext.VBoxNetFlt   5.1.26
org.virtualbox.kext.VBoxUSB   5.1.26
org.virtualbox.kext.VBoxDrv   5.1.26
com.nvidia.CUDA   1.1.0
com.nvidia.web.GeForceTeslaWeb   10.1.8
com.nvidia.web.GeForceWeb   10.1.8
com.nvidia.web.NVDAGP100HalWeb   10.1.8
com.nvidia.web.NVDAResmanWeb   10.1.8
com.nvidia.web.NVDANV50HalTeslaWeb   10.1.8
com.nvidia.web.NVDAResmanTeslaWeb   10.1.8
net.lundman.zfs   1.6.1
net.lundman.spl   1.6.1
com.apple.driver.AudioAUUC   1.70
com.apple.driver.AppleHWSensor   1.9.5d0
com.apple.iokit.IOBluetoothSerialManager   5.0.5f1
com.apple.driver.AGPM   110.23.17
com.apple.driver.AppleTyMCEDriver   1.0.2d2
com.apple.filesystems.autofs   3.0
com.apple.driver.AppleMikeyHIDDriver   131
com.apple.driver.pmtelemetry   1
com.apple.iokit.IOUserEthernet   1.0.1
com.apple.driver.AppleUpstreamUserClient   3.6.4
com.apple.driver.AppleMCCSControl   1.3.4
com.apple.driver.AppleHDA   279.48
com.apple.driver.AppleMikeyDriver   279.48
com.apple.Dont_Steal_Mac_OS_X   7.0.0
com.apple.driver.AppleHV   1
com.apple.driver.AppleOSXWatchdog   1
com.apple.driver.AppleIntelSlowAdaptiveClocking   4.0.0
com.apple.driver.AppleLPC   3.1
com.apple.driver.ACPI_SMC_PlatformPlugin   1.0.0
com.apple.iokit.SCSITaskUserClient   394.50.1
com.apple.AppleFSCompression.AppleFSCompressionTypeDataless   1.0.0d1
com.apple.AppleFSCompression.AppleFSCompressionTypeZlib   1.0.0
com.apple.BootCache   40
com.apple.driver.AppleFWOHCI   5.5.5
com.apple.filesystems.hfs.kext   366.70.1
com.apple.iokit.IOAHCIBlockStorage   295.20.1
com.apple.driver.Intel82574LEthernet   2.7.2
com.apple.driver.AirPort.Brcm4331   800.20.26
com.apple.driver.AppleAHCIPort   326.60.1
com.apple.driver.AppleRTC   2.0
com.apple.driver.AppleHPET   1.8
com.apple.driver.AppleACPIButtons   5.0
com.apple.driver.AppleSMBIOS   2.1
com.apple.driver.AppleACPIEC   5.0
com.apple.driver.AppleAPIC   1.7
com.apple.driver.AppleIntelCPUPowerManagementClient   219.0.0
com.apple.nke.applicationfirewall   172
com.apple.security.quarantine   3
com.apple.security.TMSafetyNet   8
com.apple.driver.AppleIntelCPUPowerManagement   219.0.0
com.apple.driver.AppleUSBAudio   308.7
com.apple.iokit.IOSerialFamily   11
com.apple.iokit.IOUSBUserClient   900.4.1
com.apple.kext.triggers   1.0
com.apple.iokit.IOSurface   159.9
com.apple.driver.DspFuncLib   279.48
com.apple.kext.OSvKernDSPLib   525
com.apple.iokit.IOAcceleratorFamily2   311.14
com.apple.AppleGraphicsDeviceControl   3.14.49
com.apple.driver.AppleSSE   1.0
com.apple.driver.AppleSMBusController   1.0.18d1
com.apple.iokit.IOSlowAdaptiveClockingFamily   1.0.0
com.apple.iokit.IONDRVSupport   516.1
com.apple.driver.AppleHDAController   279.48
com.apple.iokit.IOGraphicsFamily   515.3
com.apple.iokit.IOHDAFamily   279.48
com.apple.iokit.IOAudioFamily   205.15
com.apple.vecLib.kext   1.2.0
com.apple.driver.AppleSMBusPCI   1.0.14d1
com.apple.driver.IOPlatformPluginLegacy   1.0.0
com.apple.driver.IOPlatformPluginFamily   6.0.0d8
com.apple.iokit.IOFireWireIP   2.2.7
com.apple.driver.AppleSMC   3.1.9
com.apple.iokit.BroadcomBluetoothHostControllerUSBTransport   5.0.5f1
com.apple.iokit.IOBluetoothHostControllerUSBTransport   5.0.5f1
com.apple.iokit.IOBluetoothHostControllerTransport   5.0.5f1
com.apple.iokit.IOBluetoothFamily   5.0.5f1
com.apple.driver.usb.IOUSBHostHIDDevice   1.1
com.apple.driver.usb.networking   5.0.0
com.apple.driver.usb.AppleUSBHostCompositeDevice   1.1
com.apple.driver.usb.AppleUSBHub   1.1
com.apple.iokit.IOSCSIMultimediaCommandsDevice   394.50.1
com.apple.iokit.IOBDStorageFamily   1.8
com.apple.iokit.IODVDStorageFamily   1.8
com.apple.iokit.IOCDStorageFamily   1.8
com.apple.iokit.IOFireWireFamily   4.6.6
com.apple.filesystems.hfs.encodings.kext   1
com.apple.iokit.IOAHCISerialATAPI   266.50.1
com.apple.iokit.IOSCSIArchitectureModelFamily   394.50.1
com.apple.iokit.IO80211Family   1200.12.2
com.apple.iokit.IONetworkingFamily   3.2
com.apple.driver.corecapture   1.0.4
com.apple.iokit.IOAHCIFamily   288
com.apple.driver.usb.AppleUSBEHCIPCI   1.1
com.apple.driver.usb.AppleUSBUHCIPCI   1.1
com.apple.driver.usb.AppleUSBUHCI   1.1
com.apple.driver.usb.AppleUSBEHCI   1.1
com.apple.driver.usb.AppleUSBHostPacketFilter   1.0
com.apple.iokit.IOUSBFamily   900.4.1
com.apple.driver.AppleUSBHostMergeProperties   1.1
com.apple.driver.AppleEFINVRAM   2.1
com.apple.driver.AppleEFIRuntime   2.1
com.apple.iokit.IOHIDFamily   2.0.0
com.apple.iokit.IOSMBusFamily   1.1
com.apple.security.sandbox   300.0
com.apple.kext.AppleMatch   1.0.0d1
com.apple.driver.AppleKeyStore   2
com.apple.driver.AppleMobileFileIntegrity   1.0.5
com.apple.driver.AppleCredentialManager   1.0
com.apple.driver.KernelRelayHost   1
com.apple.iokit.IOUSBHostFamily   1.1
com.apple.driver.AppleBusPowerController   1.0
com.apple.driver.DiskImages   444.50.16
com.apple.iokit.IOStorageFamily   2.1
com.apple.iokit.IOReportFamily   31
com.apple.driver.AppleFDEKeyStore   28.30
com.apple.driver.AppleACPIPlatform   5.0
com.apple.iokit.IOPCIFamily   2.9
com.apple.iokit.IOACPIFamily   1.4
com.apple.kec.Libm   1
com.apple.kec.pthread   1
com.apple.kec.corecrypto   1.0
Model: MacPro5,1, BootROM MP51.007F.B03, 8 processors, Quad-Core Intel Xeon, 2.26 GHz, 20 GB, SMC 1.39f5
Graphics: NVIDIA GeForce GT 120, NVIDIA GeForce GT 120, PCIe, 512 MB
Graphics: NVIDIA GeForce GTX 1060 6GB, NVIDIA GeForce GTX 1060 6GB, PCIe, 6143 MB
Memory Module: DIMM 1, 4 GB, DDR3 ECC, 1066 MHz, 0x859B, 0x435435313237324241313036372E4D313846
Memory Module: DIMM 2, 4 GB, DDR3 ECC, 1066 MHz, 0x802C, 0x31384A53463531323732415A2D3147314431
Memory Module: DIMM 3, 1 GB, DDR3 ECC, 1066 MHz, 0x830B, 0x4E54314743373242383941304E462D424520
Memory Module: DIMM 4, 1 GB, DDR3 ECC, 1066 MHz, 0x830B, 0x4E54314743373242383941304E462D424520
Memory Module: DIMM 5, 4 GB, DDR3 ECC, 1066 MHz, 0x859B, 0x435435313237324241313036372E4D313846
Memory Module: DIMM 6, 4 GB, DDR3 ECC, 1066 MHz, 0x802C, 0x31384A53463531323732415A2D3147314431
Memory Module: DIMM 7, 1 GB, DDR3 ECC, 1066 MHz, 0x830B, 0x4E54314743373242383941304E462D424520
Memory Module: DIMM 8, 1 GB, DDR3 ECC, 1066 MHz, 0x830B, 0x4E54314743373242383941304E462D424520
AirPort: spairport_wireless_card_type_airport_extreme (0x14E4, 0x8E), Broadcom BCM43xx 1.0 (5.106.98.100.26)
Bluetooth: Version 5.0.5f1, 3 services, 27 devices, 1 incoming serial ports
Network Service: Wi-Fi, AirPort, en2
Serial ATA Device: HL-DT-ST DVD-RW GH41N
Serial ATA Device: APPLE SSD TS256C, 251 GB
Serial ATA Device: SAMSUNG MZ7LM480HCHP-000G3, 480.1 GB
Serial ATA Device: WDC WD40EFRX-68WT0N0, 4 TB
Serial ATA Device: Hitachi HDS5C4040ALE630, 4 TB
Serial ATA Device: Hitachi HDS5C4040ALE630, 4 TB
USB Device: USB Bus
USB Device: Hub in Apple Pro Keyboard
USB Device: Contour Perfit Mouse
USB Device: Apple Pro Keyboard
USB Device: USB Bus
USB Device: BRCM2046 Hub
USB Device: Bluetooth USB Host Controller
USB Device: USB Bus
USB Device: Apple Watch Magnetic Charging Cable
USB Device: USB Bus
USB Device: USB Bus
USB Device: USB Bus
USB Device: USB 2.0 Bus
USB Device: USB 2.0 Bus
USB Device: Hub
USB Device: AK5370
FireWire Device: built-in_hub, Up to 800 Mb/sec
Thunderbolt Bus:


Any comments about what I should do? Try again, do more debugging...?
kgreene
 
Posts: 12
Joined: Sun Jul 05, 2015 8:10 am

Re: zfs panic (reboot) during zfs upgrade -a

Postby kgreene » Sat Sep 09, 2017 6:11 pm

I was poking around, assuming that the panic meant it failed. It looks like it claims it succeeded:

Code: Select all
$ zfs upgrade
This system is currently running ZFS filesystem version 5.

All filesystems are formatted with the current version.


Anything I can do to verify it's working as expected, other than doing a scrub?
kgreene
 
Posts: 12
Joined: Sun Jul 05, 2015 8:10 am

Re: zfs panic (reboot) during zfs upgrade -a

Postby lundman » Sun Sep 10, 2017 4:23 pm

I suspect this could be related to the known ZFS filesystem version upgrade: https://www.illumos.org/issues/6183

When we tested it locally, the pool was fine afterwards, as ZFS updates the blocks dynamically on demand.
User avatar
lundman
 
Posts: 363
Joined: Thu Mar 06, 2014 2:05 pm
Location: Tokyo, Japan

Re: zfs panic (reboot) during zfs upgrade -a

Postby kgreene » Mon Sep 11, 2017 8:03 am

Does that mean a scrub will force an upgrade for everything?
kgreene
 
Posts: 12
Joined: Sun Jul 05, 2015 8:10 am


Return to Non-OpenZFS Pools

Who is online

Users browsing this forum: No registered users and 0 guests