Upcoming testing...

Developer discussions.

Upcoming testing...

Postby roemer » Sat Mar 18, 2023 3:33 am

lundman wrote under "News & Annoucnements":
So we should probably look at release-candidate for a 2.1.7, and all the fixes we've done already.
I'm also testing out assembly files for M1/arm64.
I have tested "aesv8-armx" which came out as
Code: Select all
 icp_aes_impl=generic READ: bw=239MiB/s (250MB/s), 239MiB/s-239MiB/s
          (250MB/s-250MB/s), io=69.9GiB (75.1GB), run=300003-300003msec

 icp_aes_impl=aesv8   READ: bw=566MiB/s (593MB/s), 566MiB/s-566MiB/s
          (593MB/s-593MB/s), io=166GiB (178GB), run=300001-300001msec

Which should help those using encryption on arm64. Any other area in particular that could need speedup with assembly?

Sorry, but I don't think we can reply in the announcements board - hence copied over here...

Faster encryption on arm64 would be absolutely great!
Currently, without assembly code, it is heavily CPU-bound with about ⅓ of the unencrypted speed...

There is an open issue re ZFS encryption performance on arm64 in the main OpenZFS project: https://github.com/openzfs/zfs/issues/12171
Is this an attempted fix to this? Or are these two different things?
roemer
 
Posts: 73
Joined: Sat Mar 15, 2014 2:32 pm

Re: Upcoming testing...

Postby o3x_prokyon » Sun Mar 19, 2023 2:09 pm

(...moved from my previous extra post...)

icp_aes_impl=aesv8
on Apple Silicon arm64 looks amazing!

(Shouldn't icp_aes_impl=fastest (the default) choose the best option on it's own? On my older Mac mini i7 I set icp_aes_impl=aesni manually, but couldn't see any difference. The reason might be the „slow“ USB 3.0 (5 GBit) connection to the external case or the two mirrored rotational disks in there.)

You asked for any areas that could need speedup with assembly... I'm not so deep in programming (especially assembly...), but what about checksumming and transparent compression?
o3x_prokyon
 
Posts: 33
Joined: Thu Dec 01, 2022 1:17 pm

Re: Upcoming testing...

Postby lundman » Sun Mar 19, 2023 6:58 pm

OK, so AES v8, SHA256 v8 and SHA512 v8 all appear to work, and give some improvements.

At the moment, I was just trying to see if I could put in new assembly for aarch64, and make it work. Seems I can.
Currently, I'm not making any big changes to ZFS, just putting in aarch64 versions where there is already a
x86_64 version. So aes, gash, sha are about it.

The other guys are working on "zfs_impl" which lets us plug in various kinds and pick which one to use, just
like blake3 and fletcher has - just more generic. Aes/ccm/gcm are a bit more baked in, but hopefully they
can make it a bit more generic.

"fastest" will pick armv8 in the new code. Not from benchmarking, that is in "zfs_impl" designs.

I could make a pkg of this if people want to check it is a little faster. Currently, it only makes AES part of
encryption faster, there are other parts (GCM, GHASH) that can also be made faster (after zfs_impl).


Oh and thanks for sharing the PR, I was unaware of it. I wonder if we can all cooperate and help each other.
User avatar
lundman
 
Posts: 1327
Joined: Thu Mar 06, 2014 2:05 pm
Location: Tokyo, Japan

Re: Upcoming testing...

Postby roemer » Mon Mar 20, 2023 3:42 am

On my datasets, I am currently typically using
Code: Select all
checksum=edonr

Does this have aarch64 assembly acceleration too?
Or are the preferred checksum implementations at the moment fletcher4, sha256/512, and blake3?
roemer
 
Posts: 73
Joined: Sat Mar 15, 2014 2:32 pm

Re: Upcoming testing...

Postby lundman » Mon Mar 20, 2023 4:48 pm

In .2.1.6 there is none:

https://github.com/openzfsonosx/openzfs ... sm-aarch64

since I enabled blake after.

But next release should have sha256/sha512/blake3/aes
User avatar
lundman
 
Posts: 1327
Joined: Thu Mar 06, 2014 2:05 pm
Location: Tokyo, Japan

Re: Upcoming testing...

Postby lundman » Wed Mar 22, 2023 4:27 pm

Any feed back on the 2.1.7 arm64 assembly? Does it work, do you trust it?
User avatar
lundman
 
Posts: 1327
Joined: Thu Mar 06, 2014 2:05 pm
Location: Tokyo, Japan

Re: Upcoming testing...

Postby roemer » Thu Mar 23, 2023 5:11 am

I did some first tests using the test version of 2.1.7 on an M1 Ventura 13.2 with a 4xSSD RAIDZ1.
Performance looks much better than 2.1.6 - many thanks!
Regarding 'trust': sorry, I hadn't time to stress-test the consistency yet. Just a handful of files into a new encrypted dataset, which all worked.

A few things to note:
  • 2.1.7 encryption is in general faster than 2.1.6 encryption, but depending on which encryption setting, is still quite a bit slower than un-encrypted.
  • I found AES-CCM is in general faster than AES-GCM. So for example I can reach basically un-encrypted write speed with aes-256-ccm, but only about 40% write speed with aes-256-gcm; I think aes-256-gcm is the default for encryption=on
  • Example write speeds on a dataset with recordsize=1M using AmorphousDiskMark4 and 1GiB data:
    unencrypted: 1121 MB/s
    aes-256-gcm + lz4: 415 MB/s
    aes-256-ccm + lz4: 1095 MB/s
    With ZFS 2.1.6 I got the same unencrypted write speed, but aes-256-gcm encrypted was only 340 MB/s and aes-128-ccm was only 633 MB/s.
I assume the AES256-gcm performance will improve with a future arm-assembly implementation of gcm?

A few more questions:
I do not see an arm-assembly of blake3 mentioned in sysctl, nor any acceleration for, e.g., lz4 compression:
Code: Select all
zfs version
zfs-2.1.99-1723_gae7a8aca0
zfs-kmod-2.1.99-1723_gae7a8aca0

sysctl -a | grep zfs | grep _impl
kstat.zfs.darwin.tunable.zfs_vdev_raidz_impl: cycle [fastest] original scalar
kstat.zfs.darwin.tunable.icp_gcm_impl: cycle [fastest] generic
kstat.zfs.darwin.tunable.icp_aes_impl: cycle [fastest] generic aesv8
kstat.zfs.darwin.tunable.zfs.fletcher_4_impl: [fastest] scalar superscalar superscalar4
kstat.zfs.darwin.tunable.zfs.blake3_impl: cycle [fastest] generic sse2 sse41
kstat.zfs.darwin.tunable.zfs.sha256_impl: cycle [fastest] generic armv7 neon armv8-ce
kstat.zfs.darwin.tunable.zfs.sha512_impl: cycle [fastest] generic armv7 armv8-ce

How can I switch to a specific implementation rather than [fastest], resp. how can I check what [fastest] actually executes (especially for aes)?

Which checksum would you suggest to use?
Actually lundman, which checksum algorithm do you normally use yourself nowadays?
roemer
 
Posts: 73
Joined: Sat Mar 15, 2014 2:32 pm

Re: Upcoming testing...

Postby o3x_prokyon » Thu Mar 23, 2023 8:14 am

roemer wrote:How can I switch to a specific implementation rather than [fastest], resp. how can I check what [fastest] actually executes (especially for aes)?


You can set all parameters in /etc/zfs/zsysctl.conf:
Change a pre-existing line or insert a new one like
Code: Select all
kstat.zfs.darwin.tunable.icp_aes_impl=aesv8


At boot-time zsysctl.conf is read via /usr/local/zfs/libexec/zfs/launch.d/zpool-import-all.sh (by the corresponding LaunchDaemon).
After changing zsysctl.conf, you can activate the new settings without a reboot by running
Code: Select all
/usr/local/zfs/bin/zsysctl -f /etc/zfs/zsysctl.conf


What I don't know is, how to check which mode [fastest] actually uses...

I just downloaded the pre v2.1.7 (OpenZFSonOsX-2.1.99-Ventura-13-arm64) and will test it later. With my rotational HDDs I expect only a small performance-boost in comparison to my older Mac mini i7. We'll see...
o3x_prokyon
 
Posts: 33
Joined: Thu Dec 01, 2022 1:17 pm

Re: Upcoming testing...

Postby lundman » Sun Mar 26, 2023 4:33 pm

So fastest is meant to pick the fastest for you, but yeah, there isn't a way to see what that happens to be.
There is also a mini-benchmark at startup, which is coming. (I don't think it is in the .pkg I made?)

Code: Select all
# sysctl kstat.zfs.misc.chksum_bench.verbose=1
kstat.zfs.misc.chksum_bench.verbose: 0 -> 1

#  sysctl kstat.zfs.misc.chksum_bench         
kstat.zfs.misc.chksum_bench.chksum_bench:
implementation               1k      4k     16k     64k    256k      1m      4m     16m
edonr-generic               504     566     584     591     586     586     585     580
skein-generic               120     127     129     129     129     127     129     129
sha256-generic              112     119     118     117     117     118     119     121
sha256-armv7                336     359     366     367     368     368     366     366
sha256-neon                 345     369     376     377     378     378     377     377
sha256-armv8-ce            1884    2215    2315    2343    2337    2352    2310    2320
sha512-generic              169     186     192     194     193     194     194     193
sha512-armv7                503     562     580     581     585     585     583     578
sha512-armv8-ce            1120    1303    1359    1374    1377    1378    1376    1352
blake3-generic               72      69      68      68      68      68      68      67
blake3-sse2                 403    1341    1446    1395    1486    1478    1441    1375
blake3-sse41                428    1390    1511    1584    1549    1540    1480    1461


I generally just leave it at "fastest".

"aes-256-gcm" will get a speed boost with ghash assembly, I only added AES for now.


In the other thread, someone pointed out the arm64 test build is missing the decmpfs fix, so be aware of that or rollback to 2.1.6.

At the moment I'm waiting on ZOL to do the zfs_impl work, so we can more easily slot in assembly versions.
User avatar
lundman
 
Posts: 1327
Joined: Thu Mar 06, 2014 2:05 pm
Location: Tokyo, Japan

Re: Upcoming testing...

Postby roemer » Fri Apr 14, 2023 3:25 am

Thanks lundman.
How do you prefer to receive bug reports on the current 2.1.99rc builds?
Here in the forum or via issue posts in github?

I started converting some datasets over to 2.1.99 and just got a kernel panic by a zfs rollback command.
I sent/received a dataset into a newly, compressed+encrypted (aes-256-gcm) dataset which got auto-mounted after being received. So fine so good.
Before then receiving further snapshots, I tried a 'rollback' statement on that just received (and still mounted) dataset, which resulted into the following panic:
Code: Select all
zfs version
zfs-2.1.99-1874_gb0e5563eb
zfs-kmod-2.1.99-1874_gb0e5563eb

panic(cpu 2 caller 0xfffffe001527ebec): Kernel data abort. at pc 0xfffffe0014cc4204, lr 0xb8c6fe00134b2730 (saved state: 0xfffffe80219ab540)
     x0:  0x0000000000000000 x1:  0xfffffe1b4efd8838  x2:  0x0000000000000000  x3:  0x0000000000000014
     x4:  0x0000001269b4defb x5:  0x000000000003a980  x6:  0x0000000000000000  x7:  0x0000000000000000
     x8:  0x00000000300dcb5e x9:  0x0000000000000000  x10: 0x0000000000000001  x11: 0xfffffe781b910000
     x12: 0x0000000000000018 x13: 0x7fffffffffffffff  x14: 0x0000000000000008  x15: 0x00001269b4defb08
     x16: 0xe9997e0014cc41c0 x17: 0xfffffe00133589b0  x18: 0x0000000000000000  x19: 0xfffffe1b4efd8838
     x20: 0x0000000000000000 x21: 0xfffffe5c3e131190  x22: 0xfffffe0017ddb000  x23: 0x0000000000000008
     x24: 0x0000000000000019 x25: 0xfffffe5c3a67f3c8  x26: 0xfffffe5c3a67f590  x27: 0xfffffe00132b5f78
     x28: 0x000002faf7d9d777 fp:  0xfffffe80219ab8c0  lr:  0xb8c6fe00134b2730  sp:  0xfffffe80219ab890
     pc:  0xfffffe0014cc4204 cpsr: 0x60401208         esr: 0x96000006          far: 0x0000000000000008

Debugger message: panic
Memory ID: 0x6
OS release type: User
OS version: 22E261
Kernel version: Darwin Kernel Version 22.4.0: Mon Mar  6 20:59:28 PST 2023; root:xnu-8796.101.5~3/RELEASE_ARM64_T6000
Fileset Kernelcache UUID: D16B98A05F8C93AE7ED3EF040209BFD7
Kernel UUID: EEE28733-BD89-3C0D-A663-1C095A86F23C
Boot session UUID: 299972CF-ECAC-4674-9C53-A06FE5D69BA6
iBoot version: iBoot-8422.100.650
secure boot?: YES
roots installed: 0
Paniclog version: 14
KernelCache slide: 0x000000000c6e0000
KernelCache base:  0xfffffe00136e4000
Kernel slide:      0x000000000c6e8000
Kernel text base:  0xfffffe00136ec000
Kernel text exec slide: 0x000000000daa0000
Kernel text exec base:  0xfffffe0014aa4000
mach_absolute_time: 0x1269c0e5b2
Epoch Time:        sec       usec
  Boot    : 0x64392891 0x0000b0f3
  Sleep   : 0x00000000 0x00000000
  Wake    : 0x00000000 0x00000000
  Calendar: 0x64393567 0x000501e7

Zone info:
  Zone map: 0xfffffe100a914000 - 0xfffffe300a914000
  . VM    : 0xfffffe100a914000 - 0xfffffe14d75e0000
  . RO    : 0xfffffe14d75e0000 - 0xfffffe1670f78000
  . GEN0  : 0xfffffe1670f78000 - 0xfffffe1b3dc44000
  . GEN1  : 0xfffffe1b3dc44000 - 0xfffffe200a910000
  . GEN2  : 0xfffffe200a910000 - 0xfffffe24d75dc000
  . GEN3  : 0xfffffe24d75dc000 - 0xfffffe29a42a8000
  . DATA  : 0xfffffe29a42a8000 - 0xfffffe300a914000
  Metadata: 0xfffffe577ed14000 - 0xfffffe5786d14000
  Bitmaps : 0xfffffe5786d14000 - 0xfffffe5792944000
  Extra   : 0 - 0

TPIDRx_ELy = {1: 0xfffffe1b4efd8838  0: 0x0000000000000002  0ro: 0x00000001f4435c20 }
CORE 0 PVH locks held: None
CORE 1 PVH locks held: None
CORE 2 PVH locks held: None
CORE 3 PVH locks held: None
CORE 4 PVH locks held: None
CORE 5 PVH locks held: None
CORE 6 PVH locks held: None
CORE 7 PVH locks held: None
CORE 8 PVH locks held: None
CORE 9 PVH locks held: None
CORE 0: PC=0xfffffe0014b32788, LR=0xfffffe0014b32788, FP=0xfffffe8021927ef0
CORE 1: PC=0xfffffe0014b32788, LR=0xfffffe0014b32788, FP=0xfffffe8021ab7ef0
CORE 2 is the one that panicked. Check the full backtrace for details.
CORE 3: PC=0xfffffe0014c22458, LR=0xfffffe0014c307c4, FP=0xfffffe80212cfe80
CORE 4: PC=0xfffffe0014c22478, LR=0xfffffe0014c307c4, FP=0xfffffe80214f7e80
CORE 5: PC=0xfffffe0014c22478, LR=0xfffffe0014c307c4, FP=0xfffffe8021293e80
CORE 6: PC=0xfffffe0014b32788, LR=0xfffffe0014b32788, FP=0xfffffe8021903ef0
CORE 7: PC=0xfffffe0014b32788, LR=0xfffffe0014b32788, FP=0xfffffe8021807ef0
CORE 8: PC=0xfffffe0014b32788, LR=0xfffffe0014b32788, FP=0xfffffe8021897ef0
CORE 9: PC=0xfffffe0014b32788, LR=0xfffffe0014b32788, FP=0xfffffe8021a33ef0
Compressor Info: 0% of compressed pages limit (OK) and 0% of segments limit (OK) with 0 swapfiles and OK swap space
Panicked task 0xfffffe200cbaf9b8: 255 pages, 1 threads: pid 2828: zfs
Panicked thread: 0xfffffe1b4efd8838, backtrace: 0xfffffe80219aacc0, tid: 46115
        lr: 0xfffffe0014afa868  fp: 0xfffffe80219aad40
        lr: 0xfffffe0014c3b618  fp: 0xfffffe80219aad60
        lr: 0xfffffe0014c2c920  fp: 0xfffffe80219aadd0
        lr: 0xfffffe0014c2aff8  fp: 0xfffffe80219aae90
        lr: 0xfffffe0014aab7f8  fp: 0xfffffe80219aaea0
        lr: 0xfffffe0014afa14c  fp: 0xfffffe80219ab250
        lr: 0xfffffe0015275054  fp: 0xfffffe80219ab270
        lr: 0xfffffe001527ebec  fp: 0xfffffe80219ab3f0
        lr: 0xfffffe0014c2c7a0  fp: 0xfffffe80219ab460
        lr: 0xfffffe0014c2b36c  fp: 0xfffffe80219ab520
        lr: 0xfffffe0014aab7f8  fp: 0xfffffe80219ab530
        lr: 0xfffffe00134b2730  fp: 0xfffffe80219ab8c0
        lr: 0xfffffe00134b2730  fp: 0xfffffe80219ab900
        lr: 0xfffffe0013470298  fp: 0xfffffe80219ab940
        lr: 0xfffffe001346cb9c  fp: 0xfffffe80219aba10
        lr: 0xfffffe00134aeb00  fp: 0xfffffe80219aba40
        lr: 0xfffffe0014cfa1e8  fp: 0xfffffe80219abaa0
        lr: 0xfffffe0014cec74c  fp: 0xfffffe80219abcb0
        lr: 0xfffffe0014f63808  fp: 0xfffffe80219abce0
        lr: 0xfffffe0014fc4d28  fp: 0xfffffe80219abdf0
        lr: 0xfffffe00150b2f08  fp: 0xfffffe80219abe50
        lr: 0xfffffe0014c2b200  fp: 0xfffffe80219abf10
        lr: 0xfffffe0014aab7f8  fp: 0xfffffe80219abf20
      Kernel Extensions in backtrace:
         org.openzfsonosx.zfs(2.1.99)[0D64A28A-6AA2-3CD6-BC2F-E78C18C79718]@0xfffffe0013364f40->0xfffffe00135a6a9f
            dependency: com.apple.iokit.IOStorageFamily(2.1)[AF2CB08B-2800-3145-B35D-C41DB03AC901]@0xfffffe0016fd3280->0xfffffe0016ff4e8f

last started kext at 10155156157: com.apple.filesystems.smbfs   5.0 (addr 0xfffffe0014029e40, size 78556)
loaded kexts:
org.openzfsonosx.zfs   2.1.99
com.apple.filesystems.smbfs   5.0
com.apple.filesystems.autofs   3.0
com.apple.UVCService   1
com.apple.driver.AppleBluetoothMultitouch   100
com.apple.driver.AppleHIDALSService   1
com.apple.driver.AppleTopCaseHIDEventDriver   6440.7
com.apple.driver.AppleBiometricServices   1
com.apple.driver.CoreKDL   1
com.apple.driver.DiskImages.ReadWriteDiskImage   493.0.0
com.apple.driver.DiskImages.UDIFDiskImage   493.0.0
com.apple.driver.DiskImages.RAMBackingStore   493.0.0
com.apple.driver.DiskImages.FileBackingStore   493.0.0
com.apple.driver.BCMWLANFirmware4387.Hashstore   1
com.apple.driver.AppleUSBDeviceNCM   5.0.0
com.apple.driver.AppleThunderboltIP   4.0.3
com.apple.driver.SEPHibernation   1
com.apple.driver.AppleFileSystemDriver   3.0.1
com.apple.nke.l2tp   1.9
com.apple.filesystems.tmpfs   1
com.apple.filesystems.nfs   1
com.apple.filesystems.lifs   1
com.apple.filesystems.apfs   2142.101.3
com.apple.IOTextEncryptionFamily   1.0.0
com.apple.filesystems.hfs.kext   627.100.6
com.apple.security.BootPolicy   1
com.apple.BootCache   40
com.apple.AppleFSCompression.AppleFSCompressionTypeZlib   1.0.0
com.apple.AppleFSCompression.AppleFSCompressionTypeDataless   1.0.0d1
com.apple.AppleEmbeddedSimpleSPINORFlasher   1
com.apple.driver.AppleSmartIO2   1
com.apple.driver.ApplePMP   1
com.apple.driver.AppleCS42L84Audio   640.8
com.apple.driver.AppleSN012776Amp   640.8
com.apple.driver.AppleT6000SOCTuner   1
com.apple.driver.AppleT6000CLPCv3   1
com.apple.driver.AppleSmartBatteryManager   161.0.0
com.apple.driver.AppleALSColorSensor   1.0.0d1
com.apple.driver.AppleAOPVoiceTrigger   240.1
com.apple.driver.ApplePMPFirmware   1
com.apple.driver.AppleEventLogHandler   1
com.apple.driver.AppleSPMIPMU   1.0.1
com.apple.driver.AppleT6001PMGR   1
com.apple.driver.AppleS5L8960XNCO   1
com.apple.driver.AppleM68Buttons   1.0.0d1
com.apple.driver.AppleDPDisplayTCON   1
com.apple.driver.AppleS8000DWI   1.0.0d1
com.apple.driver.AppleS8000AES   1
com.apple.driver.AppleInterruptControllerV2   1.0.0d1
com.apple.driver.AppleMCDP29XXUpdateSupport   1
com.apple.driver.AppleJPEGDriver   5.4.0
com.apple.driver.AppleAVD   709
com.apple.driver.AppleAVE2   660.23.1
com.apple.driver.usb.AppleSynopsysUSB40XHCI   1
com.apple.AGXG13X   227.5.14
com.apple.driver.AppleSamsungSerial   1.0.0d1
com.apple.driver.AppleSerialShim   1
com.apple.driver.AppleMobileDispT600X-DCP   140.0
com.apple.driver.AppleProResHW   275.6.1
com.apple.driver.AppleSDXC   3.4.1
com.apple.driver.AppleT8110DART   1
com.apple.driver.AppleBluetoothModule   1
com.apple.driver.AppleBCMWLANBusInterfacePCIe   1
com.apple.driver.AppleS5L8920XPWM   1.0.0d1
com.apple.driver.AudioDMAController-T600x   240.2
com.apple.driver.AppleT6000DART   1
com.apple.driver.AppleSPIMC   1
com.apple.driver.AppleS5L8940XI2C   1.0.0d2
com.apple.driver.AppleT6000   1
com.apple.iokit.IOUserEthernet   1.0.1
com.apple.driver.usb.AppleUSBUserHCI   1
com.apple.iokit.IOKitRegistryCompatibility   1
com.apple.iokit.EndpointSecurity   1
com.apple.driver.AppleDiskImages2   198.100.13
com.apple.AppleSystemPolicy   2.0.0
com.apple.nke.applicationfirewall   403
com.apple.kec.InvalidateHmac   1
com.apple.kec.Compression   1
com.apple.kec.AppleEncryptedArchive   1
com.apple.driver.driverkit.serial   6.0.0
com.apple.driver.AppleBluetoothHIDKeyboard   231
com.apple.driver.usb.IOUSBHostHIDDevice   1.2
com.apple.driver.usb.cdc.ncm   5.0.0
com.apple.driver.usb.cdc.ecm   5.0.0
com.apple.driver.usb.cdc.acm   5.0.0
com.apple.driver.usb.serial   6.0.0
com.apple.driver.usb.cdc   5.0.0
com.apple.driver.AppleUSBAudio   540.8
com.apple.iokit.IOAudioFamily   440.2
com.apple.vecLib.kext   1.2.0
com.apple.driver.usb.AppleUSBXHCIPCI   1.2
com.apple.iokit.IOAVBFamily   1140.6
com.apple.plugin.IOgPTPPlugin   1140.25
com.apple.driver.AppleHIDKeyboard   231
com.apple.driver.AppleActuatorDriver   6440.7
com.apple.driver.AppleMultitouchDriver   6440.7
com.apple.driver.AppleHSBluetoothDriver   6440.7
com.apple.driver.IOBluetoothHIDDriver   9.0.0
com.apple.driver.AppleThunderboltPCIUpAdapter   4.1.1
com.apple.driver.AppleThunderboltDPOutAdapter   8.5.1
com.apple.driver.AppleMesaSEPDriver   100.99
com.apple.iokit.IOBiometricFamily   1
com.apple.driver.DiskImages.KernelBacked   493.0.0
com.apple.driver.AppleXsanScheme   3
com.apple.driver.usb.networking   5.0.0
com.apple.driver.AppleThunderboltUSBDownAdapter   1.0.4
com.apple.driver.AppleThunderboltPCIDownAdapter   4.1.1
com.apple.driver.AppleThunderboltDPInAdapter   8.5.1
com.apple.driver.AppleThunderboltDPAdapterFamily   8.5.1
com.apple.driver.AppleSEPHDCPManager   1.0.1
com.apple.driver.AppleTrustedAccessory   1
com.apple.iokit.AppleSEPGenericTransfer   1
com.apple.nke.ppp   1.9
com.apple.driver.AppleBSDKextStarter   3
com.apple.kext.triggers   1.0
com.apple.driver.AppleHIDTransportSPI   6440.8
com.apple.driver.AppleHIDTransport   6440.8
com.apple.driver.AppleInputDeviceSupport   6440.8
com.apple.filesystems.hfs.encodings.kext   1
com.apple.driver.AppleSyntheticGameController   10.5.10
com.apple.driver.AppleBTM   1.0.1
com.apple.driver.IOHIDPowerSource   1
com.apple.driver.AppleCallbackPowerSource   1
com.apple.driver.AppleConvergedIPCOLYBTControl   1
com.apple.driver.AppleConvergedPCI   1
com.apple.driver.AppleBluetoothDebug   1
com.apple.driver.ApplePassthroughPPM   3.0
com.apple.driver.AppleDiagnosticDataAccessReadOnly   1.0.0
com.apple.driver.AppleDCPDPTXProxy   1.0.0
com.apple.driver.DCPDPFamilyProxy   1
com.apple.driver.AppleCSEmbeddedAudio   640.8
com.apple.driver.AppleAOPAudio   240.5
com.apple.driver.AppleEmbeddedAudio   640.8
com.apple.iokit.AppleARMIISAudio   240.2
com.apple.driver.DCPAVFamilyProxy   1
com.apple.AGXFirmwareKextG13XRTBuddy   227.5.14
com.apple.AGXFirmwareKextRTBuddy64   227.5.14
com.apple.driver.AppleStockholmControl   1.0.0
com.apple.iokit.IONVMeFamily   2.1.0
com.apple.driver.AppleNANDConfigAccess   1.0.0
com.apple.driver.AppleDialogPMU   1.0.1
com.apple.driver.AppleSPMI   1.0.1
com.apple.driver.ApplePMGR   1
com.apple.driver.AppleSPU   1
com.apple.driver.AppleARMWatchdogTimer   1
com.apple.driver.AppleHPM   3.4.4
com.apple.driver.AppleUSBXDCIARM   1.0
com.apple.driver.AppleUSBXDCI   1.0
com.apple.iokit.IOUSBDeviceFamily   2.0.0
com.apple.driver.usb.AppleSynopsysUSBXHCI   1
com.apple.driver.usb.AppleUSBXHCI   1.2
com.apple.driver.AppleEmbeddedUSBHost   1
com.apple.driver.usb.AppleUSBHub   1.2
com.apple.driver.usb.AppleUSBHostCompositeDevice   1.2
com.apple.driver.AppleT6000TypeCPhy   1
com.apple.driver.AppleT8103TypeCPhy   1
com.apple.iokit.IOGPUFamily   65.40.15
com.apple.driver.AppleSART   1
com.apple.iokit.IOMobileGraphicsFamily-DCP   343.0.0
com.apple.driver.AppleDCP   1
com.apple.driver.AppleFirmwareKit   1
com.apple.iokit.IOMobileGraphicsFamily   343.0.0
com.apple.driver.AppleDisplayCrossbar   1.0.0
com.apple.iokit.IODisplayPortFamily   1.0.0
com.apple.driver.AppleTypeCPhy   1
com.apple.driver.AppleThunderboltNHI   7.2.81
com.apple.driver.AppleT6000PCIeC   1
com.apple.iokit.IOThunderboltFamily   9.3.3
com.apple.driver.ApplePIODMA   1
com.apple.driver.AppleT600xPCIe   1
com.apple.driver.AppleMultiFunctionManager   1
com.apple.driver.AppleBluetoothDebugService   1
com.apple.driver.AppleBCMWLANCore   1.0.0
com.apple.iokit.IO80211Family   1200.13.0
com.apple.driver.IOImageLoader   1.0.0
com.apple.driver.AppleOLYHAL   1
com.apple.driver.corecapture   1.0.4
com.apple.driver.AppleEmbeddedPCIE   1
com.apple.driver.AppleMCA2-T600x   740.3
com.apple.driver.AppleEmbeddedAudioLibs   240.1
com.apple.driver.AppleFirmwareUpdateKext   1
com.apple.driver.AppleH13CameraInterface   7.422.3
com.apple.driver.AppleH10PearlCameraInterface   17.0.8
com.apple.driver.AppleGPIOICController   1.0.2
com.apple.driver.AppleFireStormErrorHandler   1
com.apple.driver.AppleMobileApNonce   1
com.apple.driver.usb.AppleUSBHostPacketFilter   1.0
com.apple.iokit.IOTimeSyncFamily   1140.25
com.apple.driver.DiskImages   493.0.0
com.apple.iokit.IOGraphicsFamily   597
com.apple.iokit.IOBluetoothSerialManager   9.0.0
com.apple.iokit.IOBluetoothHostControllerUSBTransport   9.0.0
com.apple.iokit.IOBluetoothHostControllerUARTTransport   9.0.0
com.apple.iokit.IOBluetoothHostControllerTransport   9.0.0
com.apple.driver.IOBluetoothHostControllerPCIeTransport   9.0.0
com.apple.iokit.IOBluetoothFamily   9.0.0
com.apple.iokit.CSRBluetoothHostControllerUSBTransport   9.0.0
com.apple.iokit.BroadcomBluetoothHostControllerUSBTransport   9.0.0
com.apple.driver.AppleSSE   1.0
com.apple.driver.AppleSEPKeyStore   2
com.apple.driver.AppleUSBTDM   552
com.apple.iokit.IOUSBMassStorageDriver   235.100.2
com.apple.iokit.IOPCIFamily   2.9
com.apple.iokit.IOSCSIBlockCommandsDevice   482.100.4
com.apple.iokit.IOSCSIArchitectureModelFamily   482.100.4
com.apple.driver.AppleRSMChannel   1
com.apple.iokit.IORSMFamily   1
com.apple.driver.AppleIPAppender   1.0
com.apple.driver.AppleFDEKeyStore   28.30
com.apple.driver.AppleEffaceableStorage   1.0
com.apple.driver.AppleCredentialManager   1.0
com.apple.driver.KernelRelayHost   1
com.apple.iokit.IOUSBHostFamily   1.2
com.apple.driver.AppleUSBHostMergeProperties   1.2
com.apple.driver.usb.AppleUSBCommon   1.0
com.apple.driver.AppleSMC   3.1.9
com.apple.driver.RTBuddy   1.0.0
com.apple.driver.AppleEmbeddedTempSensor   1.0.0
com.apple.driver.AppleARMPMU   1.0
com.apple.iokit.IOAccessoryManager   1.0.0
com.apple.driver.AppleOnboardSerial   1.0
com.apple.iokit.IOSkywalkFamily   1.0
com.apple.driver.mDNSOffloadUserClient   1.0.1b8
com.apple.iokit.IONetworkingFamily   3.4
com.apple.iokit.IOSerialFamily   11
com.apple.driver.AppleSEPManager   1.0.1
com.apple.driver.AppleA7IOP   1.0.2
com.apple.driver.IOSlaveProcessor   1
com.apple.driver.AppleBiometricSensor   2
com.apple.iokit.IOHIDFamily   2.0.0
com.apple.driver.AppleANELoadBalancer   6.404.2
com.apple.driver.AppleH11ANEInterface   6.404.0
com.apple.AUC   1.0
com.apple.iokit.IOAVFamily   1.0.0
com.apple.iokit.IOHDCPFamily   1.0.0
com.apple.iokit.IOCECFamily   1
com.apple.iokit.IOAudio2Family   1.0
com.apple.driver.AppleIISController   240.1
com.apple.driver.AppleAudioClockLibs   240.1
com.apple.driver.AppleM2ScalerCSCDriver   265.0.0
com.apple.iokit.IOSurface   336.40.12
com.apple.driver.IODARTFamily   1
com.apple.driver.FairPlayIOKit   71.1.0
com.apple.driver.AppleARMPlatform   1.0.2
com.apple.iokit.IOSlowAdaptiveClockingFamily   1.0.0
com.apple.iokit.IOReportFamily   47
com.apple.security.quarantine   4
com.apple.security.sandbox   300.0
com.apple.iokit.IOStorageFamily   2.1
com.apple.kext.AppleMatch   1.0.0d1
com.apple.driver.AppleMobileFileIntegrity   1.0.5
com.apple.iokit.CoreAnalyticsFamily   1
com.apple.security.AppleImage4   5.0.0
com.apple.kext.CoreTrust   1
com.apple.iokit.IOCryptoAcceleratorFamily   1.0.1
com.apple.kec.pthread   1
com.apple.kec.Libm   1
com.apple.kec.corecrypto   12.0

** Stackshot Succeeded ** Bytes Traced 575360 (Uncompressed 1505072) **
roemer
 
Posts: 73
Joined: Sat Mar 15, 2014 2:32 pm

Next

Return to OpenZFS on OS X Development

Who is online

Users browsing this forum: No registered users and 3 guests

cron