Are 1.7.1 new pools compatible with 1.5.2?

[nodarkthings]

Hi!
Thanks for the long-awaited update!
The issue I had on Mavericks with v1.6.1 seems gone.
(https://openzfsonosx.org/forum/viewtopic.php?f=15&t=3012)
At least as long as I boot in my test OS 10.9.5 where I installed 1.7.1...
When I boot back to my main system (still running 1.5.2):
- the first pool I created with 1.7.1 (boldly named Test ) is not imported and the underlying HFS partition (named ZFS1) is mounted instead.
- the Console gets filled with so many of those lines:

Code: Select all

14/03/18 10:19:47,000 kernel[0]: hfs: BlockMarkFreeInternal() trying to free unallocated blocks on volume ZFS1
14/03/18 10:19:47,000 kernel[0]: hfs: Runtime corruption detected on ZFS1, fsck will be forced on next mount.


If I eject ZFS1 and import Test, I see the few files I dropped there under 1.7.1 and zpool status shows:

Code: Select all

pool: Test
state: ONLINE
status: One or more devices has experienced an error resulting in data corruption.  Applications may be affected.
action: Restore the file in question if possible.  Otherwise restore the entire pool from backup.
see: http://zfsonlinux.org/msg/ZFS-8000-8A
scan: none requested
config:
   NAME                                           STATE     READ WRITE CKSUM
   Test                                           ONLINE       0     0     4
     volume-65714D12-064E-3A8D-A56D-676C625471EA  ONLINE       0     0    15
errors: Permanent errors have been detected in the following files:
        /Volumes/Test/.VolumeIcon.icns
        /Volumes/Test/.Spotlight-V100/Store-V1/VolumeConfig.plist
        /Volumes/Test/.Spotlight-V100/VolumeConfiguration.plist

The strangest thing is that the second pool I created together with the first one automounts without issue and its zpool status shows "No known data errors".

So, before I try the move to 1.7.1 on all my OSes,I need to know if there should be a compatibility of the new pools or if this is normal behavior?
Especially if I upgrade my pools, there'll probably be no turning back...
On the good side, my old pools don't seem affected at all and mount correctly on both versions.

[nodarkthings]

Very very disappointed: I've updated all my system partitions to 1.7.1, recreated my test pool... and on the next boot, once again, the underlying hfs partition mounted instead of the zfs pool.
I think I'm good to come back to 1.5.2 once again and stay there forever...

[lundman]

What do you mean by "underlying hfs" ? What is it exactly you have done, and how did you create the pool?

[Sharko]

I think he means that he followed the instructions for created a FileVault-encrypted partition that initially looks like HFS, until you overwrite it with ZFS, as outlined here:

https://openzfsonosx.org/wiki/Encryptio ... Vault_2.29

[nodarkthings]

Hi!
No, no... I've just followed the basic instructions I follow since version 1.3:
- create a hfs partition (in this case called "ZFS1")
- unmount it
- sudo zpool create -f -o ashift=12 -O compression=lz4 -O casesensitivity=insensitive -O atime=off -O normalization=formD TEST /dev/disk2s3 (for example)

I'm using hard disks with a blend of hfs partitions and zfs pools.
Never had such issues until v1.5.2 but it seems there was a change in 1.6.1: I couldn't even boot when a drive with a newly created pool was connected! The only solution was to come back to v1.5.2.

Now, that issue is gone in 1.7.1, at least, but the newly created pool doesn't stick after reboot: the hfs partition mounts instead of the zfs pool and the Console gets filled with so many of the lines I mentioned "Runtime corruption detected", etc.
There's also, repeated a number of times:

Code: Select all

15/03/18 01:15:43,649 fseventsd[87]: get_last_event_id: checksum mis-match: file 0x00000000, calculated 0x00000001 in file /Volumes/ZFS1/.fseventsd/fc007596cb52bd56
15/03/18 01:15:43,649 fseventsd[87]: failed to get last id from log file 11 so using last id == 0xc007596cb52bd55 (ret 22; fc007596cb52bd56)
15/03/18 01:15:43,833 fseventsd[87]: process_disk_event_buf: checksum mis-match: file 0x00000000, calculated 0x00000001
15/03/18 01:15:43,833 fseventsd[87]: read_disk_log: error processing log file /Volumes/ZFS1/.fseventsd/fc007596cb4bd7be (total_read 75, size 0)
15/03/18 01:15:44,168 fseventsd[87]: process_disk_event_buf: checksum mis-match: file 0x00000000, calculated 0x00000001
15/03/18 01:15:44,176 fseventsd[87]: read_disk_log: error processing log file /Volumes/ZFS1/.fseventsd/fc007596cb52bd56 (total_read 75, size 0)


Lundman, from your answer I understand that the recommended practice would be to dedicate a complete HD to ZFS? — which is not my use case as I also have various OS partitions on every drive.
The point is that all is fine with v1.5.2... So I would just need to boot to a different OS with v1.5.2 in it if I need to create new pools, but that's a bit weird...

[chrryd]

That almost sounds like partition tables are messed... What does the output of 'diskutil list' look like ?

[nodarkthings]

Nothing strange with 'diskutil list' but you gave me the input for a few interesting tests!
I now have an OS partition with 1.5.2 and another one with 1.7.1.

- I create a first zfs pool under 1.5.2: mounts normally under 1.7.1:
=> Console shows no message "Runtime corruption detected, etc."
=> Diskutil shows no name for the "underlying" HFS partition

Code: Select all

10:                  Apple_HFS                         59.6 GB    disk1s10

- I create a second zfs pool under 1.7.1: mounts normally under 1.5.2 BUT:
=> Console shows hundreds of messages "Runtime corruption detected"
=> Diskutil still shows the name for the "underlying" HFS partition (named "ZFS2")

Code: Select all

11:                  Apple_HFS ZFS2                    59.2 GB    disk1s11

- when booting back to the OS with 1.7.1:
=> Console shows no messages "Runtime corruption detected"
=> the HFS partition named "ZFS2" mounts
=> Diskutil still shows its name, of course

Code: Select all

11:                  Apple_HFS ZFS2                    59.2 GB    disk1s11

So, I think this proves there's a difference in pool creation in 1.5.2 compared to more recent versions.
(I recall you that I'm on MacOS 10.9.5, so it's very likely that something was needed for that build that has disappeared after 1.5.2 — or on the contrary, something that has been added and creates this issue)

[Brendon]

@nodarkthings

In the interests of clarity, please post exact copies of shell sessions where you demonstrate this. This would be very helpful!

[nodarkthings]

What do you mean by "exact copies of shell sessions"? I haven't modified none of the code I mentioned, just copied as it displayed, and tried to be as compact as possible, "in the interests of clarity", indeed...
Excuse me if I don't understand perfectly, I'm French.
Did you mean the complete output of 'diskutil list' for each trial? Well it was exactly the same, except for those lines, but if you really want, I can redo my tests once again.

[nodarkthings]

I think this time we can call it FIXED!
It looks like there was some remains of older versions that were the cause of my issues.
- I ran the 1.7.1 uninstaller then the 1.5.2 uninstaller
- I searched for anything with zfs in its name and ended up ditching those files:
org.openzfsonosx.zconfigd.err
org.openzfsonosx.zconfigd.log
org.openzfsonosx.zed.err
org.openzfsonosx.zed.log
org.openzfsonosx.zpool-import-all.err
org.openzfsonosx.zpool-import-all.log
(though I don't think they matter)
- reinstalled v1.7.1
- rebooted various times on my different OS partitions
=> the newly created partitions stick and no error messages in the Console!

Conclusion: I must say that I always had errors updating to a new version (since 1.3? 1.2.7? who knows...), but as it always worked in the end I thought may be it was the 10.9 build that wasn't optimized... Indeed, when I first installed 1.7.1 on my test partition, I was surprised it installed without error, but that's probably because I had restored it to a clean system last year, before reinstalling 1.5.2, which was not the case for my main OS partition.