2.1.6RC Feedback

Developer discussions.

2.1.6RC Feedback

Postby Jimbo » Sun Oct 23, 2022 3:27 pm

Dropped it on a Monterey VM and then an iMac Prod running Monterey 12.6. Observations:

Upgrading from 2.1.99 test. Install went through without any issues after exporting my pools. Upon reboot though, the kext wasn't loaded. Running the installer again and another reboot and I could see my pools ready to import.

My pools are on Samsung T7 SSDs (each pool is a single 1Tb T7), they're encrypted with ZFS native encryption. Importing was successful. Loading the keys was successful. Mounting the datasets failed:
Code: Select all
If importing from zfs-1.9.4 (or earlier), then possibly enable features:
    project_quota & userobj_accounting
cannot mount 'Data': Input/output error


Checking zpool status -z:
Code: Select all
  pool: Data
 state: ONLINE
status: One or more devices has experienced an error resulting in data
   corruption.  Applications may be affected.
action: Restore the file in question if possible.  Otherwise restore the
   entire pool from backup.
   see: https://openzfs.github.io/openzfs-docs/msg/ZFS-8000-8A
  scan: scrub repaired 0B in 00:14:08 with 0 errors on Sun Jul 10 13:01:01 2022
config:

   NAME                                          STATE     READ WRITE CKSUM
   Data                                          ONLINE       0     0     0
     media-BC40EBBE-FF25-AB45-BD07-2AA3A20FA0B1  ONLINE       0     0     0

errors: Permanent errors have been detected in the following files:

        Data:<0x0>


I've seen this before, but I can't remember exactly when or why. I'm going to run two scrubs now and try again...
Jimbo
 
Posts: 149
Joined: Sun Sep 17, 2017 5:12 am

Re: 2.1.6RC Feedback

Postby lundman » Sun Oct 23, 2022 3:45 pm

Thanks for the feedback.

Is it encrypted at all? I know ZOL put in a "proper" fix for the MAC using project quota. I wonder how that
will affect us.
User avatar
lundman
 
Posts: 1335
Joined: Thu Mar 06, 2014 2:05 pm
Location: Tokyo, Japan

Re: 2.1.6RC Feedback

Postby Jimbo » Sun Oct 23, 2022 3:58 pm

Yes, both pools are encrypted (with the same key, but separately).

The second scrub has just finished on both pools. They're "clean" now (zero bytes repaired).

And mounting the datasets says...

Code: Select all
sudo zfs mount -a
If importing from zfs-1.9.4 (or earlier), then possibly enable features:
    project_quota & userobj_accounting
cannot mount 'Data': Input/output error
If importing from zfs-1.9.4 (or earlier), then possibly enable features:
    project_quota & userobj_accounting
cannot mount 'Work': Input/output error


And we're back to
Code: Select all
errors: Permanent errors have been detected in the following files:

        Data:<0x0>


Sad times. :( <0x0> indicates metadata issues?
Jimbo
 
Posts: 149
Joined: Sun Sep 17, 2017 5:12 am

Re: 2.1.6RC Feedback

Postby Jimbo » Sun Oct 23, 2022 4:49 pm

Well, a tiny bit of progress but no success.

Because I have a backup ( :) ) I upgraded the Data pool. Now, at least, if I try to import the pool I do not end up with
Code: Select all
errors: Permanent errors have been detected in the following files:

        Data:<0x0>


after attempting to mount the datasets. I do still get:
Code: Select all
% sudo zfs mount -a
If importing from zfs-1.9.4 (or earlier), then possibly enable features:
    project_quota & userobj_accounting
cannot mount 'Data': Input/output error


when attempting to mount.

Upgrading the pool enabled:
Code: Select all
Enabled the following features on 'Data':
  zilsaxattr
  head_errlog
  blake3


Plugging the Data pool into Fedora running zfs-2.1.6-1.fc36.x86_64, I see that the following is not supported on 2.1.6 (@Lundman, you did say it was loosely based on upstream 2.1.6, looks like one feature overshoots 2.1.6 on ZoL at least :) ):
Code: Select all
The pool uses the following feature(s) not supported on this system:
   com.delphix:head_errlog (Support for per-dataset on-disk error logs.)


Looks like I'll be using that backup! :)

I'll also see if I can't try out creating a new encrypted pool with O3X 2.1.6RC just to make sure it works. At the moment though, it doesn't look like I'll be able to roll my [encrypted] pools up to 2.1.6 from 2.1.0/2.1.99test.

Cheers!
Jimbo
 
Posts: 149
Joined: Sun Sep 17, 2017 5:12 am

Re: 2.1.6RC Feedback

Postby Jimbo » Sun Oct 23, 2022 5:09 pm

Well, with 2.1.6RC, if I create a new encrypted pool, create an encrypted dataset, everything seems to be working as I'd expect - export, import, load key, mount, copy data, etc all seem ok.

Seems to be an issue with older encrypted pools?
Jimbo
 
Posts: 149
Joined: Sun Sep 17, 2017 5:12 am

Re: 2.1.6RC Feedback

Postby lundman » Sun Oct 23, 2022 5:53 pm

Hmm ok, so that is going to be annoying.

There was a difference in how MAC was done. MAC/FreeBSD and Illumos uses user/group quotas, but not project quota, as that
did not yet exist. Linux uses all three.

So in the past, the 2.1.99 version, I added special code that will try the 2 quotas, and if that failed, try with 3 quotas. So
you can import encrypted datasets from old macOS, as well as, Linux.

Linux went and fixed the issue "their way", so now everyone runs the same code, and my special code is removed.

So if you go from old macOS straight to 2.1.6 you would see this issue. If you go from old macOS, to development version (2.1.99) with the special code, then
2.1.6 it should work fine.

I wonder if a lot of people will do this, and I should put back the 2 quota code.
User avatar
lundman
 
Posts: 1335
Joined: Thu Mar 06, 2014 2:05 pm
Location: Tokyo, Japan

Re: 2.1.6RC Feedback

Postby lundman » Sun Oct 23, 2022 5:54 pm

Oh your data is fine, and is still there. It is just the MAC used to make sure the password entered is correct. If it isn't, it doesn't do anything. So you can always go back to old version (stay away from "zpool upgrade" though, until you are happy).
User avatar
lundman
 
Posts: 1335
Joined: Thu Mar 06, 2014 2:05 pm
Location: Tokyo, Japan

Re: 2.1.6RC Feedback

Postby Jimbo » Sun Oct 23, 2022 6:53 pm

lundman wrote:So if you go from old macOS straight to 2.1.6 you would see this issue. If you go from old macOS, to development version (2.1.99) with the special code, then
2.1.6 it should work fine.

I wonder if a lot of people will do this, and I should put back the 2 quota code.


I'm not sure I'm following along there...

I created these pools with O3X 2.1.0 (maybe 2.0.1, but definitely not the 1.9.x series) and I've been running 2.1.99 (from OpenZFSonOsX-2.1.99-Catalina-10.15) with these for many months and now dropped in 2.1.6 and hitting this issue. Were you suggesting the pools would be ok if the pools were [re]built with 2.1.99?

If I need to migrate data from the old pools to new pools, I'll grumble a bit, but I could do it. However, its going to be a real pain in the arse if I can't mount the old pool and new pool to the same machine (which is the current situation). :)
Jimbo
 
Posts: 149
Joined: Sun Sep 17, 2017 5:12 am

Re: 2.1.6RC Feedback

Postby Jimbo » Sun Oct 23, 2022 8:36 pm

Created a new pool with 2.1.99:

Code: Select all
% sudo zpool create -f -o ashift=12 -O encryption=on -O keylocation=file:///Users/user/.keyFile -O keyformat=passphrase -O compression=lz4 -O atime=off -O casesensitivity=sensitive -O normalization=formD Data disk2

% sudo zfs create Data/Testing


Attempt to import on 2.1.6:

Code: Select all
% sudo zpool import Data

% sudo zfs load-key -a && sudo zfs mount -a
1 / 1 key(s) successfully loaded
If importing from zfs-1.9.4 (or earlier), then possibly enable features:
    project_quota & userobj_accounting
cannot mount 'Data': Input/output error

% sudo zpool status -v
  pool: Data
 state: ONLINE
status: One or more devices has experienced an error resulting in data
   corruption.  Applications may be affected.
action: Restore the file in question if possible.  Otherwise restore the
   entire pool from backup.
   see: https://openzfs.github.io/openzfs-docs/msg/ZFS-8000-8A
config:

   NAME                                          STATE     READ WRITE CKSUM
   Data                                          ONLINE       0     0     0
     media-E0F3403C-43BE-2943-B8B9-976E8F973126  ONLINE       0     0     0

errors: Permanent errors have been detected in the following files:

        Data:<0x0>

% sudo zfs list -t all
Password:
NAME           USED  AVAIL  REFER  MOUNTPOINT
Data           806M   898G  2.37M  /Volumes/Data
Data/Testing   796M   898G   796M  /Volumes/Data/Testing


It would seem that encrypted 2.1.99 pools and 2.1.6 don't play nice. :(
Jimbo
 
Posts: 149
Joined: Sun Sep 17, 2017 5:12 am

Re: 2.1.6RC Feedback

Postby lundman » Sun Oct 23, 2022 9:32 pm

I take it you advocate for the compatibility code to be restored, could be worth it for anyone with crypto enabled.
User avatar
lundman
 
Posts: 1335
Joined: Thu Mar 06, 2014 2:05 pm
Location: Tokyo, Japan

Next

Return to OpenZFS on OS X Development

Who is online

Users browsing this forum: No registered users and 13 guests