Absolute noob trying to setup mirrored vdevs

[dangriga]

Hi all

I am a videographer that recently bought the owc thunderbay 8 with so far 2x20TB drives. my plan is to have a mirrored setup which i can expend in the future.
I must admit that I am completely overwhelmed. I have searched the internet for hours for a real guide for absolute beginners but so far haven't been successful.

Problem is I cannot even create a pool. I used this command but don't have permission...

zpool create pool1 mirror disk4 disk6
cannot open '/dev/disk4': Permission denied
cannot open '/dev/disk6': Permission denied
cannot create 'pool1': permission denied

No idea why the disks are named disk4 and disk6 as I put them in the first two slots of the thunderbay.

I then tried to give my user permission via the zfs allow command but I only find this where people have already successfully created a pool (for example with this command; chmod A+user:marks:add_subdirectory:fd:allow /tank)


If I am finally able to create a pool I will have 20TB of useable storage. How do I get my files on there as there are two drives in use as one is a mirror? Do I just copy them on the first drive and leave the second one as is? How does that even work?

I hope you can help me here...

[jawbroken]

you'll need to put "sudo" before those commands (and most ZFS commands), so "sudo zpool create ...", which executes the command with elevated permissions. it will prompt you for your password. once done, the two drives will appear as a single drive on your mac. it sounds like you might have already formatted the drives, if they're already showing up in Finder, in which case i'm not sure if it will work or if you'll have to erase them first. if you already have files on the drives then you'll need to back them up somewhere else, create the ZFS pool, then copy them back

[dangriga]

Figured that one out in the meantime using Chat-GPT.
I have another issue now with the drives. Using diskutil list it tells me this:

/dev/disk4 (external, physical):
#: TYPE NAME SIZE IDENTIFIER
0: GUID_partition_scheme *20.0 TB disk4
1: EFI EFI 209.7 MB disk4s1
2: Apple_APFS Container disk6 20.0 TB disk4s2

/dev/disk5 (external, physical):
#: TYPE NAME SIZE IDENTIFIER
0: GUID_partition_scheme *20.0 TB disk5
1: EFI EFI 209.7 MB disk5s1
2: Apple_APFS Container disk7 20.0 TB disk5s2

/dev/disk6 (synthesized):
#: TYPE NAME SIZE IDENTIFIER
0: APFS Container Scheme - +20.0 TB disk6
Physical Store disk4s2
1: APFS Volume Disk1 958.5 KB disk6s1

/dev/disk7 (synthesized):
#: TYPE NAME SIZE IDENTIFIER
0: APFS Container Scheme - +20.0 TB disk7
Physical Store disk5s2
1: APFS Volume Disk2 966.7 KB disk7s1




Feeding this info to Chat-GPT it told me this:

Based on the output you provided, it appears that your ThunderBay enclosure is presenting the individual drives as synthesized disks (/dev/disk6 and /dev/disk7) instead of the physical disks (/dev/disk4 and /dev/disk5). This can happen when the enclosure includes a hardware RAID controller or presents a virtualization layer to the operating system.

The synthesized disks represent the logical volumes created within the enclosure's RAID or virtualization configuration. Unfortunately, ZFS does not directly support using these synthesized disks as physical devices for creating a mirrored VDEV configuration. ZFS prefers direct access to the physical disks to ensure reliability and data integrity.

To utilize ZFS with your ThunderBay 8 enclosure, you will need to disable any RAID or virtualization features within the enclosure's settings or firmware. This will allow the individual drives to appear as separate physical devices (/dev/disk4 and /dev/disk5) instead of synthesized disks.


Now I am wondering if ZFS is even possible with Thunderbay 8 or if i should just stick to RAID 10.

[FadingIntoBlue]

If your Thunderbay is not faulty, it will certainly be possible to create working pools. Search the forum for Thuderbay 8, other have done it

Below, my sequence for creating pools, refined over many years. It isn't the only way to do it, but it works for me

================== 2021 01 16 ==================

## Clean Whole Disk Creation
------------------------------------
sudo diskutil partitiondisk disk8 GPT 'Free Space' Empty 100%


## Pool create settings [zpool]
------------------------------------
sudo zpool create -n (test command will create what we want)
sudo zpool create -f (force when drives otherwise not accessible)
sudo zpool create (go live)
zpool options [those that **must** be set at creation time or we have a problem]
-o ashift=12 [block size]
-O casesensitivity=insensitive
-O normalization=formD
-O checksum=edonr [fastest checksum for zfs, faster than skein]
-O compression=lz4

Command is:

sudo zpool create -n -o ashift=12 -O casesensitivity=insensitive -O normalization=formD -O checksum=edonr -O compression=lz4 -O atime=off -O encryption=on -O keyformat=passphrase <poolname> <mirror/raidz> disk(n) disk(n+1)

Replace < > sections with actual name, pool type and disk(s)
Remove or replace -n with -f when test run with -n is working OK

For a pool with an encrypted root filesystem, command is:

sudo zpool create -n -o ashift=12 -O casesensitivity=insensitive -O normalization=formD -O checksum=edonr -O compression=lz4 -O atime=off -O encryption=on -O keyformat=passphrase <poolname> <mirror/raidz> disk(n) disk(n+1)

so for you:
sudo zpool create -n -o ashift=12 -O casesensitivity=insensitive -O normalization=formD -O checksum=edonr -O compression=lz4 -O atime=off tank mirror disk4 disk6

remember
-n is the test flag, remove for actual creation

## Pool property settings post-creation
------------------------------------

sudo zfs set reservation=1m <poolname>
sudo zfs set com.apple.ignoreowner=on <poolname>
sudo zfs set com.apple.mimic=hfs <poolname>


### And to work well with the macOS side after creation:

Dataset property settings post-creation
------------------------------------

### User Delegations
------------------------------------

sudo zfs allow -g zfsusers compression,create,destroy,mount,mountpoint,send,receive,snapshot,rollback <poolname>

## MacOS settings on Pool/Dataset
### User Permissions
sudo chmod -R 775 /Volumes/<poolname>

I hope that is of some help

[FadingIntoBlue]

Oops, noticed I'd doubled up on the encryption instructions. If you don't want encryption, omit these flags on creation

-O encryption=on -O keyformat=passphrase

[dangriga]

Ok I was able to create a pool now.
I did everything with your commands except for the ones below.
I then restarted the mac but now the pool is not found again. what am I doing wrong?

This I don't understand how to put in the terminal:

### And to work well with the macOS side after creation:

Dataset property settings post-creation
------------------------------------

### User Delegations - Did not work for me....
------------------------------------

sudo zfs allow -g zfsusers compression,create,destroy,mount,mountpoint,send,receive,snapshot,rollback <poolname>

## MacOS settings on Pool/Dataset
### User Permissions
sudo chmod -R 775 /Volumes/<poolname>

[FadingIntoBlue]

Ok I was able to create a pool now.
I did everything with your commands except for the ones below.
I then restarted the mac but now the pool is not found again. what am I doing wrong?

I would guess that the pool isn't imported automatically:

Code: Select all

sudo pool import

will give you a list of importable pools and instructions about how to mount them

This I don't understand how to put in the terminal:

### And to work well with the macOS side after creation:

Dataset property settings post-creation
------------------------------------

Sorry to be a bit clearer, 3 separate commands, and for pool name. substitute the name of the pool you created

Code: Select all

sudo zfs set reservation=1m <poolname>

Code: Select all

sudo zfs set com.apple.ignoreowner=on <poolname>

Code: Select all

sudo zfs set com.apple.mimic=hfs  <poolname>

The first one leaves some spare space in the pool, as things don't work well when the complete allocation of space is used up
The second avoids 'some' macOS permission issues, if you use the pool on more than one computer or use it with more than one user account
The third one avoids issues with some Apple primary programs, such as Music or Photos, by filling macOS into treating the zfs filesystem as an hfs+ drive

These are just a small number of the setting options available. either to the pool as a whole, or to the filesystems in the pool.

### User Delegations - Did not work for me....
------------------------------------

sudo zfs allow -g zfsusers compression,create,destroy,mount,mountpoint,send,receive,snapshot,rollback <poolname>

Sorry that is entirely my fault, I setup each of my users as within the group zfsusers, makes it simpler to provide consistent permissions

Instead , try this:

Code: Select all

sudo zfs allow <username> compression,create,destroy,mount,mountpoint,send,receive,snapshot,rollback <poolname>

Where username is replaced by the the short name for your macOS user, and <poolname> replaced the name of the pool

## MacOS settings on Pool/Dataset
### User Permissions

Code: Select all

sudo chmod -R 775 /Volumes/<poolname>

The issue here is probably a case of where your pool is mounted. Make sure you have the complete pathname, either by using the df command in terminal and finding the line ending with your poolname
eg:

/dev/disk19s1 11519180480 86215176 11432965304 1% 19289 1429120663 0% /Volumes/NewPool

- where New Pool is the name I am looking for, and /Volumes/NewPool the complete pathname

Some pools or filesystems may be mounted under root: /NewPool, or further down the tree /Volumes/NewPool/Media

Hope that clarifies things a bit

[lundman]

Please skip the user-delegation "zpool allow" parts, just use "sudo" to work with the zpool/zfs commands.

When you boot, launchctl is supposed to run "import-all" script to bring in your pools, there has been a bug here so it is possible that it
isnt importing them. You can import manually of course. Anyone remember which version had imports fixed?

[jawbroken]

at some point i had to give Full Disk Access permission to /bin/bash or /bin/zsh to get auto-import to work. it works for me in 2.1.6

edit: for completeness, i also did the same for /bin/sh and for zfs and zpool in /usr/local/zfs/bin/. i have no idea which of those is actually required

[dangriga]

You guys rule! This all helped me alot.
In the meantime, I expanded my pool by 2 additional mirrored vdevs

It looks like this:

Code: Select all

   NAME                                            STATE     READ WRITE CKSUM
   tank                                            ONLINE       0     0     0
     mirror-0                                      ONLINE       0     0     0
       media-430F38EF-905F-E841-86A0-F4F813B4BD0C  ONLINE       0     0     0
       media-69F261D0-FC13-234B-8012-41AE575BA769  ONLINE       0     0     0
     mirror-1                                      ONLINE       0     0     0
       media-FD527821-CC7B-FA46-9569-D19CF2196ECC  ONLINE       0     0     0
       media-3F2F7D1C-3EE5-064B-B7E0-6BCECC79F89A  ONLINE       0     0     0

Currently I am still importing the pool after starting the computer but that's fine. I am still waiting for my new m2 mac to arrive anyways and will set the autoimport then.

My question now are 2 things before I start loading data onto the tank:

1. GPT IDs necessary? Do I need this and if yes for what?
2. Would be neat if my disks were renamed with their serialnumber (which i wrote down on each drive in the front of the thunderbay). In the case of one failing, i can be 100% sure which drive it was.
How can I do this? If i have to remake a pool so be it. Wouldn't be too much work.

Thanks again!