dataset or volume?

[mbunal]

hi

which should i choose dataset or volume? why? i wa started to copy my archive to dataset. but i wonder what is dataset and volume?

[Sharko]

Well, I can probably address half of your question, since I've become familiar with the limitations and benefits of ZFS volumes.

ZFS volumes, good points to not so good points:
+ offer absolute compatibility with the host operating system, since the OS X host sees a true HFS+ filesystem
+ all data is checksummed so you can be sure that the data has not been altered or corrupted
+ give you the option of building a self-healing system with redundancy through either mirroring (faster, but less efficient) or RAIDZ (slower, possibly more redundant)
+ allows you to utilize compression (recommended) or even deduplication (not recommended) to save storage space
+/- offer you the ability to snapshot the volume (good), but with the apparent limitation that the volume must be unmounted to snapshot it (not so good)
- requires you to choose a fixed size for the volume that cannot be changed later - you have to copy the data to a larger volume if you outgrow it (just like HFS+ requires)
- you can't go into snapshots and pull out files - all you can seemingly do with snapshots is rollback or destroy or clone to new volume
- possibly a little slower than native ZFS filesystem, since there are more layers in the software stack
- I believe you lose the benefits of copy-on-write, since the ZFS is just acting as block storage, and knows nothing about the atomicity of file system writes (since the writes are managed by HFS+)


ZFS native filesystems (datasets) have ALL the goodies: no fixed size, copy on write, checksums, redundancy, compression, deduplication, ability to snapshot on the fly, ability to access files in snapshots, ability to use scripts to rotate snapshots on live filesystems, etc.

Probably the only concern about native ZFS filesystems is compatibility with certain applications and/or portions of the operating system. I believe that ZFS emulates all the essential features of HFS (permissions, and I think even ACLs), but there may be some esoteric features of HFS+ that are not in ZFS. I have an open question regarding compatibility in this same section of the forum; no one has responded to it yet, which I suspect means that for most uses you won't run into a problem. I have seen some threads concerning Time Machine and Spotlight that make me think that these might not work out of the box on ZFS volume - or maybe they do in the latest version. Anyway, I've copied my user data to a test dataset, and I'm going to dive in and see what I encounter for compatibility first-hand.

Kurt

(post edited to reflect my better understanding of copy on write applicability to volumes)

[mbunal]

thank you very muck Kurt.

I've copied my user data to a test dataset, and I'm going to dive in and see what I encounter for compatibility first-hand.
Kurt

me too ((-;<

[mbunal]

just one question too. it was very slowly copy my archive hdd's to new 8tb zfs dataset. there was lots of files less then 10kb. it took 4 days. anyway, i am good now.

if i use volume and hfs+ format, how was change the copy speed? (i just try myself but i am out for a while...)

[haer22]

just one question too. it was very slowly copy my archive hdd's to new 8tb zfs dataset. there was lots of files less then 10kb. it took 4 days. anyway, i am good now. if i use volume and hfs+ format, how was change the copy speed? (i just try myself but i am out for a while...)

Your pool, how is it set up? mirrors? raid? single? Do you have a cache device? Do you have a log device? How much memory do you have?

Small files are really painful speed-wise, especially if you do not have a spare log-device on an SSD.

[mbunal]

Your pool, how is it set up? mirrors? raid? single? Do you have a cache device? Do you have a log device? How much memory do you have?

creation comand: sudo zpool create -f -o ashift=12 -O compression=lz4 -O casesensitivity=insensitive -O atime=off -O normalization=formD zfsArchive disk2
no mirror, no raid, only single 8 tb nas hdd. i dont have any cache device. i dont have any log device. i5 gen6 and 2x16 gb ddr4 ram (it is hackintosh).

Small files are really painful speed-wise, especially if you do not have a spare log-device on an SSD

should i use spare log-device? how much size?

[Brendon]

Perhaps you should do some basic benchmarking - copy a large video file onto that drive while running "zpool iostat 1" Post results.

Sounds like the drive is an internal SATA drive, I'd be amazed if you couldnt push 100-180MB/sec onto the drive.

- Brendon

[mbunal]

Perhaps you should do some basic benchmarking - copy a large video file onto that drive while running "zpool iostat 1" Post results.

Sounds like the drive is an internal SATA drive, I'd be amazed if you couldnt push 100-180MB/sec onto the drive.

- Brendon

1.05 gb mp4 copied from zfsroot/videos to zfsroot:

Code: Select all

~$ zpool iostat 1
               capacity     operations    bandwidth
pool        alloc   free   read  write   read  write
----------  -----  -----  -----  -----  -----  -----
zfs8tb      3,14T  4,11T      0      0  10,1K  43,9K
zfs8tb      3,14T  4,11T    960     63   119M  8,00M
zfs8tb      3,14T  4,11T  1,02K     69   129M  8,75M
zfs8tb      3,14T  4,11T    600     39  74,5M  4,98M
zfs8tb      3,14T  4,11T    407     26  50,5M  3,37M
zfs8tb      3,14T  4,11T     45      3  5,75M   512K
zfs8tb      3,14T  4,11T     43      1  5,48M   255K
zfs8tb      3,14T  4,11T    165     11  20,4M  1,49M
zfs8tb      3,14T  4,11T    140      8  17,5M  1,12M
zfs8tb      3,14T  4,11T    293     19  36,5M  2,50M
zfs8tb      3,14T  4,11T     74      4  9,21M   637K
zfs8tb      3,14T  4,11T    400     25  49,7M  3,25M
zfs8tb      3,14T  4,11T     16      0  2,12M   128K
zfs8tb      3,14T  4,11T    219     79  27,3M  10,0M
zfs8tb      3,14T  4,11T     23    100  2,99M  11,5M
zfs8tb      3,14T  4,11T     25     64  3,24M  7,55M
zfs8tb      3,14T  4,11T     38     77  4,74M  8,94M
zfs8tb      3,14T  4,11T     34     86  4,36M  10,1M
zfs8tb      3,14T  4,11T     16     30  2,12M  3,75M
zfs8tb      3,14T  4,11T     16     20  2,12M  2,61M
zfs8tb      3,14T  4,11T     27     48  3,37M  6,11M
zfs8tb      3,14T  4,11T     25     39  3,24M  4,98M
zfs8tb      3,14T  4,11T     14     29  1,87M  3,73M
zfs8tb      3,14T  4,11T     44     79  5,62M  9,99M
zfs8tb      3,14T  4,11T     11      9  1,49M  1,24M
zfs8tb      3,14T  4,11T     28     49  3,62M  6,24M
zfs8tb      3,14T  4,11T     36     60  4,49M  7,59M
zfs8tb      3,14T  4,11T     12     28  1,62M  3,61M
zfs8tb      3,14T  4,11T     28     49  3,62M  6,23M
zfs8tb      3,14T  4,11T     23     29  2,99M  3,73M
zfs8tb      3,14T  4,11T     27     49  3,37M  6,23M
zfs8tb      3,14T  4,11T     42     69  5,35M  8,72M
zfs8tb      3,14T  4,11T     12     19  1,62M  2,49M
zfs8tb      3,14T  4,11T     47     79  5,99M  9,98M
zfs8tb      3,14T  4,11T     17     29  2,24M  3,74M
zfs8tb      3,14T  4,11T     12     29  1,50M  3,73M
zfs8tb      3,14T  4,11T     46     70  5,86M  8,85M
zfs8tb      3,14T  4,11T     17     38  2,24M  4,86M
zfs8tb      3,14T  4,11T     23     39  2,99M  4,98M
zfs8tb      3,14T  4,11T     36     59  4,49M  7,47M
zfs8tb      3,14T  4,11T     11     19  1,50M  2,50M
zfs8tb      3,14T  4,11T     46     69  5,85M  8,71M
zfs8tb      3,14T  4,11T     29     49  3,74M  6,24M
zfs8tb      3,14T  4,11T     11     29  1,50M  3,74M
zfs8tb      3,14T  4,11T     35     59  4,38M  7,49M
zfs8tb      3,14T  4,11T     35     59  4,49M  7,48M
zfs8tb      3,14T  4,11T     35     50  4,48M  6,35M
zfs8tb      3,14T  4,11T     62    108  7,72M  13,6M
zfs8tb      3,14T  4,11T     26     39  3,37M  5,00M
zfs8tb      3,14T  4,11T      5      9   764K  1,24M
zfs8tb      3,14T  4,11T     47     80  5,99M  10,1M
zfs8tb      3,14T  4,11T     29     49  3,73M  6,22M
zfs8tb      3,14T  4,11T     64    108  7,98M  13,6M
zfs8tb      3,14T  4,11T     31     59  3,99M  7,48M
zfs8tb      3,14T  4,11T     46     79  5,73M  9,95M
zfs8tb      3,14T  4,11T     54     89  6,86M  11,2M
zfs8tb      3,14T  4,11T     40     59  5,10M  7,46M
zfs8tb      3,14T  4,11T     29     60  3,63M  7,61M
zfs8tb      3,14T  4,11T     34     59  4,37M  7,49M
zfs8tb      3,14T  4,11T     47     69  5,97M  8,71M
zfs8tb      3,14T  4,11T     78    139  9,72M  17,4M
zfs8tb      3,14T  4,11T     59     99  7,46M  12,4M
zfs8tb      3,14T  4,11T     52     89  6,48M  11,2M
zfs8tb      3,14T  4,11T     95    152  11,9M  19,0M
zfs8tb      3,14T  4,11T     26     46  3,24M  5,85M
zfs8tb      3,14T  4,11T     86    143  10,8M  17,9M
zfs8tb      3,14T  4,11T     18     35  2,37M  4,49M
zfs8tb      3,14T  4,11T     86    139  10,8M  17,4M
zfs8tb      3,14T  4,11T     73    129  9,22M  16,2M
zfs8tb      3,14T  4,11T     59     89  7,47M  11,2M
zfs8tb      3,14T  4,11T     44     79  5,62M  10,0M
zfs8tb      3,14T  4,11T     62    107  7,86M  13,5M
zfs8tb      3,14T  4,11T     48     81  6,11M  10,2M
zfs8tb      3,14T  4,11T     55     89  6,98M  11,2M
zfs8tb      3,14T  4,11T     33     59  4,17M  7,47M
zfs8tb      3,14T  4,11T     23     39  2,99M  4,99M
zfs8tb      3,14T  4,11T      3  1,03K   511K   132M
zfs8tb      3,14T  4,11T      0  1,24K      0   159M
zfs8tb      3,14T  4,11T      0  1,04K      0   120M
zfs8tb      3,14T  4,11T      0  1,20K      0   151M
zfs8tb      3,14T  4,11T      0    760      0  93,8M
zfs8tb      3,14T  4,11T      0    312      0  35,3M
zfs8tb      3,14T  4,11T      0  1,00K      0   129M
zfs8tb      3,14T  4,11T      0    918      0   114M
zfs8tb      3,14T  4,11T      0     30      0   699K
zfs8tb      3,14T  4,11T      0      0      0      0