Testing Corrupted data detection by OpenZFS

New to OpenZFS on OS X (Or ZFS in general)? Ask your questions here!

Testing Corrupted data detection by OpenZFS

Postby daifuku74 » Thu Jun 30, 2016 4:41 am

Hello everyone,
I am interested in getting to know openZFS better.

For now I have created a pool which displays in the finder.
I copied a zip file inside of it through finder.
When I status the pool it's all right.
Then I open the zip file with an hex editor, delete some bits here and there, save,
run the status, still ok.
So I scrub, and scrubs says it repaired nothing (!?)
NAME STATE READ WRITE CKSUM
testpool ONLINE 0 0 0
mirror-0 ONLINE 0 0 0
media-BC68C94A-CCA4-3342-A297-08FA94B481B2 ONLINE 0 0 0
media-43F53337-25AF-2245-B393-5224BD580B1E ONLINE 0 0 0

Is my bit rot simulation bad? or my scrubbing method wrong?
Or do I have it all wrong?

The wiki is really great for install but I can't really find info about file integrity, which is the reason I'm primarily interested in OpenZFS

Thanks for your clues and your time ;)
daifuku74
 
Posts: 4
Joined: Thu Jun 30, 2016 4:34 am

Re: Testing Corrupted data detection by OpenZFS

Postby macz » Thu Jun 30, 2016 5:41 am

I will take a stab at this until someone smarter comes along and corrects me or confirms it..

if you save a file in ZFS then open it (while still using ZFS as the file system) and make changes then save it again... than nothing is wrong..

you have a file that you saved and then modified.

you have to corrupt the data without the file system knowing about it. dont ask me how as that is over my pay grade.

most people will monkey with learing zfs error handling by creating a multi vdev pool and then remove a vdev... that will throw all kinds of errors, degrade the pool and require that you replace the offending vdev and rebuild.

I would think one possible way to do it the way your are doing it, would be the following....

make a pool of vdevs using files (google it) rather than spinning disks.

then export the pool

therefore zfs relenquished control over those files...

then get into one of those files with the hex edditor in the native file system not ZFS ... then re-import the pool.. now with zfs back in control you scrub the pool and it should throw checksums


my 2 cents and it may not be worth that much...
macz
 
Posts: 53
Joined: Wed Feb 03, 2016 4:54 am

Re: Testing Corrupted data detection by OpenZFS

Postby Tip » Thu Jun 30, 2016 5:52 am

I think macz is correct.

Someone will chime in and correct me if I'm wrong but I think OpenZFS(and ZFS in that matter) keeps checksums for every specific number of bits. For easier understanding let's assume that it keeps individual checksums for each file. When you open your file in an Hex editor then make some changes and save again, hex editor tells OS that it wants to save the new modified file, OS tells the file system that it needs the new file to be saved, and OpenZFS saves the new, modified file also updating its checksum. So for OpenZFS it is a new file file with a correct checksum, hence it doesn't throw an error.

To simulate a bit rot, I think you have to alter files at a deeper level, without OpenZFS knowing, for which I don't have an answer. macz's suggestion seems plausible to me though.
Tip
 
Posts: 2
Joined: Thu Jun 30, 2016 5:42 am

Re: Testing Corrupted data detection by OpenZFS

Postby Brendon » Thu Jun 30, 2016 1:56 pm

All,

There are unit tests in the testing project that demonstrate/test corruption of pools/datasets. Typically they work by DDing data into the underlying storage device.

Naturally hex-editing a file is not going to trigger ZFS errors as the filesystem is not damaged.

Cheers
Brendon

P.S. Please don't attempt to run the tests without seeking specific advice from the devs. They are/can be destructive.
Brendon
 
Posts: 286
Joined: Thu Mar 06, 2014 12:51 pm

Re: Testing Corrupted data detection by OpenZFS

Postby lundman » Thu Jun 30, 2016 4:16 pm

As said, the point with checksums, is it guarantees that the data you ask to write to disk, remains the same. If you ask ZFS to save a corrupted file, it does just that. What checksumming is for, is guarding against your data going corrupted over time, from HDD age, bad transport, cables etc.

Easiest way to corrupt that, is to write directly to the disk without the top of ZFS knowing about it.

Create yourself a 3 disk raidz (just make large empty files and create a pool on it) copy some data to it, then "accidentally" over-write one of those large-empty-files the pool is using. Watch ZFS kick into action.
User avatar
lundman
 
Posts: 1334
Joined: Thu Mar 06, 2014 2:05 pm
Location: Tokyo, Japan

Re: Testing Corrupted data detection by OpenZFS

Postby daifuku74 » Mon Jul 04, 2016 1:08 pm

I see it a lot clearer now, thanks everyone for your input!
daifuku74
 
Posts: 4
Joined: Thu Jun 30, 2016 4:34 am


Return to Absolute Beginners Section

Who is online

Users browsing this forum: No registered users and 4 guests