Recover file with bad checksum

Moderators: jhartley, MSR734, nola

Recover file with bad checksum

Post by si-ghan-bi » Fri Nov 16, 2012 3:20 am

This is a theoretical question, I don't (yet) need the answer, but I'm curious.
At my parents' house I will be converting an old Athlon 64 X2 3800+ 4 GB ECC as nas4free box to serve some files to the other computers, to store backups and to store my remote backups too (cheaper and bigger than an online service).
Since I want to keep costs down (data stored are almost exclusively backups), I won't be using mirroring or RAID, only a plain single 3 TB disk.

In case one file gets corrupted but I still need to have something, how should I proceed? I expect ZFS to usually throw an error and completely abort the read request.
si-ghan-bi Offline


 
Posts: 145
Joined: Sat Sep 15, 2012 5:55 am

thoughts

Post by grahamperrin » Fri Nov 16, 2012 4:20 am

si-ghan-bi wrote:This is a theoretical question … to store backups and to store my remote backups … keep costs down (data stored are almost exclusively backups) … only a plain single …


    copies=2
+
    compression=gzip-9

For some types of file, that level of compression should compensate for the loss of free space that we associate with duplication.

If you're unlucky, both copies of a file may be written to an area of hard disk that's not good, in which case scrub may not repair without error. I guess that respecting the eighty percent threshold will reduce the risk of both copies being written in that way.

It's better, of course, to reduce risks with an additional physical disk (instead of the copies property at the file system level) but that will make costs a little higher, and your question specifies no more than one disk.

copies=1 | 2 | 3 (ZFS dataset property)
grahamperrin Offline

User avatar
 
Posts: 1596
Joined: Fri Sep 14, 2012 10:21 pm
Location: Brighton and Hove, United Kingdom

Re: Recover file with bad checksum

Post by si-ghan-bi » Fri Nov 16, 2012 9:15 am

Sorry, I wasn't clear, I already know how to secure data on a single disk. My question is about the best way to retrieve a damaged file if the drive has some bad sectors (checksum failed) but copies=1 and no other good sectors exists. I don't care (in this very specific case) about integrity anymore, I just want to retrieve as many bytes as possible from the file/sectors.
An option is to use "dd" to copy each sector of the file while replacing the unretrievable sectors with zeroes, this is easy.
Is it also possible to retrieve damaged sectors ignoring their checksum? Some bytes may still be good. This may make sense in installations using big sectors (striped data, for example) or store data more or less error-tolerant (images, music).
si-ghan-bi Offline


 
Posts: 145
Joined: Sat Sep 15, 2012 5:55 am

Re: Recover file with bad checksum

Post by grahamperrin » Mon Nov 19, 2012 11:44 am

If it helps: I sometimes find that ditto succeeds where other commands/apps fail. But it's a long time since I used ditto in that situation. From the man page:

> ditto almost never gives up, preferring to report errors along the way
grahamperrin Offline

User avatar
 
Posts: 1596
Joined: Fri Sep 14, 2012 10:21 pm
Location: Brighton and Hove, United Kingdom


Return to General Discussion

Who is online

Users browsing this forum: ilovezfs and 3 guests

cron