RAID-Z and 4K sector size (Advanced Format)

Moderators: jhartley, MSR734, nola

RAID-Z and 4K sector size (Advanced Format)

Post by grahamperrin » Sat Oct 06, 2012 9:53 am

For reference only (I see relevant issues discussed elsewhere, beyond the GreenBytes area)

----

ZFS Day (2012-10-02): Y4K? – George Wilson, Delphix.

At 16:12 on the timeline, under the heading Drawbacks of 4K and ZFS, RAID-Z and 4K is "not recommended". Then, spoken:

… a hard choice to make. RAID-Z and 4K has really become a problem, and I'm not sure that I would recommend it – at least today, but I think, as a community we need to go off and invest and understand – "What does it mean to be using 4K sectors in a RAID-Z world?". It probably works fine as long as you're doing large records, so if you're doing 128K you might be good. If you're doing 8K you're probably throwing away a lot of space …


– there's more to it, I recommend taking time to watch and listen to the whole of that half-hour session (and others from the day).

----

Highly inefficient use of space observed when using raidz2 with ashift=12 · Issue #548 · zfsonlinux/zfs – referred from a post by Reinis Rozitis under RaidZ and 4k sectors in zfs-discuss on Google Groups.

---

In Stack Exchange there's an answered question, software raid - Mixing 4k/512 drives with ZFS (FreeNAS) - Unix and Linux

Elsewhere there's a recommendation to set ashift=12 at time of creation for any pool that might in the future include a drive with those characteristics.
grahamperrin Offline

User avatar
 
Posts: 1596
Joined: Fri Sep 14, 2012 10:21 pm
Location: Brighton and Hove, United Kingdom

Re: RAID-Z and 4K sector size (Advanced Format)

Post by si-ghan-bi » Sun Oct 07, 2012 1:04 pm

I checked the streams but I was unable to find the one about 4K sectors on ustream.
Anyway, who was the smarty that filmed the speaker instead of filming the slides????
si-ghan-bi Offline


 
Posts: 145
Joined: Sat Sep 15, 2012 5:55 am

Re: RAID-Z and 4K sector size (Advanced Format)

Post by grahamperrin » Sun Oct 07, 2012 4:25 pm

George Wilson on stage. Around 16:12 on the timeline the camera pans away from the slides, to the speaker.

around 16-12 on the timeline.png
around 16-12 on the timeline.png (175.73 KiB) Viewed 149 times


Whilst USTREAM might not represent all text chat, I recall a reassurance that Illumos/ZFS day pages will be updated, in due course, to include links to speakers' presentations etc.. Sure enough, some of those things are already added. Thanks to Deirdré Straughan.
grahamperrin Offline

User avatar
 
Posts: 1596
Joined: Fri Sep 14, 2012 10:21 pm
Location: Brighton and Hove, United Kingdom

Re: RAID-Z and 4K sector size (Advanced Format)

Post by NakkiNyan » Sun Oct 07, 2012 6:55 pm

His example was more for small sections of data like ones used in databases, if you have large files where sections are static, large photos, audio or video, it does not really fly.
NakkiNyan Offline


 
Posts: 47
Joined: Tue Oct 02, 2012 12:19 pm

Re: RAID-Z and 4K sector size (Advanced Format)

Post by grahamperrin » Mon Oct 08, 2012 3:32 am

Understood. Would you agree with the 8K and 128K observations in the opening post?
grahamperrin Offline

User avatar
 
Posts: 1596
Joined: Fri Sep 14, 2012 10:21 pm
Location: Brighton and Hove, United Kingdom

Re: RAID-Z and 4K sector size (Advanced Format)

Post by NakkiNyan » Mon Oct 08, 2012 8:40 am

Sure, with some differences to the wording.
It probably works fine as long as you're doing large records, so if you're writing 128K you might be good. If you're writing 8K you're probably throwing away a lot of space...

I say writing because if you have a 4GB database you may only change a few bytes at a time. If it could waste the rest of a 4k space compared to a small wasting the remainder of 512. The same database coud be opened faster though for querying on 4k with a downside of the individual record opening could be slower. Sidenote: Photo, audio, and video editing is dealt with in memory and cache so essentially it makes a copy and writes the whole thing over so editing those files does not apply to the example.

I say this because my "sections are static" comment was too vague.
NakkiNyan Offline


 
Posts: 47
Joined: Tue Oct 02, 2012 12:19 pm

Cross reference

Post by grahamperrin » Wed Mar 27, 2013 1:40 am

Broader discussion: ashift property values
grahamperrin Offline

User avatar
 
Posts: 1596
Joined: Fri Sep 14, 2012 10:21 pm
Location: Brighton and Hove, United Kingdom


Return to General Discussion

Who is online

Users browsing this forum: No registered users and 1 guest

cron