Daily Shaarli

All links of one day in a single page.

August 17, 2021

ZFS RAIDZ expansion is awesome but has a small caveat

One of my most popular blog articles is this article about the "Hidden Cost of using ZFS for your home NAS". To summarise the key argument of this article:

Expanding ZFS-based storge can be relatively expensive / inefficient.

For example, if you run a ZFS pool based on a single 3-disk RAIDZ vdev (RAID5 equivalent2), the only way to expand a pool is to add another 3-disk RAIDZ vdev1.

You can't just add a single disk to the existing 3-disk RAIDZ vdev to create a 4-disk RAIDZ vdev because vdevs can't be expanded.

The impact of this limitation is that you have to buy all storage upfront even if you don't need the space for years to come.

Otherwise, by expanding with additional vdevs you lose capacity to parity you may not really want/need, which also limits the maximum usable capacity of your NAS.
RAIDZ vdev expansion

Fortunately, this limitation of ZFS is being addressed!

ZFS founder Matthew Ahrens created a pull request around June 11, 2021 detailing a new ZFS feature that would allow for RAIDZ vdev expansion.

Finally, ZFS users will be able to expand their storage by adding just one single drive at a time. This feature will make it possible to expand storage as-you-go, which is especially of interest to budget conscious home users3.

Jim Salter has written a good article about this on Ars Technica.
https://arstechnica.com/gadgets/2021/06/raidz-expansion-code-lands-in-openzfs-master/

There is still a caveat

Existing data will be redistributed or rebalanced over all drives, including the freshly added drive. However, the data that was already stored on the vdev will not be restriped after the vdev is expanded. This means that this data is stored with the older, less efficient parity-to-data ratio.

I think Matthew Ahrends explains it best in his own words:

After the expansion completes, old blocks remain with their old data-to-parity ratio (e.g. 5-wide RAIDZ2, has 3 data to 2 parity), but distributed among the larger set of disks. New blocks will be written with the new data-to-parity ratio (e.g. a 5-wide RAIDZ2 which has been expanded once to 6-wide, has 4 data to 2 parity). However, the RAIDZ vdev's "assumed parity ratio" does not change, so slightly less space than is expected may be reported for newly-written blocks, according to zfs list, df, ls -s, and similar tools.

So, if you add a new drive to a RAIDZ vdev, you'll notice that after expansion, you will have less capacity available than you would theoretically expect. //

The overhead or 'lost capacity' can be recovered by rewriting existing data after the vdev has been expanded, because the data will then be written with the more efficient parity-to-data ratio of the larger vdev.

Rewriting all data may take quite some time and you may opt to postpone this step until the vdev has been expanded a couple of times so the parity-to-data ratio is now 'good enough' that significant storage gains can be had by rewriting the data. //

Release timeline

According to the Ars Technica article by Jim Salter, this feature will probably become available in August 2022, so we need to have some patience.

ZFS Capacity Calculator - WintelGuy.com

RAID type - Supported RAID levels are:

  • Mirror (two-way mirror - RAID1 / RAID10 equivalent);
  • RAID-Z1 (single parity with variable stripe width);
  • RAID-Z2 (double parity with variable stripe width);
  • RAID-Z3 (triple parity with variable stripe width).

Drive capacity - we expect this number to be in gigabytes (powers of 10), in-line with the way disk capacity is marked by the manufacturers. This number will be converted to tebibytes (powers of 2). The results will be presented in both tebibytes (TiB) and terabytes (TB). Note: 1 TB = 1000 GB = 1000000000000 B and 1 TiB = 1024 GiB = 1099511627776 B

Single drive cost - monetary cost/price of a single drive; used to calculate the Total cost and the Cost per TiB. The parameter is optional and has no impact on capacity calculations.

Number of RAID groups - the number of top-level vdevs in the pool.

Number of drives per RAID group - the number of drives per vdev.

Slop space allocation - 1/32 of the capacity of the pool or at least 128 MiB, but never more than half the pool size. [1,2]

% free space limit - recommended free pool space required to ensure best performance. Usually around 20%. [4]

Outputs:

Total raw storage capacity - the sum of physical size of all drives in the pool.

Zpool storage capacity - calculated as the difference between the total raw storage capacity and the loss for drive partitioning and metaslab allocation, but without taking into account parity and padding. This number should be reasonably close to the SIZE value reported by the zpool list command.

Reservation for parity and padding - calculated as described by Matthew Ahrens in [3]. (Not applicable to Mirror vdevs.)

Zpool usable storage capacity - calculated as the difference between the zpool storage capacity and the reservation for parity and padding.

Slop space allocation - see the Inputs section above for description.

ZFS usable storage capacity - calculated as the difference between the zpool usable storage capacity and the slop space allocation value. This number should be reasonably close to the sum of the USED and AVAIL values reported by the zfs list command.

Can PCIe X4 Card Fit in X16 Slot? - Can it Work? - PC Guide 101

Simply put, a PCIe x4 card CAN fit in a PCIe x16 slot. The main reason this is possible lies within the fact that PCIe standard is cross-compatible as well as backward compatible.

Meaning, PCIe devices can work in different slots as long as they are provided the required PCIe lanes. They can also work across different generation provided that they get enough bandwidth.

So Can PCIe x4 Card Fit in x16 slot? Yes, it can. However, whether this is a good idea or not is something we explore further in this article.

The key point to note here is that the PCIe device and slot work at the speeds of the slowest component. So putting the x4 device in an x16 slot will not result in any performance boost.

Mind-Boggling Pic of People Who Made It on to the C-17 Out of Kabul – RedState

But what did it look like on the inside of one of the C-17s taking off? Normally, it might take 150 fully geared-up members of the U.S. military. But today it may have set a record, according to Defense One. Many more vetted panicked Afghans piled onto the plane than the military had thought they would be taking, but they didn’t want to turn anyone away, given what was going on, so they just took as many as they could.

Blue Origin sues NASA for awarding SpaceX $3bn contract to land next American boots on the Moon • The Register

In Jeff's world, it's not 'you snooze, you lose', it's 'you lose, you sues'

Perl vs Python: Compared and Contrasted | Career Karma

Perl is a high-level programming language that’s easier to learn when compared with Python. Python is more robust, scalable, and stable when compared to Perl. While Perl code can be messy, featuring many paths to accomplish the same goal, Python is clean and streamlined. //

Ask someone who’s worked with both Perl and Python what the most noticeable difference between the two is, and they will most likely reply “readability.” Perl, although designed for beginners to master, is consistently reported to look unreadable, especially compared to the sleek interface brought by Python. Editing code can be difficult enough. Having to decipher the code you’re trying to edit at the same time? That may be pushing it.