[gruturo]

Uh? Let's say you have a simple mirror of 2 vdevs, you go like this:

zpool attach <poolname> <first existing small vdev> <first larger new vdev>

zpool attach <poolname> <second existing small vdev> <second larger new vdev>

[...wait for the resilvering of the new vdevs...] You now have a 4-way mirror with 2 small and 2 large vdevs. Detach the small ones:

zpool detach <poolname> <first old vdev>

zpool detach <poolname> <second old vdev>

Now you have a pool made only of large vdevs. You just give the pool the permission to expand and occupy all this new space:

zpool set autoexpand=on <poolname>

Done. Did it oodles of times on Solaris with SAN storage, but did it at home too, and in the weirdest ways (no SATA ports available? No problem, attach the new drive via USB3, then when finished take it out of the enclosure and install it inside displacing the old drive), sometimes even rather unsafely (creating a pre-broken raidz with 4 good disks plus a sparse file, migrating data off a different pool, then decommissioning that old pool and using one of its disk to replace the sparse file).

[detaro]

The commonly wished-for feature is not to entirely replace disks as you describe, but to expand it bit by bit. To be able to have e.g. a z2 setup spanning 4 disks of equal size and then add a 5th one later to get one disk's worth of extra capacity.

[gruturo]

If you mean turning a 4-disk raidz2 into a 5-disk raidz2, indeed that's unfortunately not yet possible (unless you had built it with an extra vdev sparse file and run it in this degraded state - effectively you had a raidz and not a z2, and it only works once).

If you want, though, you can add capacity by attaching a second raidz2 to an existing pool - this forces you to add 4 new drives instead of one, but it works:

zpool add <poolname> raidz2 <first new vdev> <second new vdev> <third new vdev> <fourth new vdev>

You now have a concat of 2 raidz2's, in a single pool.

Nowhere as elegant as merging a new slice in an existing z2, I concur. Does the job though. Oh and if you are insane you could probably even add a single device, resulting in a concat of a z2 + unmirrored single device. I don't think it will stop you from doing that.

[tjoff]

Does the job though.

Nope.

It will result in very uneven utilization which leads to poor performance. Not recommended. Is also extremely wasteful. The whole point of the sought feature is to minimize waste and cost. So no, I'd strongly disagree that it does the job.

[gruturo]

I'm not trying to defend ZFS' lack of the ability to add a vdev to an existing RAIDZ/Z2. That would be extremely nice to have and has tons of legitimate uses, for many of which there is no acceptable substitute as you point out.

That said, even if that function existed, expecting it to magically rebalance the whole pool to take into account the existing data is rather unrealistic (it's basically a full rebuild in that case, the most terrifying operation you can perform on any pool/array already containing data).

While I agree with the performance aspects (your data still comes from either the first volume or the second, so you're only using the performance of half your spindles - unless you simultaneously access files in either half), I wouldn't call it wasteful: You are still losing the same % of your total capacity. e.g.:

6x2TB RaidZ2 = 8TB usable (4TB lost to parity)

let's add a second Z2 6x6TB volume:

6x6TB RaidZ2 = 24TB usable (12 TB lost to parity)

Total: 32 TB usable, 16 lost to parity.

What if we had built it from scratch using 6x8TB drives?

Total: 32 TB usable, 16 lost to parity.

Same.

[tjoff]

That said, even if that function existed, expecting it to magically rebalance the whole pool to take into account the existing data is rather unrealistic

That is exactly what is required (well, you re-balance the vdev, not the pool) and is what is proposed and has been in the works for basically forever. SUN never prioritized this because it has few uses within the enterprise (but extremely valuable for home users).

Your example is quite misleading. Here's is a much more common and realistic scenario.

6x2TB RaidZ2 = 8 usable, 4 lost to parity.

Now, imagine that you want to expand the pool with 4 TB. Being able to add to a raidz vdev would result in this:

8x2TB RaidZ2 = 12 Usable, 4 lost to parity. (ridiculously cheap, no further waste at all)

Current scenario:

6x2TB RaidZ2 + 4x2TB RaidZ2 = 12 Usable, 8 lost to parity. (expensive, lower performance, more power, more noise, requires more harddrive slots (this is often a very important aspect for home-setups))

The waste is outrageous. And the consequences from this waste reflects every aspect of designing a home-setup (as to avoid the above scenario) with ZFS and is vastly different to how you would design a system with, for instance, raid6 with expansion in mind.

[tscs37]

The comparison is a bit off.

If you build a RAIDZ2 wiht a 3x8TB and a 3x8TB pool, you loose 2+2x8TB = 32TB and have only 16TB usuable.

6x2+6x6 is a mixed pool so it's hard to compare to actual disks.

An alternative, better, comparison, would be a 6x8TB pool and another 6x8TB pool.

This time you loose 32TB to parity and have 8x8=48TB usuable. If you rebuild it to a 12x8TB pool, you loose 16TB to parity and have 80TB usuable.

[Annatar]

Expanding bit by bit is extremely fragile, arcanely complex and error prone. Not worth the amount of financial savings, especially when the cost of time and effort is added on top of that.

[nske]

It might be what you say, but in many cases it might also be a necessity. I built my first array with 10x1TB disks (starting with 3) for my home server when I was a student/got my first job and could only afford one disk every now and then. Linux' expandable mdraid worked like a charm multiple times. I probably wouldn't have bothered (and would have lost data or limited my hoarding) if I had to save for a year just for a fileserver. I am not talking about precious data but data don't have to be irreplaceable to want to protect them.