Optimize RAIDZ expansion

[amotin]

• Instead of copying one ashift-sized block per ZIO, copy as much as we have contiguous data up to 16MB per old vdev. To avoid data moves use gang ABDs, so that read ZIOs can directly fill buffers for write ZIOs. ABDs have much smaller overhead than ZIOs in both memory usage and processing time, plus big I/Os do not depend on I/O aggregation and scheduling to reach decent performance on HDDs.
• Reduce raidz_expand_max_copy_bytes to 16MB on 32bit platforms.
• Use 32bit range tree when possible (practically always now) to slightly reduce memory usage.
• Use ZIO_PRIORITY_REMOVAL for early stages of expansion, same as for main ones.
• Fix rate overflows in zpool status reporting.

Closes #15680

How Has This Been Tested?

With these changes expanding RAIDZ1 from 4 to 5 children I am able to reach 6-12GB/s rate on SSDs and ~500MB/s on HDDs, both are limited by devices instead of CPU.

Types of changes

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Performance enhancement (non-breaking change which improves efficiency)
• Code cleanup (non-breaking change which makes code smaller or more readable)
• Breaking change (fix or feature that would cause existing functionality to change)
• Library ABI change (libzfs, libzfs_core, libnvpair, libuutil and libzfsbootenv)
• Documentation (a change to man pages or other documentation)

Checklist:

• My code follows the OpenZFS code style requirements.
• I have updated the documentation accordingly.
• I have read the contributing document.
• I have added tests to cover my changes.
• I have run the ZFS Test Suite with this change applied.
• All commit messages are properly formatted and contain Signed-off-by.

[snajpa]

Fix rate overflows in zpool status reporting.

so that'll probably fix this one #16803, right? I'm thinking if it wouldn't be better to have that as a separate commit which could be backported to older releases without the rest of the change?

[amotin]

so that'll probably fix this one #16803, right? I'm thinking if it wouldn't be better to have that as a separate commit which could be backported to older releases without the rest of the change?

@snajpa It won't. This should fix improperly reported expansion rate and estimated completion time. It has nothing to do with percentage completed. And I am not sure couple line change of cosmetic issue worth a commit, though might be.

[snajpa]

And I am not sure couple line change of cosmetic issue worth a commit, though might be.

If those are going to get us new open tickets I'd say it might be worth it :D

[behlendorf]

This should however address #15680.

[liyimeng]

Will this be part of 2.3 release? :)