Add the ability to add ZFS pool configurations/properties

[simondeziel]

It'd be nice if LXD allowed setting zpool level configurations/properties when creating the storage pool. This was discussed in this thread.

[simondeziel]

ATM, if one asks LXD to create a zpool while running on a recent kernel (say 6.5.0 with zfs 2.2) and later wants to go back to an older kernel (say 5.15 with zfs 2.1), the zpool won't import due to being incompatible with the older zfs module coming with the older kernel.

This prevents one from switching from the Ubuntu HWE kernel back to the GA kernel for example.

[tomponline]

How would one allow this to proceed? Do you have an example of the settings needed?

[simondeziel]

For the case I described, there is no nice solution I guess.

If we grow the ability to set the compatibility level, one could be foreseeing the need to using an older zfs module and thus set things accordingly at pool creation time.

[tomponline]

@simondeziel so if a pool is created with ZFS 2.2 there's no way to downgrade it back to 2.1 compat?

[simondeziel]

@tomponline that is correct. In this example, the pool z650 was created while running on kernel 6.5.0/zfs 2.2. Now that I rebooted back into 5.15.0/zfs 2.1, it cannot be imported:

root@v1:/# uname -a
Linux v1 5.15.0-1053-kvm #58-Ubuntu SMP Tue Mar 12 12:41:48 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
root@v1:/# /snap/lxd/current/zfs-2.2/bin/zpool import -d /var/snap/lxd/common/lxd/disks/z650.img 
   pool: z650
     id: 10172191822658267680
  state: UNAVAIL
status: The pool uses the following feature(s) not supported on this system:
	com.klarasystems:vdev_zaps_v2
action: The pool cannot be imported. Access the pool on a system that supports
	the required feature(s), or recreate the pool from backup.
 config:

	z650                                       UNAVAIL  unsupported feature(s)
	  /var/snap/lxd/common/lxd/disks/z650.img  ONLINE

Only the zpool created while running 5.15.0/zfs 2.1 is usable:

root@v1:/# /snap/lxd/current/zfs-2.2/bin/zpool list -v
NAME                                        SIZE  ALLOC   FREE  CKPOINT  EXPANDSZ   FRAG    CAP  DEDUP    HEALTH  ALTROOT
z515                                        960M   630K   959M        -         -     0%     0%  1.00x    ONLINE  -
  /var/snap/lxd/common/lxd/disks/z515.img   960M   630K   959M        -         -     0%  0.06%      -    ONLINE

[simondeziel]

@tomponline One way to improve things would be to take into consideration if we are running on a Ubuntu LTS where HWE kernels are a thing. If the snap detects an Ubuntu LTS env, it could set the compat level suitable for the GA kernel instead of the currently running one.

This isn't super nice as this feels like added maintenance and is Ubuntu (and derivative) specific.