"Persistent datasets" term is confusing. Consider changing it into "Unmanaged datasets"

[lukasz-zaroda]

Describe the bug
Persistency suggest some kind of a feature, but it seems that it's rather lack of features. Datasets outside of the namespace are simply not managed by Zsys.

To Reproduce
Steps to reproduce the behavior:

1. Read https://didrocks.fr/2020/06/16/zfs-focus-on-ubuntu-20.04-lts-zsys-dataset-layout/
2. By confused by description of persistent datasets.

Expected behavior
User should understand what persistent datasets are without paragraphs of explanation. Term "Unmanaged datasets" is mostly self explanatory, or at least provides the better starting point in understanding them as featureless datasets, and not ones with some persistent feature added by Zsys.

[almereyda]

Yes, I agree very much with the notion that zsys cannot make any assumptions about non-managed datasets.

Similarily to what is expressed in

• backup - How does one exclude a zpool from zfs-auto-snapshot - Ask Ubuntu

for zfs-auto-snapshot --default-exclude, zsys should only consider to manage datasets that have a special metadata entry set, and not only infer its role by the namespace of a dataset, and eventual snapshot names. This anti-pattern could be read in similarity to how zfsnap/zfsnap: A portable, performant script to make rolling ZFS snapshots easy. (https://github.com/zfsnap/zfsnap/issues/109) worked:

zfsnap stores all the information it needs about a snapshot directly in its name; no database or special ZFS properties are needed. The information is stored in a way that is human readable, making it much easier for a sysadmin to manage and audit backup schedules.

Snapshot names are in the format of pool/fs@[prefix]Timestamp--TimeToLive (e.g. pool/fs@weekly-2014-04-07_05.30.00--6m). The prefix is optional but can be quite useful for filtering, Timestamp is the date and time when the snapshot was created, and TimeToLive (TTL) is the amount of time the snapshot will be kept until it can be deleted.

Coming to a shared understanding within the zsys maintainers and the affected, distinct cohort of its users could help mitigating other issues, too:

• Unmanaged datasets destroyed at boot time #196
• ZSys alters unmanaged datasets #203

Or we consider this an upstream issue at Docker's, and ask them to implement (an optional) -R (e.g. when -f is being used) for their removal strategy in the ZFS Docker driver, as suggested by the error output:

ERROR: for jellyfin_web_1  container 2c171a8b59bc26712cc7c8a226119811ce2c45aa4a7ee049548f247af6e6dae2: driver "zfs" failed to remove root filesystem: exit status 1: "/usr/sbin/zfs fs destroy -r rpool/ROOT/ubuntu_vno50r/var/lib/docker/34f57f25cd7a91ddade2dac9454eea665c23408422f2059eb33cde86971082ac" => cannot destroy 'rpool/ROOT/ubuntu_vno50r/var/lib/docker/34f57f25cd7a91ddade2dac9454eea665c23408422f2059eb33cde86971082ac': filesystem has dependent clones
use '-R' to destroy the following datasets:
rpool/ROOT/ubuntu_vno50r/var/lib/docker/34f57f25cd7a91ddade2dac9454eea665c23408422f2059eb33cde86971082ac-init@autozsys_jf6uy5
rpool/ROOT/ubuntu_vno50r/var/lib/docker/34f57f25cd7a91ddade2dac9454eea665c23408422f2059eb33cde86971082ac-init@autozsys_m6aasw
rpool/ROOT/ubuntu_vno50r/var/lib/docker/34f57f25cd7a91ddade2dac9454eea665c23408422f2059eb33cde86971082ac-init@autozsys_ky9eok
rpool/ROOT/ubuntu_vno50r/var/lib/docker/34f57f25cd7a91ddade2dac9454eea665c23408422f2059eb33cde86971082ac-init@autozsys_2slvbw
rpool/ROOT/ubuntu_vno50r/var/lib/docker/34f57f25cd7a91ddade2dac9454eea665c23408422f2059eb33cde86971082ac-init@autozsys_rlrjqk
rpool/ROOT/ubuntu_vno50r/var/lib/docker/34f57f25cd7a91ddade2dac9454eea665c23408422f2059eb33cde86971082ac-init@autozsys_01upqm
rpool/ROOT/ubuntu_vno50r/var/lib/docker/34f57f25cd7a91ddade2dac9454eea665c23408422f2059eb33cde86971082ac-init@autozsys_3jrfz9
rpool/ROOT/ubuntu_vno50r/var/lib/docker/34f57f25cd7a91ddade2dac9454eea665c23408422f2059eb33cde86971082ac-init@autozsys_cjkajc
rpool/ROOT/ubuntu_vno50r/var/lib/docker/34f57f25cd7a91ddade2dac9454eea665c23408422f2059eb33cde86971082ac-init@autozsys_d0et0x
rpool/ROOT/ubuntu_vno50r/var/lib/docker/34f57f25cd7a91ddade2dac9454eea665c23408422f2059eb33cde86971082ac-init@autozsys_bvsi3n
rpool/ROOT/ubuntu_vno50r/var/lib/docker/34f57f25cd7a91ddade2dac9454eea665c23408422f2059eb33cde86971082ac-init@autozsys_fj8yvz
rpool/ROOT/ubuntu_vno50r/var/lib/docker/34f57f25cd7a91ddade2dac9454eea665c23408422f2059eb33cde86971082ac-init


ERROR: for web  container 2c171a8b59bc26712cc7c8a226119811ce2c45aa4a7ee049548f247af6e6dae2: driver "zfs" failed to remove root filesystem: exit status 1: "/usr/sbin/zfs fs destroy -r rpool/ROOT/ubuntu_vno50r/var/lib/docker/34f57f25cd7a91ddade2dac9454eea665c23408422f2059eb33cde86971082ac" => cannot destroy 'rpool/ROOT/ubuntu_vno50r/var/lib/docker/34f57f25cd7a91ddade2dac9454eea665c23408422f2059eb33cde86971082ac': filesystem has dependent clones
use '-R' to destroy the following datasets:
rpool/ROOT/ubuntu_vno50r/var/lib/docker/34f57f25cd7a91ddade2dac9454eea665c23408422f2059eb33cde86971082ac-init@autozsys_jf6uy5
rpool/ROOT/ubuntu_vno50r/var/lib/docker/34f57f25cd7a91ddade2dac9454eea665c23408422f2059eb33cde86971082ac-init@autozsys_m6aasw
rpool/ROOT/ubuntu_vno50r/var/lib/docker/34f57f25cd7a91ddade2dac9454eea665c23408422f2059eb33cde86971082ac-init@autozsys_ky9eok
rpool/ROOT/ubuntu_vno50r/var/lib/docker/34f57f25cd7a91ddade2dac9454eea665c23408422f2059eb33cde86971082ac-init@autozsys_2slvbw
rpool/ROOT/ubuntu_vno50r/var/lib/docker/34f57f25cd7a91ddade2dac9454eea665c23408422f2059eb33cde86971082ac-init@autozsys_rlrjqk
rpool/ROOT/ubuntu_vno50r/var/lib/docker/34f57f25cd7a91ddade2dac9454eea665c23408422f2059eb33cde86971082ac-init@autozsys_01upqm
rpool/ROOT/ubuntu_vno50r/var/lib/docker/34f57f25cd7a91ddade2dac9454eea665c23408422f2059eb33cde86971082ac-init@autozsys_3jrfz9
rpool/ROOT/ubuntu_vno50r/var/lib/docker/34f57f25cd7a91ddade2dac9454eea665c23408422f2059eb33cde86971082ac-init@autozsys_cjkajc
rpool/ROOT/ubuntu_vno50r/var/lib/docker/34f57f25cd7a91ddade2dac9454eea665c23408422f2059eb33cde86971082ac-init@autozsys_d0et0x
rpool/ROOT/ubuntu_vno50r/var/lib/docker/34f57f25cd7a91ddade2dac9454eea665c23408422f2059eb33cde86971082ac-init@autozsys_bvsi3n
rpool/ROOT/ubuntu_vno50r/var/lib/docker/34f57f25cd7a91ddade2dac9454eea665c23408422f2059eb33cde86971082ac-init@autozsys_fj8yvz
rpool/ROOT/ubuntu_vno50r/var/lib/docker/34f57f25cd7a91ddade2dac9454eea665c23408422f2059eb33cde86971082ac-init

and also

root@acacia:/srv/jellyfin# docker ps -a
CONTAINER ID   IMAGE               COMMAND                CREATED          STATUS                PORTS                                                                                                                                                                        NAMES
7a381f4e2822   jellyfin/jellyfin   "/jellyfin/jellyfin"   20 minutes ago   Up 19 minutes         0.0.0.0:1900->1900/udp, :::1900->1900/udp, 0.0.0.0:8096->8096/tcp, :::8096->8096/tcp, 0.0.0.0:7359->7359/udp, :::7359->7359/udp, 0.0.0.0:8920->8920/tcp, :::8920->8920/tcp   jellyfin_web_1
2c171a8b59bc   360585541fa2        "/jellyfin/jellyfin"   4 months ago     Removal In Progress                                                                                                                                                                                2c171a8b59bc_jellyfin_web_1
root@acacia:/srv/jellyfin# docker rm -f 2c171a8b59bc
Error response from daemon: container 2c171a8b59bc26712cc7c8a226119811ce2c45aa4a7ee049548f247af6e6dae2: driver "zfs" failed to remove root filesystem: exit status 1: "/usr/sbin/zfs fs destroy -r rpool/ROOT/ubuntu_vno50r/var/lib/docker/34f57f25cd7a91ddade2dac9454eea665c23408422f2059eb33cde86971082ac" => cannot destroy 'rpool/ROOT/ubuntu_vno50r/var/lib/docker/34f57f25cd7a91ddade2dac9454eea665c23408422f2059eb33cde86971082ac': filesystem has dependent clones
use '-R' to destroy the following datasets:
rpool/ROOT/ubuntu_vno50r/var/lib/docker/34f57f25cd7a91ddade2dac9454eea665c23408422f2059eb33cde86971082ac-init@autozsys_jf6uy5
rpool/ROOT/ubuntu_vno50r/var/lib/docker/34f57f25cd7a91ddade2dac9454eea665c23408422f2059eb33cde86971082ac-init@autozsys_m6aasw
rpool/ROOT/ubuntu_vno50r/var/lib/docker/34f57f25cd7a91ddade2dac9454eea665c23408422f2059eb33cde86971082ac-init@autozsys_ky9eok
rpool/ROOT/ubuntu_vno50r/var/lib/docker/34f57f25cd7a91ddade2dac9454eea665c23408422f2059eb33cde86971082ac-init@autozsys_2slvbw
rpool/ROOT/ubuntu_vno50r/var/lib/docker/34f57f25cd7a91ddade2dac9454eea665c23408422f2059eb33cde86971082ac-init@autozsys_rlrjqk
rpool/ROOT/ubuntu_vno50r/var/lib/docker/34f57f25cd7a91ddade2dac9454eea665c23408422f2059eb33cde86971082ac-init@autozsys_01upqm
rpool/ROOT/ubuntu_vno50r/var/lib/docker/34f57f25cd7a91ddade2dac9454eea665c23408422f2059eb33cde86971082ac-init@autozsys_3jrfz9
rpool/ROOT/ubuntu_vno50r/var/lib/docker/34f57f25cd7a91ddade2dac9454eea665c23408422f2059eb33cde86971082ac-init@autozsys_cjkajc
rpool/ROOT/ubuntu_vno50r/var/lib/docker/34f57f25cd7a91ddade2dac9454eea665c23408422f2059eb33cde86971082ac-init@autozsys_d0et0x
rpool/ROOT/ubuntu_vno50r/var/lib/docker/34f57f25cd7a91ddade2dac9454eea665c23408422f2059eb33cde86971082ac-init@autozsys_bvsi3n
rpool/ROOT/ubuntu_vno50r/var/lib/docker/34f57f25cd7a91ddade2dac9454eea665c23408422f2059eb33cde86971082ac-init@autozsys_fj8yvz
rpool/ROOT/ubuntu_vno50r/var/lib/docker/34f57f25cd7a91ddade2dac9454eea665c23408422f2059eb33cde86971082ac-init

Please note that the removal in progress* mesage explicitly hints us at what is described in moby/moby#40132 and moby/moby#41055 (comment)

• Permanent "Removal In Progress" for old containers, using zfs storage driver moby/moby#40132
• docker ZFS driver creates hundreds of datasets and doesn’t clean them moby/moby#41055