Rule promotion process

[juju4]

fqcn-builtin has been added to default rule set and roles CI are all blinking red.

It seems that has been added more than 6 months ago - #1614
I don't understand how it can be made default directly.

I value a lot linting but when possible, it should have advance notice to avoid last minute burden.
Please have a clear promotion process. something like

• x months/releases as opt-in
• x months/releases as experimental/warning mode

In case above, with 3 months as opt-in and 3 months in warning mode, that gives plenty of time for people to notice and remediate.

Currently, we are moving from CI lint empty to fail like
https://github.com/juju4/ansible-ntpclient/runs/5474680210?check_suite_focus=true (7d ago)
https://github.com/juju4/ansible-ntpclient/runs/5564280422?check_suite_focus=true (today)

When you have few roles to maintain and simple, easy.
When you have dozens and some more complex, less easy.

Also documentation should have a recommended ansible-lint config files that emphasize visibility on those newer rules and how to manage best
https://ansible-lint.readthedocs.io/en/latest/usage/

last, my local install of 5.4.0 has not this opt-in rule and currently not updating to 6.0.0 even if visible on pypi. not sure why

$ ansible-lint -v -T
WARNING  Loading custom .yamllint config file, this extends our internal yamllint config.
# List of tags and rules they cover
command-shell:  # Specific to use of command and shell modules
  - command-instead-of-module
  - command-instead-of-shell
  - deprecated-command-syntax
  - inline-env-var
  - no-changed-when
  - risky-shell-pipe
core:  # Related to internal implementation of the linter
  - internal-error
  - load-failure
  - parser-error
  - syntax-check
deprecations:  # Indicate use of features that are removed from Ansible
  - deprecated-bare-vars
  - deprecated-command-syntax
  - deprecated-local-action
  - deprecated-module
  - no-jinja-when
  - role-name
experimental:  # Newly introduced rules, by default triggering only warnings
  - ignore-errors
  - risky-file-permissions
  - var-naming
formatting:  # Related to code-style
  - no-jinja-nesting
  - no-tabs
  - playbook-extension
  - risky-octal
  - var-spacing
  - yaml
idempotency:  # Possible indication that consequent runs would produce different results
  - git-latest
  - hg-latest
  - no-changed-when
  - package-latest
idiom:  # Anti-pattern detected, likely to cause undesired behavior
  - command-instead-of-module
  - command-instead-of-shell
  - empty-string-compare
  - inline-env-var
  - literal-compare
  - no-handler
  - no-loop-var-prefix
  - no-relative-paths
  - unnamed-task
  - var-naming
metadata:  # Invalid metadata, likely related to galaxy, collections or roles
  - meta-incorrect
  - meta-no-info
  - meta-no-tags
  - meta-video-links
  - role-name
unpredictability:
  - ignore-errors
  - partial-become
  - risky-file-permissions
unskippable:
  - syntax-check
yaml:  # External linter which will also produce its own rule codes.
  - yaml
$  pip install ansible-lint==6.0.0
Defaulting to user installation because normal site-packages is not writeable
ERROR: Could not find a version that satisfies the requirement ansible-lint==6.0.0 (from versions: 1.0.0, 1.0.1, 1.0.2, 1.0.3, 1.0.4, 2.0.1, 2.0.3, 2.1.0, 2.1.3, 2.2.0, 2.3.0, 2.3.1, 2.3.2, 2.3.3, 2.3.5, 2.3.6, 2.3.8, 2.3.9, 2.4.0, 2.4.1, 2.4.2, 2.5.0, 2.6.1, 2.6.2, 2.7.0rc1, 2.7.0, 2.7.1, 3.0.0rc1, 3.0.0rc2, 3.0.0rc3, 3.0.0rc4, 3.0.0rc5, 3.0.0rc6, 3.0.0rc7, 3.0.0rc8, 3.0.0rc9, 3.0.0rc10, 3.0.0rc11, 3.0.0rc12, 3.0.0, 3.0.1, 3.0.2rc1, 3.1.0rc1, 3.1.0, 3.1.1, 3.1.2, 3.1.3, 3.2.0, 3.2.1rc1, 3.2.1rc2, 3.2.1, 3.2.2rc1, 3.2.2, 3.2.3, 3.2.4, 3.2.5, 3.3.0rc1, 3.3.0rc2, 3.3.0rc3, 3.3.0rc4, 3.3.0, 3.3.2, 3.3.3, 3.4.0rc1, 3.4.0rc2, 3.4.0, 3.4.1, 3.4.3, 3.4.4, 3.4.5, 3.4.6, 3.4.7, 3.4.8, 3.4.9, 3.4.10, 3.4.11, 3.4.12, 3.4.13, 3.4.15, 3.4.16, 3.4.17, 3.4.18, 3.4.19, 3.4.20, 3.4.21, 3.4.22, 3.4.23, 3.5.0rc1, 3.5.0, 3.5.1, 4.0.0a1, 4.0.0a2, 4.0.0, 4.0.1, 4.1.0a0, 4.1.0, 4.1.1a0, 4.1.1a1, 4.1.1a2, 4.1.1a3, 4.1.1a4, 4.1.1a5, 4.1.1a6, 4.2.0a1, 4.2.0rc1, 4.2.0rc2, 4.2.0, 4.3.0a0, 4.3.0a1, 4.3.0a2, 4.3.0a3, 4.3.0a4, 4.3.0a5, 4.3.0a6, 4.3.0, 4.3.1, 4.3.2, 4.3.3, 4.3.4, 4.3.5, 4.3.6, 4.3.7, 5.0.0a0, 5.0.0a1, 5.0.0a2.dev12, 5.0.0a3.dev23, 5.0.0a3.dev27, 5.0.0a3, 5.0.0, 5.0.1, 5.0.2, 5.0.3a0, 5.0.3a1, 5.0.3rc1, 5.0.3, 5.0.4, 5.0.5a0, 5.0.5a1, 5.0.5, 5.0.6, 5.0.7, 5.0.8, 5.0.9, 5.0.10, 5.0.11, 5.0.12, 5.1.0a0, 5.1.0a1, 5.1.1, 5.1.2, 5.1.3, 5.2.0, 5.2.1, 5.3.0, 5.3.1, 5.3.2, 5.4.0)
ERROR: No matching distribution found for ansible-lint==6.0.0

maybe because python 3.6.9 (bionic)

[cognifloyd]

As far as I understand, we will only switch the default set of rules in a major version bump.

If you want to avoid changes like this, please pin ansible-lint between major versions, and then test the upgrade before rolling it out in your CI process. You can fix your CI by pinning to ansible-lint<6.0 or you can add the rule you don't want to the either the warn or skip list in your ansible-lint config file.

[juju4]

IMHO, pinning does not solve advanced notice.
A structure documented rule promotion process would, ideally without having to modify each CI at each major or minor release of ansible-lint.

[MarkusTeufelberger]

There also needs to be a --strict or --werror or whatever mode that promotes warnings to errors, otherwise there will be no signal (and in the case of pre-commit: no output at all) for CI tools to react to.

[ssbarnea]

I support adding a --strict option mirrored by a strict: true config option that makes it fail even on warnings, very similar to other tools.

[MarkusTeufelberger]

Great, here's a feature request for that: #1742

[cognifloyd]

As far as your local install issue, yes py3.6 is not supported by ansible-lint 6.0. this is in the release notes:

Remove support for py36 and py37 (#1850)