[add_kubernetes_metadata][Metricset state_namespace] Fix namespace filter

[constanca-m]

Proposed commit message

This PR:

1. Adds namespace filter to add_kubernetes_metadata watchers.
2. At metricset level, it fixes the state_namespace to also consider namespace filter like all the other namespaced resources.

Checklist

• My code follows the style guidelines of this project
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• I have made corresponding change to the default configuration files
• I have added tests that prove my fix is effective or that my feature works
• I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

Related issues

Relates to #38978.

How to test locally

Learn how to run your own filebeat here. In next section, I explain my tests.

Results

I have a cluster with two namespaces: default and kube-system.

This is an example of filestream using the processor filtered by namespace `default`.

    filebeat.inputs:
      - type: filestream
        id: kubernetes-container-logs
        paths:
          - /var/log/containers/*.log
        parsers:
          - container: ~
        prospector:
          scanner:
            fingerprint.enabled: true
            symlinks: true
        file_identity.fingerprint: ~
        processors:
          - add_kubernetes_metadata:
              host: ${NODE_NAME}
              namespace: default
              matchers:
                - logs_path:
                    logs_path: "/var/log/containers/"

The documents fields annotated with data coming from the processor need to be coming from pods inside default namespace. Checking in discover I can see:

And:

So only pods from default get annotated as expected.

If I now remove the namespace from the processor and leave it like this:

          - add_kubernetes_metadata:
              host: ${NODE_NAME}
              matchers:
                - logs_path:
                    logs_path: "/var/log/containers/"

I can see all documents get annotated with metadata:

[mergify]

This pull request does not have a backport label.
If this is a bug or security fix, could you label this PR @constanca-m? 🙏.
For such, you'll need to label your PR with:

• The upcoming major version of the Elastic Stack
• The upcoming minor version of the Elastic Stack (if you're not pushing a breaking change)

To fixup this pull request, you need to add the backport labels for the needed
branches, such as:

• backport-v8./d.0 is the label to automatically backport to the 8./d branch. /d is the digit

[constanca-m]

The lint errors specificMetaGen declared and not used (typecheck) are annoying. I have tried to surpass it with nolint:all, nolint:unused and nolint:typecheck and none worked, so I am leaving them like this for now.

[MichaelKatsoulis]

Thanks for that! Just address the changelog comment.