[Filebeat] cel - support AWS request signing

[andrewkroh]

Describe the enhancement:

Make it possible for CEL to issue API requests to AWS that are signed using credentials that are read from the host environment. It should not be required to provide a static aws access key / secret key in the config.

Describe a specific use case for the enhancement or feature:

To be able to sign API requests like

https://github.com/elastic/integrations/blob/0c425904e85e8acc92a9c6132ddfa378927c90b7/packages/aws/data_stream/securityhub_insights/agent/stream/httpjson.yml.hbs#L28

when deploying integrations inside of AWS where service account credentials are available on the host. For example, when deploying Agent in EKS we use EKS Pod Identity association which manages a token inside of the container that can be use to authenticate requests.

[elasticmachine]

Pinging @elastic/security-service-integrations (Team:Security-Service Integrations)

[efd6]

This should be possible in the input by populating a global with the result of calling os.Environ either unmodified as the []string or, probably better, as a map[string]string so that presence can be assessed directly with a has call rather than by a map over the array.

Do we want to allow list the set of vars that the eval can see? I think probably yes.

[andrewkroh]

It think it is more complicated than using env vars.

In the case of the EKS pod identity association there are several env vars injected into the container. Using the env vars the AWS SDK finds the auth token on disk and it makes a request into AWS STS to AssumeRole, and finally uses the returned access key / secret key to make signed requests to services like SQS/S3. The SDK will ensure that the access key / secret key are refreshed when they get close to expiry.

All that is to say, I think in some way we would want to defer to the AWS SDK to handle finding the creds and getting the access key for us.

Refs

• https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_temp_use-resources.html (goes over another case which is credentials being pulled from EC2 metadata service)

[efd6]

With the addition of env var support, the remaining parts are already available in the input since we have file reading support and can make HTTP requests. We also have all the cryptographic functions needed to do the signing. It may be easier to use the SDK, though maybe not.