Support for match_enhancements in the helm chart

[tsboris]

Hello,

Is there support for match_enhancements in the helm chart?

I'm trying to convert/format the @timestamp value in slack's alert body to a nicer date and time format.
Based on the documentation, I need to use Enhancements.
But that only gives me the following error in the pods logs:

Elastic Version: 7.2.0
Reading Elastic 6 index mappings:
Reading index mapping 'es_mappings/6/silence.json'
Reading index mapping 'es_mappings/6/elastalert_status.json'
Reading index mapping 'es_mappings/6/elastalert.json'
Reading index mapping 'es_mappings/6/past_elastalert.json'
Reading index mapping 'es_mappings/6/elastalert_error.json'
Index elastalert already exists. Skipping index creation.
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/elastalert/util.py", line 27, in get_module
    base_module = __import__(module_path, globals(), locals(), [module_class])
ModuleNotFoundError: No module named 'elastalert_modules'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/elastalert/loaders.py", line 126, in load
    rule = self.load_configuration(rule_file, conf, args)
  File "/usr/local/lib/python3.9/site-packages/elastalert/loaders.py", line 189, in load_configuration
    self.load_modules(rule, args)
  File "/usr/local/lib/python3.9/site-packages/elastalert/loaders.py", line 426, in load_modules
    enhancement = get_module(enhancement_name)
  File "/usr/local/lib/python3.9/site-packages/elastalert/util.py", line 30, in get_module
    raise EAException("Could not import module %s: %s" % (module_name, e)).with_traceback(sys.exc_info()[2])
  File "/usr/local/lib/python3.9/site-packages/elastalert/util.py", line 27, in get_module
    base_module = __import__(module_path, globals(), locals(), [module_class])
elastalert.util.EAException: Could not import module elastalert_modules.timestamp_conversion.TimestampConversion: No module named 'elastalert_modules'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/bin/elastalert", line 8, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.9/site-packages/elastalert/elastalert.py", line 2108, in main
    client = ElastAlerter(args)
  File "/usr/local/lib/python3.9/site-packages/elastalert/elastalert.py", line 150, in __init__
    self.rules = self.rules_loader.load(self.conf, self.args)
  File "/usr/local/lib/python3.9/site-packages/elastalert/loaders.py", line 134, in load
    raise EAException('Error loading file %s: %s' % (rule_file, e))
elastalert.util.EAException: Error loading file /opt/rules/test_alert.yaml: Could not import module elastalert_modules.timestamp_conversion.TimestampConversion: No module named 'elastalert_modules'

My helm directory looks like this:

elastalert2
├── Chart.yaml
├── elastalert_modules
│   ├── __init__.py
│   └── timestamp_conversion.py
├── README.md
├── templates
│   ├── config.yaml
│   ├── deployment.yaml
│   ├── _helpers.tpl
│   ├── NOTES.txt
│   ├── podsecuritypolicy.yaml
│   ├── rolebinding.yaml
│   ├── role.yaml
│   ├── rules.yaml
│   └── serviceaccount.yaml
├── values-dev.yaml
├── values-prod.yaml
└── values.yaml

Thanks! :-)

[jertel]

You wouldn't normally modify the helm directory in this situation. Instead, use the volume mount settings:

Then, per the various Kubernetes volume mount methods, you would source the custom enhancement contents from a ConfigMap, Secret, host file, etc. A simple way to accomplish this is by using a ConfigMap. Ex:

apiVersion: v1
kind: ConfigMap
metadata:
  name: elastalert2-enhancements
  namespace: default
data:
  timestamp_conversion.py: |
    from elastalert.enhancements import BaseEnhancement

    class ConvertTime(BaseEnhancement):
    def process(self, match):
        # your custom converter code goes here

Apply that ConfigMap to your ElastAlert2 namespace. Then customize the volumes for the ElastAlert2 chart by adding the following to your value overrides file (looks like you are calling them values-dev.yaml, etc):

extraVolumeMounts:
  - name: enhancements
    mountPath: /opt/elastalert/elastalert_modules
extraVolumes:
  - name: enhancements
    configMap:
      name: elastalert2-enhancements

This is all written from memory so some paths or syntax errors will need corrected, but it should point you in the right direction. The Kubernetes documentation is useful for this type of information. Here's a good doc page to start with: https://kubernetes.io/docs/tasks/configure-pod-container/configure-pod-configmap/

[tsboris]

Thanks!
I'll try that :-)