Get raw logs/documents of an event firing alert when use_terms_query

[eveningcafe]

I'm trying to get raw logs/documents of an event firing alert. I have already read the documentation, which mentions a method to bring documents into alert:

However, when I applied it, the result only showed the value of the query_key firing, not the full document:

(I used the option "use_term_query" set to true)

On the other hand, when I didn't use "use_term_query", it worked as expected:

Is this a limitation of "use_term_query"? Can we create an if condition to re-query Elasticsearch a second time to get the raw log? In my system, Elasticsearch is strong, but ElastAlert's performance is a concern when it can't run in HA (High Availability) mode.

[eveningcafe]

By the way, seeing the raw log in the message is quite frustrating. Can we configure it to hide it, in case I have already sent it as a related_events separate field?

[jertel]

Is this a limitation of "use_term_query"?

Yes, it is, because performing an aggregate query against elasticsearch results in only counts of each aggregation returned.

Can we create an if condition to re-query Elasticsearch a second time to get the raw log?

ElastAlert 2 doesn't support a concept of follow-up queries. However, the Enhancements feature allows developers to add small Python scripts during the alerting phase, which could perform additional queries if coded to do so. A similar request was discussed in #1207.

In my system, Elasticsearch is strong, but ElastAlert's performance is a concern when it can't run in HA (High Availability) mode.

I don't understand this statement. How is ElastAlert's performance connected to its high availability?

By the way, seeing the raw log in the message is quite frustrating. Can we configure it to hide it, in case I have already sent it as a related_events separate field?

It depends on which alerter you are using. Several alerters offer the ability to customize which fields are included in the alert, and/or control the inclusion of the raw message. Some also offer Jinja templating of the alert message itself, which provides a highly customizable option. Supporting information should be available in the documentation. If you cannot find a combination of alerters and options that meet your needs you can use a custom Enhancement to directly modify the event content before it's sent to the alert.

[eveningcafe]

ElastAlert 2 doesn't support a concept of follow-up queries. However, the Enhancements feature allows developers to add small Python scripts during the alerting phase, which could perform additional queries if coded to do so. A similar request was discussed in #1207.

Great, I will try it. Some stubit question here: what in the match[] in the doc Enhancements Can you suggest me some debug mode setup when i code that module? I don't want to blind printf everything.

I don't understand this statement. How is ElastAlert's performance connected to its high availability?

When ElastAlert can run in some type of HA cluster, if one node dies, it will automatically load the rules it currently handles to another node. Therefore, high memory load (even memory leaks) will not be a problem. That's my imagined design

It depends on which alerter you are using. Several alerters offer the ability to customize which fields are included in the alert, and/or control the inclusion of the raw message. Some also offer Jinja templating of the alert message itself, which provides a highly customizable option. Supporting information should be available in the documentation. If you cannot find a combination of alerters and options that meet your needs you can use a custom Enhancement to directly modify the event content before it's sent to the alert.

Nice, but why some offer Jinja template and some don't?

[jertel]

what in the match[] in the doc Enhancements

Match is a dict object containing all the event fields from the matched record. So match['foo'] will contain the value that was stored in the foo key in the document that triggered the alert.

Can you suggest me some debug mode setup when i code that module? I don't want to blind printf everything.

This discussion shows one method for enabling all debug logging. In your enhancement code you can use elastalert_logger.debug("something") to write logs at the debug log level.

When ElastAlert can run in some type of HA cluster, if one node dies, it will automatically load the rules it currently handles to another node. Therefore, high memory load (even memory leaks) will not be a problem. That's my imagined design

#544 and #865 have some discussion and alternatives for HA.

Nice, but why some offer Jinja template and some don't?

That's because this project is dependent upon contribution from the community, and Jinja templates came later in a couple of newer alerters. Anyone is welcome to contribute PRs to expand the Jinja support to all alerters.