Handling log delay with ElastAlert to prevent alerts until log arrival

[CaGix22]

Hi everyone! I'm encountering an issue with my flatline rule setup and would appreciate any guidance or suggestions you might have.

Is there a way to configure EA to account for potential delays in log arrival?

This is the scenario:
My app generates a "Success" log at specific times every hour. For example, the log timestamp is set to 11:50:23.222
However, due to Logstash processing delays, logs sometimes arrive "late" to ElasticSearch. For instance, a log created at 11:50:23.222 might not arrive until 12:10:45, reflected by its @logstash_timestamp.

In my global config I have set run_every: hours: 1 and my rule is a type: flatline If ElastAlert executes the query, for example, from 11:00 to 12:00 and the log has not been processed by Logstash in time, it results in a match, thus triggering an alert.

The problem is, that we consider that alert to be a "false alert" because the logs did arrived, late but arrived. Is there a way to configure the rule to account for potential delays in log arrival and to prevent alerts until a certain waiting period has passed??? Is for this case a flatline rule suitable?

Thanks for your help!!

[jertel]

query_delay might help with this situation: https://elastalert2.readthedocs.io/en/latest/ruletypes.html#query-delay

[CaGix22]

Thanks Jertel! I checked and I was using query_delay combined with limit_execution and the result wasn't the expected hehe.
query_delay alone works fine!