getting multiple match for frequency rule in a specific time period

[m-asri-w]

I have a config like this

type: frequency

index: to_monitor

run_every:
  minutes: 1

buffer_time:
  minutes: 2

num_events: 3

timeframe:

  minutes: 1

filter: []

realert:
  minutes: 0

if have 3 documents in the past 1 min i get 1 match and one alert BUT, if i have 6 documents in past 1 min then i would get 2 matches and two alerts. in general is seems the number of match and alerts is equal to something like floor(number_of_docs / num_events). is it the expected behaviour or am i doing something wrong? cause based on definition of the frequency rule in documents it seems to i should get one alert if the number of documents stored in my index is more than 3 in the past 1 minute, regardless of how many more than 3.

[jertel]

There's a possibility this could occur depending on how ElastAlert's run time correlates to the event times. Imagine the following timeline of minutes with the corresponding x events:

00:00:00              00:01:00                   00:02:00                  00:03:00                     00:04:00         
0--------------------------1--------------------------2--------------------------3--------------------------4
                                            x    x x      x x   x                                          

In this situation while there are 6 events within a minute span (00:01:30 - 00:02:30), this could still trigger two alerts, if ElastAlert 2 runs the rule at 00:03:00, since it will see 3 events in the first 1m timerange window (00:01:00 to 00:02:00) and then 3 more events in the next 1m window (00:02:00 to 00:03:00). So both timerange windows would have hit the threshold and caused an alert to be sent for each one.

[m-asri-w]

First of all thanks for the answer with the quite nice visualization. So i guess maybe i misunderstood they way elastaler2 works. Just to check, based on your answer, lets say i a have a config like as follow:

run_every: 3
buffer_time: 2
time_frame: 1

something like the following will happen

the buffer_time will specify the query time range that will happen on 00:00:03 and 00:00:06 and then the buffer_time is divided into time_frames and num_events is compared with number of events happened in each time_frame.

But my problem is that even when all the events happen at the same time_frame still i get more than one match. and for example in case of

run_every: 1
buffer_time: 1
time_frame: 1
num_event: 1

there must be no way to get more than 1 match/alert? right? but i still get more than one alert even for this setting. to be precise if the there is n event i get n match. Is this the intended behaviour or am i doing something wrong? should i try to solve this problem by modifying realert?

[jertel]

In a single timeframe, the matching events are accumulated into that timeframe and then the count of the accumulated matches are compared to the threshold. It does not reset once the threshold is exactly matched and start counting again within that same timeframe. So the behavior you're seeing suggests a configuration nuance, such as the use of query keys, that is causing it. I suggest enabling debug logs and carefully stepping through them to understand what's happening.

[m-asri-w]

Honestly, i can't see what is going wrong. I have been just going around the circle for two days. Here is my configuration and rules.

config.yaml :

rules_folder: /home/net/tmp/multiAlert/rules

run_every:
  minutes: 1

buffer_time:
  minutes: 1

es_host: localhost
es_port: 9200
use_ssl: False
es_username: elastic
es_password: elasticpass
writeback_index: elastalert_status
alert_time_limit:
  days: 2

my frequency rule, freq.yaml

name: Example frequency rule

type: frequency

index: to_monitor

num_events: 2

timeframe:
  minutes: 1

filter: []

realert:
  minutes: 0

alert:
  - mattermost

mattermost_webhook_url: 'http://192.168.7.190:8065/hooks/gtyc969j7t87xexxjupz48zdoh'

with this setting when there is 1 doc in my elasticsearch index in the past 1 min i get no message(alert) in mattermost, when there is 2 doc i get 1 message(alert) and if there is 4 or 6 i get 2 and 3 message(alert) in mattermost and so on.

the only thing that work is to set up

realert:
 minutes: 1

but based on your last comment even with

realert:
 minutes: 0

it had to send one alert per each time frame (here past 1 min) regardless of number of documents, whether it is 2 or 4 or 6, any count of of document more than or equal to 2 in the past 1 min would generate only one alert.