Help with my first alert

[wmashal]

Hi

I have now my opensearch and elastalert2 dockers deployed, what I am trying to do is just run my first simple alert as a test for my POC. but in the Elastalert2 rule still no matching see the following

2024-03-26 15:27:05 INFO:elastalert:Exemple email alert range 900
2024-03-26 15:27:09 INFO:elastalert:Background configuration change check run at 2024-03-26 13:27 UTC
2024-03-26 15:27:09 INFO:elastalert:Background alerts thread 0 pending alerts sent at 2024-03-26 13:27 UTC
2024-03-26 15:27:09 INFO:elastalert:Disabled rules are: []
2024-03-26 15:27:09 INFO:elastalert:Sleeping for 9.999276 seconds
2024-03-26 15:27:18 INFO:elastalert:Queried rule Exemple email alert from 2024-03-26 13:12 UTC to 2024-03-26 13:27 UTC: 0 / 0 hits
2024-03-26 15:27:18 INFO:elastalert:Ran Exemple email alert from 2024-03-26 13:12 UTC to 2024-03-26 13:27 UTC: 0 query hits (0 already seen), 0 matches, 0 alerts sent

I add some records in the mariadblog-index like the following

curl -X POST http://localhost:9200/mariadblog-index/_doc -H 'Content-Type: application/json' -d '{"id":"5503","@timestamp": "2024-03-26T17:13:00Z", "blog_title": "My Awesome MariaDB Experience", "author": "Tech Enthusiast","content": "Testttttt"}'

My Target
send email if the document contains id 2501 OR 5503

My Rule and Alert

name: "Exemple email alert"
type: any
index: "mariadblog-index"
use_strftime_index: true

# Exemple query
filter:
- terms:
    id: ["2501", "5503"]
#- query:
#    query_string:
#      query: "id: 2501 OR id: 5503"

realert:
  minutes: 0

alert_subject: "Test {} 123 aa"
alert_subject_args:
  - "message"
  - "@log_name"
alert_text: "Test {}  123 bb"
alert_text_args:
  - "message"

# Needed
alert:
  - "email"
from_addr: "[email protected]"
email: "[email protected]"

I can retrieve the records from my index

(base) ➜  elastalert2 git:(master) ✗ curl http://localhost:9200/mariadblog-\*/_search\?_source_includes\=%40timestamp%2C%2A\&ignore_unavailable\=true\&scroll\=30s\&size\=10000 | jq .
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  2556  100  2556    0     0   120k      0 --:--:-- --:--:-- --:--:--  124k
{
  "_scroll_id": "FGluY2x1ZGVfY29udGV4dF91dWlkDXF1ZXJ5QW5kRmV0Y2gBFnV4NF95WmR4U2cyLWNrMWNqN0xzZ2cAAAAAAAAD6RY4TE93a2NMZFI1MlhlZ19hUTdJZENn",
  "took": 11,
  "timed_out": false,
  "_shards": {
    "total": 1,
    "successful": 1,
    "skipped": 0,
    "failed": 0
  },
  "hits": {
    "total": {
      "value": 10,
      "relation": "eq"
    },
    "max_score": 1,
    "hits": [
      {
        "_index": "mariadblog-index",
        "_id": "1",
        "_score": 1,
        "_source": {
          "@timestamp": "2024-03-26T16:40:00Z",
          "author": "Tech Enthusiast",
          "blog_title": "My Awesome MariaDB Experience",
          "content": "Today, I explored MariaDB and here is what I learned..."
        }
      },
      {
        "_index": "mariadblog-index",
        "_id": "T-7Deo4B80pDeqdsfHl-",
        "_score": 1,
        "_source": {
          "@timestamp": "2024-03-26T16:41:00Z",
          "author": "Tech Enthusiast",
          "blog_title": "My Awesome MariaDB Experience",
          "content": "Today, I explored MariaDB and here is what I learned..."
        }
      },
      {
        "_index": "mariadblog-index",
        "_id": "VO7Eeo4B80pDeqdsSHk3",
        "_score": 1,
        "_source": {
          "@timestamp": "2024-03-26T16:42:00Z",
          "author": "Tech Enthusiast",
          "blog_title": "My Awesome MariaDB Experience",
          "content": "Test 1"
        }
      },
      {
        "_index": "mariadblog-index",
        "_id": "Wu7Feo4B80pDeqdsP3k4",
        "_score": 1,
        "_source": {
          "@timestamp": "2024-03-26T16:44:00Z",
          "author": "Tech Enthusiast",
          "blog_title": "My Awesome MariaDB Experience",
          "content": "Test 2"
        }
      },
      {
        "_index": "mariadblog-index",
        "_id": "Y-7Geo4B80pDeqdsrXm6",
        "_score": 1,
        "_source": {
          "author": "Tech Enthusiast",
          "blog_title": "My Awesome MariaDB Experience",
          "content": "Test 2",
          "timestamp": "2024-03-26T16:45:00Z"
        }
      },
      {
        "_index": "mariadblog-index",
        "_id": "r-7Veo4B80pDeqdsgnlB",
        "_score": 1,
        "_source": {
          "@timestamp": "2024-03-26T16:45:00Z",
          "author": "Tech Enthusiast",
          "id": "2501",
          "blog_title": "My Awesome MariaDB Experience",
          "content": "Test 2"
        }
      },
      {
        "_index": "mariadblog-index",
        "_id": "s-7Weo4B80pDeqdsPnkP",
        "_score": 1,
        "_source": {
          "@timestamp": "2024-03-26T16:45:00Z",
          "author": "Tech Enthusiast",
          "id": "5503",
          "blog_title": "My Awesome MariaDB Experience",
          "content": "Testttttt"
        }
      },
      {
        "_index": "mariadblog-index",
        "_id": "Ee7ieo4B80pDeqdsV3qi",
        "_score": 1,
        "_source": {
          "@timestamp": "2024-03-26T17:11:00Z",
          "author": "Tech Enthusiast",
          "id": "5503",
          "blog_title": "My Awesome MariaDB Experience",
          "content": "Testttttt"
        }
      },
      {
        "_index": "mariadblog-index",
        "_id": "E-7ieo4B80pDeqdsiHov",
        "_score": 1,
        "_source": {
          "@timestamp": "2024-03-26T17:12:00Z",
          "author": "Tech Enthusiast",
          "id": "5503",
          "blog_title": "My Awesome MariaDB Experience",
          "content": "Testttttt"
        }
      },
      {
        "_index": "mariadblog-index",
        "_id": "Fe7ieo4B80pDeqdstXoy",
        "_score": 1,
        "_source": {
          "@timestamp": "2024-03-26T17:13:00Z",
          "author": "Tech Enthusiast",
          "id": "5503",
          "blog_title": "My Awesome MariaDB Experience",
          "content": "Testttttt"
        }
      }
    ]
  }
}

[jertel]

You can enable debug logging so you can see the actual query being sent to Elastic.

But from what I see in your post if looks like you are searching a different time range from the document timestamps being inserted.

[wmashal]

@jertel I fixed the timestamp format and I see matches but still the email not sent

2024-03-27 12:10:50 INFO:elastalert:Queried rule Exemple email alert from 2024-03-27 09:55 UTC to 2024-03-27 10:10 UTC: 3 / 3 hits
2024-03-27 12:10:50 INFO:elastalert:Alert for Exemple email alert at 2024-03-27T09:59:18Z:
2024-03-27 12:10:50 INFO:elastalert:Test <MISSING VALUE>  123 bb
2024-03-27 12:10:50 
2024-03-27 12:10:50 @timestamp: 2024-03-27T09:59:18Z
2024-03-27 12:10:50 _id: yJ1kf44BgsG2vE3ODUbh
2024-03-27 12:10:50 _index: mariadblog-index
2024-03-27 12:10:50 author: Tech Enthusiast
2024-03-27 12:10:50 blog_title: My Awesome MariaDB Experience
2024-03-27 12:10:50 content: Test 2
2024-03-27 12:10:50 id: 5503
2024-03-27 12:10:50 num_hits: 3
2024-03-27 12:10:50 num_matches: 1
2024-03-27 12:10:50 
2024-03-27 12:10:50 INFO:elastalert:Skipping writing to ES: {'rule_name': 'Exemple email alert', 'endtime': '2024-03-27T10:10:50.904147Z', 'starttime': '2024-03-27T09:55:50.904147Z', 'matches': 1, 'hits': 3, '@timestamp': '2024-03-27T10:10:50.914767Z', 'time_taken': 0.01061248779296875}
2024-03-27 12:10:50 INFO:elastalert:Ran Exemple email alert from 2024-03-27 09:55 UTC to 2024-03-27 10:10 UTC: 3 query hits (2 already seen), 1 matches, 0 alerts sent

[jertel]

That log output appears to show you are using debug mode. How did you run that command line?