Experimental: Do If rules

This is experimental feature and represents an advanced version of match_fields. The Do If rules are a tree of nodes. The tree is stored in the Do If Checker instance. When Do If Checker's Match func is called it calls to the root Match func and then the chain of Match func calls are performed across the whole tree.

Node types

FieldOp Type of node where matching rules for fields are stored.

LengthCmpOp Type of node where matching rules for byte length and array length are stored.

TimestampCmpOp Type of node where matching rules for timestamps are stored.

LogicalOp Type of node where logical rules for applying other rules are stored.

Field op node

DoIf field op node is considered to always be a leaf in the DoIf tree. It contains operation to be checked on the field value, the field name to extract data and the values to check against.

Params:

op - value from field operations list. Required.
field - name of the field to apply operation. Required.
values - list of values to check field. Required non-empty.
case_sensitive - flag indicating whether checks are performed in case sensitive way. Default true. Note: case insensitive checks can cause CPU and memory overhead since every field value will be converted to lower letters.

Example:

pipelines:
  tests:
    actions:
      - type: discard
        do_if:
          op: suffix
          field: pod
          values: [pod-1, pod-2]
          case_sensitive: true

Field operations

Equal checks whether the field value is equal to one of the elements in the values list.

Example:

pipelines:
  test:
    actions:
      - type: discard
        do_if:
          op: equal
          field: pod
          values: [test-pod-1, test-pod-2]

result:

{"pod":"test-pod-1","service":"test-service"}   # discarded
{"pod":"test-pod-2","service":"test-service-2"} # discarded
{"pod":"test-pod","service":"test-service"}     # not discarded
{"pod":"test-pod","service":"test-service-1"}   # not discarded

Contains checks whether the field value contains one of the elements the in values list.

Example:

pipelines:
  test:
    actions:
      - type: discard
        do_if:
          op: contains
          field: pod
          values: [my-pod, my-test]

result:

{"pod":"test-my-pod-1","service":"test-service"}     # discarded
{"pod":"test-not-my-pod","service":"test-service-2"} # discarded
{"pod":"my-test-pod","service":"test-service"}       # discarded
{"pod":"test-pod","service":"test-service-1"}        # not discarded

Prefix checks whether the field value has prefix equal to one of the elements in the values list.

Example:

pipelines:
  test:
    actions:
      - type: discard
        do_if:
          op: prefix
          field: pod
          values: [test-1, test-2]

result:

{"pod":"test-1-pod-1","service":"test-service"}   # discarded
{"pod":"test-2-pod-2","service":"test-service-2"} # discarded
{"pod":"test-pod","service":"test-service"}       # not discarded
{"pod":"test-pod","service":"test-service-1"}     # not discarded

Suffix checks whether the field value has suffix equal to one of the elements in the values list.

Example:

pipelines:
  test:
    actions:
      - type: discard
        do_if:
          op: suffix
          field: pod
          values: [pod-1, pod-2]

result:

{"pod":"test-1-pod-1","service":"test-service"}   # discarded
{"pod":"test-2-pod-2","service":"test-service-2"} # discarded
{"pod":"test-pod","service":"test-service"}       # not discarded
{"pod":"test-pod","service":"test-service-1"}     # not discarded

Regex checks whether the field matches any regex from the values list.

Example:

pipelines:
  test:
    actions:
      - type: discard
        do_if:
          op: regex
          field: pod
          values: [pod-\d, my-test.*]

result:

{"pod":"test-1-pod-1","service":"test-service"}       # discarded
{"pod":"test-2-pod-2","service":"test-service-2"}     # discarded
{"pod":"test-pod","service":"test-service"}           # not discarded
{"pod":"my-test-pod","service":"test-service-1"}      # discarded
{"pod":"my-test-instance","service":"test-service-1"} # discarded
{"pod":"service123","service":"test-service-1"}       # not discarded

Logical op node

DoIf logical op node is a node considered to be the root or an edge between nodes. It always has at least one operand which are other nodes and calls their checks to apply logical operation on their results.

Params:

op - value from logical operations list. Required.
operands - list of another do-if nodes. Required non-empty.

Example:

pipelines:
  test:
    actions:
      - type: discard
        do_if:
          op: and
          operands:
            - op: equal
              field: pod
              values: [test-pod-1, test-pod-2]
              case_sensitive: true
            - op: equal
              field: service
              values: [test-service]
              case_sensitive: true

Logical operations

Or accepts at least one operand and returns true on the first returned true from its operands.

Example:

pipelines:
  test:
    actions:
      - type: discard
        do_if:
          op: or
          operands:
            - op: equal
              field: pod
              values: [test-pod-1, test-pod-2]
            - op: equal
              field: service
              values: [test-service]

result:

{"pod":"test-pod-1","service":"test-service"}   # discarded
{"pod":"test-pod-2","service":"test-service-2"} # discarded
{"pod":"test-pod","service":"test-service"}     # discarded
{"pod":"test-pod","service":"test-service-1"}   # not discarded

And accepts at least one operand and returns true if all operands return true (in other words returns false on the first returned false from its operands).

Example:

pipelines:
  test:
    actions:
      - type: discard
        do_if:
          op: and
          operands:
            - op: equal
              field: pod
              values: [test-pod-1, test-pod-2]
            - op: equal
              field: service
              values: [test-service]

result:

{"pod":"test-pod-1","service":"test-service"}   # discarded
{"pod":"test-pod-2","service":"test-service-2"} # not discarded
{"pod":"test-pod","service":"test-service"}     # not discarded
{"pod":"test-pod","service":"test-service-1"}   # not discarded

Not accepts exactly one operand and returns inverted result of its operand.

Example:

pipelines:
  test:
    actions:
      - type: discard
        do_if:
          op: not
          operands:
            - op: equal
              field: service
              values: [test-service]

result:

{"pod":"test-pod-1","service":"test-service"}   # not discarded
{"pod":"test-pod-2","service":"test-service-2"} # discarded
{"pod":"test-pod","service":"test-service"}     # not discarded
{"pod":"test-pod","service":"test-service-1"}   # discarded

Length comparison op node

DoIf length comparison op node is considered to always be a leaf in the DoIf tree like DoIf field op node. It contains operation that compares field length in bytes or array length (for array fields) with certain value.

Params:

op - must be byte_len_cmp or array_len_cmp. Required.
field - name of the field to apply operation. Required.
cmp_op - comparison operation name (see below). Required.
value - integer value to compare length with. Required non-negative.

Example 1 (byte length comparison):

pipelines:
  test:
    actions:
      - type: discard
        do_if:
          op: byte_len_cmp
          field: pod_id
          cmp_op: lt
          value: 5

Result:

{"pod_id":""}      # discarded
{"pod_id":123}     # discarded
{"pod_id":12345}   # not discarded
{"pod_id":123456}  # not discarded

Example 2 (array length comparison):

pipelines:
  test:
    actions:
      - type: discard
        do_if:
          op: array_len_cmp
          field: items
          cmp_op: lt
          value: 2

Result:

{"items":[]}         # discarded
{"items":[1]}        # discarded
{"items":[1, 2]}     # not discarded
{"items":[1, 2, 3]}  # not discarded
{"items":"1"}        # not discarded ('items' is not an array)
{"numbers":[1]}      # not discarded ('items' not found)

Possible values of field cmp_op: lt, le, gt, ge, eq, ne. They denote corresponding comparison operations.

Name	Op
`lt`	`<`
`le`	`<=`
`gt`	`>`
`ge`	`>=`
`eq`	`==`
`ne`	`!=`

Timestamp comparison op node

DoIf timestamp comparison op node is considered to always be a leaf in the DoIf tree like DoIf field op node. It contains operation that compares timestamps with certain value.

Params:

op - must be ts_cmp. Required.
field - name of the field to apply operation. Required. Field will be parsed with time.Parse function.
format - format for timestamps representation. Optional; default = time.RFC3339Nano.
cmp_op - comparison operation name (same as for length comparison operations). Required.
value - timestamp value to compare field timestamps with. It must have RFC3339Nano format. Required. Also, it may be now or file_d_start. If it is now then value to compare timestamps with is periodically updated current time. If it is file_d_start then value to compare timestamps with will be program start moment.
value_shift - duration that adds to value before comparison. It can be negative. Useful when value is now. Optional; default = 0.
update_interval - if value is now then you can set update interval for that value. Optional; default = 10s. Actual cmp value in that case is now + value_shift + update_interval.

Example (discard all events with timestamp field value LESS than 2010-01-01T00:00:00Z):

pipelines:
  test:
    actions:
      - type: discard
        do_if:
          op: ts_cmp
          field: timestamp
          cmp_op: lt
          value: 2010-01-01T00:00:00Z
          format: 2006-01-02T15:04:05.999999999Z07:00

Result:

{"timestamp":"2000-01-01T00:00:00Z"}         # discarded
{"timestamp":"2008-01-01T00:00:00Z","id":1}  # discarded

{"pod_id":"some"}    # not discarded (no field `timestamp`)
{"timestamp":123}    # not discarded (field `timestamp` is not string)
{"timestamp":"qwe"}  # not discarded (field `timestamp` is not parsable)

{"timestamp":"2011-01-01T00:00:00Z"}  # not discarded (condition is not met)

Generated using insane-doc

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

README.md

README.md

Experimental: Do If rules

Node types

Field op node

Field operations

Logical op node

Logical operations

Length comparison op node

Timestamp comparison op node

Files

README.md

Latest commit

History

README.md

File metadata and controls

Experimental: Do If rules

Node types

Field op node

Field operations

Logical op node

Logical operations

Length comparison op node

Timestamp comparison op node