Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add new namespace "rule.*" #903

Open
wants to merge 44 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
44 commits
Select commit Hold shift + click to select a range
36792a1
add new namespace rule
trisch-me Apr 8, 2024
12dea45
Merge branch 'main' into rule_new
trisch-me Apr 8, 2024
ede2cfb
Merge branch 'main' into rule_new
trisch-me Apr 17, 2024
4dc213c
Merge branch 'main' into rule_new
trisch-me Apr 30, 2024
f775ba5
Merge branch 'main' of github.com:open-telemetry/semantic-conventions…
trisch-me May 7, 2024
ca5b8ea
updated for the weaver
trisch-me May 7, 2024
5400212
Merge branch 'main' into rule_new
trisch-me May 7, 2024
c9f9e98
Merge branch 'main' of github.com:open-telemetry/semantic-conventions…
trisch-me May 10, 2024
d8ce7fa
remove author until further discussion
trisch-me May 10, 2024
84f54a6
Merge branch 'rule_new' of github.com:trisch-me/semantic-conventions …
trisch-me May 10, 2024
6c9c593
fix registry
trisch-me May 10, 2024
b7f73f3
Merge branch 'main' into rule_new
trisch-me May 14, 2024
5c948f7
Merge branch 'main' into rule_new
trisch-me May 22, 2024
749e3ea
Merge branch 'main' into rule_new
trisch-me May 24, 2024
24bfc5a
Merge branch 'main' into rule_new
trisch-me Jun 5, 2024
fef0e7c
Merge branch 'main' into rule_new
trisch-me Jul 2, 2024
6fcc7ce
Merge branch 'main' of github.com:open-telemetry/semantic-conventions…
trisch-me Jul 5, 2024
bb8bd05
Merge branch 'main' of github.com:open-telemetry/semantic-conventions…
trisch-me Jul 18, 2024
eedc6d0
update rule sub namespace to be security
trisch-me Jul 18, 2024
1404699
Merge branch 'main' into rule_new
trisch-me Jul 22, 2024
6c370f9
Merge branch 'main' of github.com:open-telemetry/semantic-conventions…
trisch-me Jul 23, 2024
8560c69
Merge branch 'rule_new' of github.com:trisch-me/semantic-conventions …
trisch-me Jul 23, 2024
7b90fc8
rename rule.security to security_rule
trisch-me Jul 23, 2024
ffc315a
update templates for the new name
trisch-me Jul 23, 2024
1ba6d43
Merge branch 'main' into rule_new
joaopgrassi Jul 29, 2024
e57150c
Merge branch 'main' into rule_new
trisch-me Jul 29, 2024
908b0f2
remove prefix
trisch-me Aug 5, 2024
dc29cad
Merge branch 'rule_new' of github.com:trisch-me/semantic-conventions …
trisch-me Aug 5, 2024
cdf1286
Merge branch 'main' of github.com:open-telemetry/semantic-conventions…
trisch-me Aug 5, 2024
95afc21
Merge branch 'main' into rule_new
trisch-me Aug 12, 2024
dd320a9
Merge branch 'main' into rule_new
trisch-me Aug 13, 2024
9ea87ba
Merge branch 'main' of github.com:open-telemetry/semantic-conventions…
trisch-me Aug 20, 2024
962a1e7
remove rule.id from namespace
trisch-me Aug 20, 2024
0c5e4b9
Merge branch 'main' of github.com:open-telemetry/semantic-conventions…
trisch-me Sep 23, 2024
14ae893
update to the new structure
trisch-me Sep 23, 2024
5409311
Merge branch 'main' into rule_new
trisch-me Sep 23, 2024
1de85cb
Merge branch 'main' into rule_new
trisch-me Sep 30, 2024
31e4ff4
Merge branch 'main' of github.com:open-telemetry/semantic-conventions…
trisch-me Sep 30, 2024
1a057d6
Merge branch 'main' into rule_new
trisch-me Oct 7, 2024
a8a7f67
Merge branch 'main' into rule_new
trisch-me Oct 14, 2024
3209ed0
Merge branch 'main' into rule_new
trisch-me Oct 24, 2024
0dcddd5
Merge branch 'rule_new' of github.com:trisch-me/semantic-conventions …
trisch-me Oct 28, 2024
ad27236
Merge branch 'main' of github.com:open-telemetry/semantic-conventions…
trisch-me Oct 28, 2024
177e10d
update markdown
trisch-me Oct 28, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 22 additions & 0 deletions .chloggen/rule_new.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
# Use this changelog template to create an entry for release notes.
#
# If your change doesn't affect end users you should instead start
# your pull request title with [chore] or use the "Skip Changelog" label.

# One of 'breaking', 'deprecation', 'new_component', 'enhancement', 'bug_fix'
change_type: new_component

# The name of the area of concern in the attributes-registry, (e.g. http, cloud, db)
component: security-rule

# A brief description of the change. Surround your text with quotes ("") if it needs to start with a backtick (`).
note: Introducing a new security rule namespace

# Mandatory: One or more tracking issues related to the change. You can use the PR number here if no issue exists.
# The values here must be integers.
issues: [903]

# (Optional) One or more lines of additional information to render under the primary note.
# These lines will be padded with 2 spaces and then inserted directly into the document.
# Use pipe (|) for multiline entries.
subtext:
1 change: 1 addition & 0 deletions .github/ISSUE_TEMPLATE/bug_report.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -70,6 +70,7 @@ body:
- area:process
- area:profile
- area:rpc
- area:security-rule
- area:server
- area:service
- area:session
Expand Down
1 change: 1 addition & 0 deletions .github/ISSUE_TEMPLATE/change_proposal.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -62,6 +62,7 @@ body:
- area:process
- area:profile
- area:rpc
- area:security-rule
- area:server
- area:service
- area:session
Expand Down
1 change: 1 addition & 0 deletions .github/ISSUE_TEMPLATE/new-conventions.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -71,6 +71,7 @@ body:
- area:process
- area:profile
- area:rpc
- area:security-rule
- area:server
- area:service
- area:session
Expand Down
1 change: 1 addition & 0 deletions docs/attributes-registry/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -83,6 +83,7 @@ Currently, the following namespaces exist:
- [Process](process.md)
- [Profile](profile.md)
- [RPC](rpc.md)
- [Security Rule](security-rule.md)
- [Server](server.md)
- [Service](service.md)
- [Session](session.md)
Expand Down
24 changes: 24 additions & 0 deletions docs/attributes-registry/security-rule.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
<!--- Hugo front matter used to generate the website version of this page:
--->

<!-- NOTE: THIS FILE IS AUTOGENERATED. DO NOT EDIT BY HAND. -->
<!-- see templates/registry/markdown/attribute_namespace.md.j2 -->

# Security Rule

## Security Rule
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I know I'm a bit late here, I've been following this.

To some extent, it's unclear to me who will be generating this data, so if you can speak to how it's used in Elastic Common Schema and instances of logs that generate this data that could help.

Some major comments:

  • Rule vs Security Rule vs. Policy - it's hard to understand what the name/scope should be here, as it's very broad. E.g. the way this is phrased now, this could apply to OPA, which has a very clear definition of what is a policy and a rule (and I'd argue, a good foundation we can align with ECS as a way forward). I'm not in the details, but I think what you have aligns with rule there, so policy.rule may be a good namespace here, not that I want to litigate all that right now.
  • Namespacing and re-use within Semantic Conventions. I'm afraid we haven't resolved the discussion of embed sufficiently AND I feel like if we did, there'd be a LOT less contention on this PR.
  • The lack of an "Event", "span" or "Metric" defined for these makes it hard for general semconv maintainers (like myself) to understand what the eventual us case will be. However, given where Event stands in semconv, that would also block this PR so I know you want to make progress.
  • A couple nit-picky things, e.g. license being something that I don't understand yet (I'm not a domain expert here, so don't take this as a blocker, just something I think would need to be justified).

In the description, you call out several examples of who would use these attributes. Do you have example events that are produced by those things we could look at or point to? Until we unblock Events effectively and/or make progress on embed I think this could do a lot for "general" reviewers to be able to evaluate this.

TL;DR; I don't feel capable to approve this PR right now. I suggest one of two options:

  1. Help educate general semconv approvers on this topic, with more links, example instrumentation, etc.
  2. Bring in security experts (e.g. OPA folks, network policy folks, etc.) who can comment on this PR or approve it.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can try to address some of these, @trisch-me might know more or have more feedback

The usual security use case of these rule fields would be in alert event logs. A security product would run on endpoints monitoring/protecting systems by evaluating sets of rules loaded into the product. When a rule evaluates to true, the endpoint generates an alert event that includes information on what rule triggered the alert. SIEM products can also run detection rules on the data they have already ingested, which is an important use case, but less relevant to open telemetry, since the data is staying within the backend.

Examples of security rules:

The Falco documentation has some easy to understand documentation of what a security rule is: https://falco.org/docs/rules/basic-elements/

I think within security, the differences of rule vs. policy isn't always consistent or clear. Sometimes "rule" is only the detection condition, and sometimes the conditional and response action combined, so more like a OPA policy. Falco rules define the detection condition and response, by defining the priority and output content of the resulting alert. Elastic and Sigma rules only define the detection condition, and the response action is defined separately.

I'm unsure if it's common for OPA rules, or other non-security rules to be published in the same way security rules are, so I don't know if things like version, license, etc apply to them. If they do, I agree this namespace could be made more broad to cover other use cases too.

I could provide a raw alert event with ECS if you want, but it could be hard to understand and might not be too useful. Instead here's a screenshot of the alert UI, which is generated with the ECS event fields:
Screenshot 2024-08-20 at 2 57 13 PM
Screenshot 2024-08-20 at 2 59 51 PM

For the license, the rules are usually published as code, so the license is just the normal code copyright license that applies to it.


Describes security rule attributes. Rule fields are used to capture the specifics of any observer or agent rules that generate alerts or other notable events.

| Attribute | Type | Description | Examples | Stability |
|---|---|---|---|---|
| <a id="security-rule-category" href="#security-rule-category">`security_rule.category`</a> | string | A categorization value keyword used by the entity using the rule for detection of this event | `Attempted Information Leak` | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
| <a id="security-rule-description" href="#security-rule-description">`security_rule.description`</a> | string | The description of the rule generating the event. | `Block requests to public DNS over HTTPS / TLS protocols` | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
| <a id="security-rule-license" href="#security-rule-license">`security_rule.license`</a> | string | Name of the license under which the rule used to generate this event is made available. | `Apache 2.0` | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
| <a id="security-rule-name" href="#security-rule-name">`security_rule.name`</a> | string | The name of the rule or signature generating the event. | `BLOCK_DNS_over_TLS` | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
| <a id="security-rule-reference" href="#security-rule-reference">`security_rule.reference`</a> | string | Reference URL to additional information about the rule used to generate this event. [1] | `https://en.wikipedia.org/wiki/DNS_over_TLS` | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
| <a id="security-rule-ruleset" href="#security-rule-ruleset">`security_rule.ruleset`</a> | string | Name of the ruleset, policy, group, or parent category in which the rule used to generate this event is a member. | `Standard_Protocol_Filters` | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
| <a id="security-rule-uuid" href="#security-rule-uuid">`security_rule.uuid`</a> | string | A rule ID that is unique within the scope of a set or group of agents, observers, or other entities using the rule for detection of this event. | `550e8400-e29b-41d4-a716-446655440000`; `1100110011` | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
| <a id="security-rule-version" href="#security-rule-version">`security_rule.version`</a> | string | The version / revision of the rule being used for analysis. | `1.0.0` | ![Experimental](https://img.shields.io/badge/-experimental-blue) |

**[1]:** The URL can point to the vendor’s documentation about the rule. If that’s not available, it can also be a link to a more general page describing this type of alert.
60 changes: 60 additions & 0 deletions model/security-rule/registry.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,60 @@
groups:
- id: registry.security_rule
display_name: Security Rule
type: attribute_group
brief: >
Describes security rule attributes. Rule fields are used to capture the specifics of any observer or agent rules
that generate alerts or other notable events.
attributes:
- id: security_rule.category
type: string
stability: experimental
brief: >
A categorization value keyword used by the entity using the rule for detection of this event
examples: ['Attempted Information Leak']
- id: security_rule.description
type: string
stability: experimental
brief: >
The description of the rule generating the event.
examples: ['Block requests to public DNS over HTTPS / TLS protocols']
- id: security_rule.license
type: string
stability: experimental
brief: >
Name of the license under which the rule used to generate this event is made available.
examples: ['Apache 2.0']
- id: security_rule.name
type: string
stability: experimental
brief: >
The name of the rule or signature generating the event.
examples: ['BLOCK_DNS_over_TLS']
- id: security_rule.reference
type: string
stability: experimental
brief: >
Reference URL to additional information about the rule used to generate this event.
note: >
The URL can point to the vendor’s documentation about the rule.
If that’s not available, it can also be a link to a more general page describing this type of alert.
examples: ['https://en.wikipedia.org/wiki/DNS_over_TLS']
- id: security_rule.ruleset
type: string
stability: experimental
brief: >
Name of the ruleset, policy, group, or parent category in which the rule used to generate this event is a member.
examples: ['Standard_Protocol_Filters']
- id: security_rule.uuid
type: string
stability: experimental
brief: >
A rule ID that is unique within the scope of a set or group of agents, observers, or other entities
using the rule for detection of this event.
examples: ['550e8400-e29b-41d4-a716-446655440000', '1100110011']
- id: security_rule.version
type: string
stability: experimental
brief: >
The version / revision of the rule being used for analysis.
examples: ['1.0.0']
Loading