Skip to content

Commit

Permalink
Initial proposal for GenAI Security fields
Browse files Browse the repository at this point in the history
  • Loading branch information
susan-shu-c committed Aug 14, 2024
1 parent a4fc971 commit a673664
Show file tree
Hide file tree
Showing 2 changed files with 52 additions and 2 deletions.
17 changes: 15 additions & 2 deletions docs/attributes-registry/gen-ai.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,10 @@

# Gen AI

- [GenAI Attributes](#genai-attributes)
- [Deprecated GenAI Attributes](#deprecated-genai-attributes)
- [Gen AI](#gen-ai)
- [GenAI Attributes](#genai-attributes)
- [GenAI Security Attributes](#genai-security-attributes)
- [Deprecated GenAI Attributes](#deprecated-genai-attributes)

## GenAI Attributes

Expand Down Expand Up @@ -73,6 +75,17 @@ If none of these options apply, the `gen_ai.system` SHOULD be set to `_OTHER`.
| `input` | Input tokens (prompt, input, etc.) | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
| `output` | Output tokens (completion, response, etc.) | ![Experimental](https://img.shields.io/badge/-experimental-blue) |

## GenAI Security Attributes

| Attribute | Type | Description | Examples | Stability |
| ---------------------------------- | -------- | ------------------------------------------------------------------------------------------------ | ----------------------------------------------------------------------- | ---------------------------------------------------------------- |
| `gen_ai.policy.name` | string | Name of the specific policy that was triggered. | `contentPolicy` | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
| `gen_ai.policy.action` | string | Action taken due to a policy violation, such as blocking, alerting, or modifying the content. | `INTERVENED` | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
| `gen_ai.policy.confidence` | double | Confidence level in the policy match that triggered the action, quantifying how closely the identified content matched the policy criteria. | `0.8` | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
| `gen_ai.compliance.violation_detected` | boolean | Indicates if any compliance violation was detected during the interaction. | `True`;`False` | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
| `gen_ai.compliance.violation_code` | string | Code identifying the specific compliance rule that was violated. | `FedRAMP` | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
| `gen_ai.performance.request_size` | int | Size of the request payload in bytes. | `456000` | ![Experimental](https://img.shields.io/badge/-experimental-blue) |

## Deprecated GenAI Attributes

Describes deprecated `gen_ai` attributes.
Expand Down
37 changes: 37 additions & 0 deletions model/registry/gen-ai.yaml
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
// edit this file
groups:

Check failure on line 2 in model/registry/gen-ai.yaml

View workflow job for this annotation

GitHub Actions / yamllint

syntax error: mapping values are not allowed here (syntax)
- id: registry.gen_ai
type: attribute_group
Expand Down Expand Up @@ -148,3 +149,39 @@ groups:
If one of the predefined values applies, but specific system uses a different name it's RECOMMENDED to document it in the semantic
conventions for specific GenAI system and use system-specific name in the instrumentation.
If a different name is not documented, instrumentation libraries SHOULD use applicable predefined value.
- id: gen_ai.policy.name
stability: experimental
type: string
brief: Name of the specific policy that was triggered.
note:

Check failure on line 156 in model/registry/gen-ai.yaml

View workflow job for this annotation

GitHub Actions / yamllint

[trailing-spaces] trailing spaces
examples: contentPolicy
- id: gen_ai.policy.action
stability: experimental
type: string
brief: Action taken due to a policy violation, such as blocking, alerting, or modifying the content.
note:

Check failure on line 162 in model/registry/gen-ai.yaml

View workflow job for this annotation

GitHub Actions / yamllint

[trailing-spaces] trailing spaces
examples: INTERVENED
- id: gen_ai.policy.confidence
stability: experimental
type: double
brief: Confidence level in the policy match that triggered the action, quantifying how closely the identified content matched the policy criteria.
note:

Check failure on line 168 in model/registry/gen-ai.yaml

View workflow job for this annotation

GitHub Actions / yamllint

[trailing-spaces] trailing spaces
examples: 0.8
- id: gen_ai.compliance.violation_detected
stability: experimental
type: boolean
brief: Indicates if any compliance violation was detected during the interaction.
note:

Check failure on line 174 in model/registry/gen-ai.yaml

View workflow job for this annotation

GitHub Actions / yamllint

[trailing-spaces] trailing spaces
examples: True
- id: gen_ai.compliance.violation_code
stability: experimental
type: string
brief: Code identifying the specific compliance rule that was violated.
note:

Check failure on line 180 in model/registry/gen-ai.yaml

View workflow job for this annotation

GitHub Actions / yamllint

[trailing-spaces] trailing spaces
examples: FedRAMP
- id: gen_ai.performance.request_size
stability: experimental
type: int
brief: Size of the request payload in bytes.
note:

Check failure on line 186 in model/registry/gen-ai.yaml

View workflow job for this annotation

GitHub Actions / yamllint

[trailing-spaces] trailing spaces
examples: 456000

Check failure on line 187 in model/registry/gen-ai.yaml

View workflow job for this annotation

GitHub Actions / yamllint

[new-line-at-end-of-file] no new line character at the end of file

0 comments on commit a673664

Please sign in to comment.