Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Registry manifest and Schema diff #400

Open
wants to merge 54 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 8 commits
Commits
Show all changes
54 commits
Select commit Hold shift + click to select a range
edab834
feat(semconv): create registry manifest
lquerel Oct 3, 2024
fe7d7b5
feat(schema): Parse OTEL schema 1.1
lquerel Oct 4, 2024
76db2aa
feat(cli): Add registry diff command (WIP)
lquerel Oct 4, 2024
ad77910
Merge branch 'main' into generate-otel-schema
lquerel Oct 4, 2024
6de07ee
feat(schema): Support new deprecated field format
lquerel Oct 17, 2024
5efe6b1
Merge remote-tracking branch 'upstream/main' into generate-otel-schema
lquerel Oct 23, 2024
6cdfbfa
feat(diff): Add registry diff command.
lquerel Oct 23, 2024
880649f
feat(diff): Add diff for metrics
lquerel Oct 24, 2024
a87807b
feat(diff): Add diff for events, spans, and resources
lquerel Oct 24, 2024
8f8bdaf
feat(diff): Add template rendering capability
lquerel Oct 25, 2024
a818f4e
feat(diff): Add diff-format for ansi, ansi-stats, json, and yaml
lquerel Oct 28, 2024
05256ea
feat(diff): Add diff-format for markdown
lquerel Oct 29, 2024
c584f30
chore(diff): Fix unit tests
lquerel Oct 29, 2024
374def2
feat(cli): Prepare registry update-schema command
lquerel Oct 30, 2024
1b7c9c2
Merge branch 'main' into generate-otel-schema
lquerel Oct 30, 2024
2abd024
feat(cli): Prepare registry update-schema command
lquerel Oct 30, 2024
83f45eb
feat(cli): Prepare registry update-schema command
lquerel Oct 31, 2024
6312e42
chore(typo): fix typo in comment
lquerel Dec 2, 2024
6532bf3
Merge remote-tracking branch 'upstream/main' into generate-otel-schema
lquerel Dec 2, 2024
f5efacc
chore(build): Resolve merging with main upstream
lquerel Dec 2, 2024
9bda8cb
chore(build): Clean up code to prepare PR
lquerel Dec 2, 2024
edf85b0
chore(build): Clean up code to prepare PR
lquerel Dec 2, 2024
3b41c58
Merge branch 'main' into generate-otel-schema
lquerel Dec 6, 2024
bde3eef
chore(build): Prepare resolve_telemetry_schema to return WResult
lquerel Dec 6, 2024
904e856
Merge branch 'generate-otel-schema' of https://github.com/lquerel/ote…
lquerel Dec 6, 2024
3c4b6dd
chore(build): Fix `registry diff` to return a diff even when there ar…
lquerel Dec 9, 2024
c0609b5
Merge branch 'open-telemetry:main' into generate-otel-schema
lquerel Dec 17, 2024
6ba427c
chore(merge): Merge with main branch
lquerel Dec 17, 2024
ebcd15f
feat(diff): Test schema diff feature
lquerel Dec 19, 2024
8dcd311
feat(diff): Remove complex logic infering the type of deprecation bas…
lquerel Dec 19, 2024
b58d200
feat(diff): Rename the field Deprecated::Renamed::new_name into Depre…
lquerel Dec 19, 2024
a1ab93a
chore(doc): Fix documentation of the ResolvedTelemetrySchema::groups …
lquerel Dec 19, 2024
396450f
chore(doc): Fix documentation based on @jsuereth feedback
lquerel Dec 19, 2024
12c79c2
Merge remote-tracking branch 'upstream/main' into generate-otel-schema
lquerel Dec 19, 2024
0a63cd8
chore(merge): Merge with upstream main branch
lquerel Dec 19, 2024
e067b7f
Merge branch 'main' into generate-otel-schema
lquerel Dec 30, 2024
80121f5
feat(diff): Documentation of the schema diff command and the underlyi…
lquerel Dec 30, 2024
dbe4316
feat(diff): Compute a diff only on the registry attributes.
lquerel Dec 30, 2024
7e0a813
Merge branch 'main' into generate-otel-schema
lquerel Dec 30, 2024
bfdcfa9
feat(diff): Use the group.id as unique identifier for the groups.
lquerel Dec 30, 2024
e258547
feat(diff): Update unit tests to check deprecated attributes
lquerel Jan 2, 2025
eef3eff
Update crates/weaver_resolved_schema/src/lib.rs
lquerel Jan 2, 2025
619c885
Update crates/weaver_resolved_schema/src/lib.rs
lquerel Jan 2, 2025
b4c49e7
Merge branch 'generate-otel-schema' of https://github.com/lquerel/ote…
lquerel Jan 2, 2025
99b725d
chore: Merge with main upstream
lquerel Jan 2, 2025
0f430a8
chore(build): Update allowed_external_types to allow classic dependen…
lquerel Jan 2, 2025
78bfb77
chore(build): Update allowed_external_types to allow classic dependen…
lquerel Jan 2, 2025
98b7ad5
Merge branch 'main' into generate-otel-schema
lquerel Jan 17, 2025
06f3d19
chore(merge): Fix merge issue with main with main branch.
lquerel Jan 17, 2025
8fbdf8f
feat(schema-diff): Describe various situations of schema changes.
lquerel Jan 21, 2025
ab7936d
feat(schema-diff): Describe various situations of schema changes.
lquerel Jan 21, 2025
33fefc7
feat(schema-diff): Update schema changes intro
lquerel Jan 22, 2025
a141296
feat(schema-diff): Update schema changes intro
lquerel Jan 22, 2025
1388d1e
chore(doc): Update document talking about schema changes
lquerel Jan 23, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 11 additions & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

10 changes: 3 additions & 7 deletions crates/weaver_codegen_test/build.rs
Original file line number Diff line number Diff line change
Expand Up @@ -61,13 +61,9 @@ fn main() {
engine
.import_jq_package(SEMCONV_JQ)
.unwrap_or_else(|e| process_error(&logger, e));
let template_registry = ResolvedRegistry::try_from_resolved_registry(
schema
.registry(REGISTRY_ID)
.expect("Failed to get the registry from the resolved schema"),
schema.catalog(),
)
.unwrap_or_else(|e| process_error(&logger, e));
let template_registry =
ResolvedRegistry::try_from_resolved_registry(&schema.registry, schema.catalog())
.unwrap_or_else(|e| process_error(&logger, e));
let target_dir: PathBuf = target_dir.into();
engine
.generate(
Expand Down
3 changes: 1 addition & 2 deletions crates/weaver_common/src/diagnostic.rs
Original file line number Diff line number Diff line change
Expand Up @@ -180,8 +180,7 @@ impl DiagnosticMessages {
Self(vec![DiagnosticMessage::new(error)])
}

/// Returns true if all the diagnostic messages are explicitly marked as
/// warnings or advices.
/// Returns true if at least one diagnostic message has an error severity.
#[must_use]
pub fn has_error(&self) -> bool {
let non_error_count = self
Expand Down
36 changes: 16 additions & 20 deletions crates/weaver_forge/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -779,16 +779,14 @@ mod tests {
let schema = SchemaResolver::resolve_semantic_convention_registry(&mut registry)
.expect("Failed to resolve registry");

let template_registry = ResolvedRegistry::try_from_resolved_registry(
schema.registry(registry_id).expect("registry not found"),
schema.catalog(),
)
.unwrap_or_else(|e| {
panic!(
"Failed to create the context for the template evaluation: {:?}",
e
)
});
let template_registry =
ResolvedRegistry::try_from_resolved_registry(&schema.registry, schema.catalog())
.unwrap_or_else(|e| {
panic!(
"Failed to create the context for the template evaluation: {:?}",
e
)
});

// Delete all the files in the observed_output/target directory
// before generating the new files.
Expand Down Expand Up @@ -959,16 +957,14 @@ mod tests {
let schema = SchemaResolver::resolve_semantic_convention_registry(&mut registry)
.expect("Failed to resolve registry");

let template_registry = ResolvedRegistry::try_from_resolved_registry(
schema.registry(registry_id).expect("registry not found"),
schema.catalog(),
)
.unwrap_or_else(|e| {
panic!(
"Failed to create the context for the template evaluation: {:?}",
e
)
});
let template_registry =
ResolvedRegistry::try_from_resolved_registry(&schema.registry, schema.catalog())
.unwrap_or_else(|e| {
panic!(
"Failed to create the context for the template evaluation: {:?}",
e
)
});

engine
.generate(
Expand Down
3 changes: 2 additions & 1 deletion crates/weaver_forge/src/registry.rs
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ use weaver_resolved_schema::catalog::Catalog;
use weaver_resolved_schema::lineage::GroupLineage;
use weaver_resolved_schema::registry::{Constraint, Group, Registry};
use weaver_semconv::any_value::AnyValueSpec;
use weaver_semconv::deprecated::Deprecated;
use weaver_semconv::group::{GroupType, InstrumentSpec, SpanKindSpec};
use weaver_semconv::stability::Stability;

Expand Down Expand Up @@ -63,7 +64,7 @@ pub struct ResolvedGroup {
/// provided as `description` MUST specify why it's deprecated and/or what
/// to use instead. See also stability.
#[serde(skip_serializing_if = "Option::is_none")]
pub deprecated: Option<String>,
pub deprecated: Option<Deprecated>,
/// Additional constraints.
/// Allow to define additional requirements on the semantic convention.
/// It defaults to an empty list.
Expand Down
20 changes: 20 additions & 0 deletions crates/weaver_otel_schema/Cargo.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
[package]
name = "weaver_otel_schema"
version = "0.10.0"
authors.workspace = true
repository.workspace = true
license.workspace = true
publish.workspace = true
edition.workspace = true
rust-version.workspace = true

[lints]
workspace = true

[dependencies]
weaver_version = { path = "../weaver_version" }

thiserror.workspace = true
serde.workspace = true
serde_yaml.workspace = true

3 changes: 3 additions & 0 deletions crates/weaver_otel_schema/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
# OpenTelemetry Schema Data Model

This crate describes the data model for the OpenTelemetry telemetry schema.
6 changes: 6 additions & 0 deletions crates/weaver_otel_schema/allowed-external-types.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
# Copyright The OpenTelemetry Authors
# SPDX-License-Identifier: Apache-2.0
# This is used with cargo-check-external-types to reduce the surface area of downstream crates from
# the public API. Ideally this can have a few exceptions as possible.
allowed_external_types = [
]
85 changes: 85 additions & 0 deletions crates/weaver_otel_schema/src/lib.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,85 @@
// SPDX-License-Identifier: Apache-2.0

//! OpenTelemetry Schema Definitions
//! Please refer to the [OpenTelemetry Schema OTEP](https://github.com/open-telemetry/oteps/blob/main/text/0152-telemetry-schemas.md)
//! for more information.

use crate::Error::{InvalidTelemetrySchema, TelemetrySchemaNotFound};
use serde::{Deserialize, Serialize};
use weaver_version::Versions;

/// Errors emitted by this crate.
#[derive(thiserror::Error, Debug, Clone, Deserialize, Serialize)]
pub enum Error {
/// OTel Telemetry schema not found.
#[error("OTel telemetry schema not found (path_or_url: {path_or_url:?}).")]
TelemetrySchemaNotFound {
/// The path or the url to the telemetry schema file.
path_or_url: String,
},

/// Invalid OTel Telemetry schema.
#[error("Invalid OTel telemetry schema (path_or_url: {path_or_url:?}). {error}")]
InvalidTelemetrySchema {
/// The path or the url to the telemetry schema file.
path_or_url: String,
/// The error that occurred.
error: String,
},
}

/// An OpenTelemetry Telemetry Schema.
#[derive(Serialize, Deserialize, Debug)]
#[serde(deny_unknown_fields)]
pub struct TelemetrySchema {
/// Version of the file structure.
pub file_format: String,
/// Schema URL that this file is published at.
pub schema_url: String,
/// Definitions for each schema version in this family.
/// Note: the ordering of versions is defined according to semver
/// version number ordering rules.
/// This section is described in more details in the OTEP 0152 and in a dedicated
/// section below.
/// <https://github.com/open-telemetry/oteps/blob/main/text/0152-telemetry-schemas.md>
#[serde(skip_serializing_if = "Option::is_none")]
pub versions: Option<Versions>,
}

impl TelemetrySchema {
/// Attempts to load a telemetry schema from a file.
pub fn try_from_file<P: AsRef<std::path::Path>>(path: P) -> Result<Self, Error> {
let schema_path_buf = path.as_ref().to_path_buf();

if !schema_path_buf.exists() {
return Err(TelemetrySchemaNotFound {
path_or_url: schema_path_buf.as_path().to_string_lossy().to_string(),
});
}

let file = std::fs::File::open(path).map_err(|e| InvalidTelemetrySchema {
path_or_url: schema_path_buf.as_path().to_string_lossy().to_string(),
error: e.to_string(),
})?;
let reader = std::io::BufReader::new(file);
let schema: TelemetrySchema =
serde_yaml::from_reader(reader).map_err(|e| InvalidTelemetrySchema {
path_or_url: schema_path_buf.as_path().to_string_lossy().to_string(),
error: e.to_string(),
})?;

Ok(schema)
}
}

#[cfg(test)]
mod tests {
use crate::TelemetrySchema;

#[test]
fn test_try_from_file() {
let schema = TelemetrySchema::try_from_file("tests/test_data/1.27.0.yaml").unwrap();
assert_eq!(schema.file_format, "1.1.0");
assert_eq!(schema.schema_url, "https://opentelemetry.io/schemas/1.27.0");
}
}
Loading
Loading