Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add geo fields from ECS into client and server attributes #351

Closed
wants to merge 6 commits into from
Closed

Add geo fields from ECS into client and server attributes #351

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
4ae792a
Add geo fields from ECS
kaiyan-sheng Sep 27, 2023
9511ad3
add changelog
kaiyan-sheng Sep 27, 2023
f9d0633
Merge remote-tracking branch 'upstream/main' into add_geo_fields
kaiyan-sheng Nov 13, 2023
6bdb716
fix lint error
kaiyan-sheng Nov 13, 2023
c4daae0
add ECS geo fields into both client and server attribute
kaiyan-sheng Nov 13, 2023
ff6ba14
add links to reference
kaiyan-sheng Nov 14, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,8 @@ release.

### Features

- Add `geo` fields for client and server attribute.
([#351](https://github.com/open-telemetry/semantic-conventions/pull/351))
- Add `code.stacktrace` attribute
([#435](https://github.com/open-telemetry/semantic-conventions/pull/435))
- Add `http.flavor` and `http.user_agent` to list of deprecated attributes
Expand Down Expand Up @@ -118,7 +120,7 @@ stabilized.

- Remove experimental Kafka metrics ([#338](https://github.com/open-telemetry/semantic-conventions/pull/338))
- Adds `session.id` and session.md to general docs and model
([#215](https://github.com/open-telemetry/semantic-conventions/pull/215))
([#215](https://github.com/open-telemetry/semantic-conventions/pull/215))
- Add `container.labels.<key>` attributes.
([#125](https://github.com/open-telemetry/semantic-conventions/pull/125))
- Add `cluster.name` and `node.name` attributes to Elasticsearch semantic conventions.
Expand Down
30 changes: 28 additions & 2 deletions docs/attributes-registry/client.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,9 +15,35 @@ This also covers UDP network interactions where one side initiates the interacti
| Attribute | Type | Description | Examples |
|---|---|---|---|
| `client.address` | string | ![Stable](https://img.shields.io/badge/-stable-lightgreen)<br>Client address - domain name if available without reverse DNS lookup; otherwise, IP address or Unix domain socket name. [1] | `client.example.com`; `10.1.2.80`; `/tmp/my.sock` |
| `client.port` | int | ![Stable](https://img.shields.io/badge/-stable-lightgreen)<br>Client port number. [2] | `65123` |
| `client.geo.city_name` | string | City name. | `Montreal`; `Berlin` |
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we expect any client.geo and server.geo attributes to be present simultaneously? I thought the purpose of a global attribute registry is to represent attributes that mean the same wherever they are used, and that suits well for the geo.* attributes. That is, geo.city_name means the same whether it's used in the context of client or server or any other that's described at the end at https://www.elastic.co/guide/en/ecs/current/ecs-geo.html

Even if the client.geo.x and server.geo.x attributes are required in the same object, I think this problem is unsolved currently - for eg., one cannot have two url.full attributes in the same object if I have to represent the referring url or the url being redirected to in the case of 302.

I think we should omit the client and server prefix.

Copy link
Member

@AlexanderWert AlexanderWert Nov 24, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we expect any client.geo and server.geo attributes to be present simultaneously?

Why not? From a web-server access log you can easily derive both and can be of interest for observability users.

I thought the purpose of a global attribute registry is to represent attributes that mean the same wherever they are used, and that suits well for the geo.* attributes. That is, geo.city_name means the same whether it's used in the context of client or server or any other that's described at the end at https://www.elastic.co/guide/en/ecs/current/ecs-geo.html

That's right! But, the meaning doesn't change if you put an attribute into a more specific context, right? geo.city_name describes a city. client.geo.city_name still describes a city, but the city of the client, while server.geo.city_name describes the city of the server. And both can coexist on a single signal / event. I agree though that we should not define geo.city_name multiple times in different places, and that's what #339 aims at. I also created a concrete proposal on how to model it yaml: open-telemetry/build-tools#240

Even if the client.geo.x and server.geo.x attributes are required in the same object, I think this problem is unsolved currently - for eg., one cannot have two url.full attributes in the same object if I have to represent the referring url or the url being redirected to in the case of 302.

@scheler This is exactly what we are trying to solve with #339. For the geo example that means:

  • we will define geo.* fields only once in the registry
  • and then, geo.* fields will be (explicitly) reusable under different namespaces (while keeping their original definition just applied into the context of the target namespace). This will result in client.geo.* and server.geo.* anyways.
  • this will also solve the problem of having the same fields on a single signal with different context (as they would have different namespaces)

I think we should omit the client and server prefix.

  • We are just defining it here explicitly (for now) because we are still working on the reuse concept for semantic conventions and don't want to block Geo fields because of that.
  • If you mean that we should not have a prefix for geo at all (even with the reuse concept), then I don't fully get the reasoning, yet. As described above it solves a concrete problem we have today, that you also mentioned yourself with being able to have the same group of attributes twice on a signal with different context.
  • If we would omit the client, server prefixes for geo, we would break ECS with no reason (no actual conflict with semconv or attribute naming guidelines), which has been explicitly stated as something to avoid whenever possible in the original OTEP.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@AlexanderWert thanks for the clarification. One issue though is that one cannot use geo.city_name itself. Is that intended here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@scheler Yes it's intended. In Elastic common schema, geo fields are not expected to be used directly at the root of the events.

| `client.geo.continent_code` | string | Two-letter code representing continent’s name. | `AF` |
| `client.geo.continent_name` | string | Name of the continent. | `North America`; `Europe` |
| `client.geo.country_iso_code` | string | Two-letter ISO Country Code ([ISO 3166-1 alpha2](https://en.wikipedia.org/wiki/ISO_3166-1#Codes)). | `CA` |
| `client.geo.country_name` | string | Country name. | `Canada` |
| `client.geo.location.lat` | double | Latitude of the geo location in [WGS84](https://en.wikipedia.org/wiki/World_Geodetic_System#WGS84). | `45.505918` |
| `client.geo.location.lon` | double | Longitude of the geo location in [WGS84](https://en.wikipedia.org/wiki/World_Geodetic_System#WGS84). | `-73.61483` |
| `client.geo.name` | string | User-defined description of a location. [2] | `boston-dc` |
| `client.geo.postal_code` | string | Postal code associated with the location. Values appropriate for this field may also be known as a postcode or ZIP code and will vary widely from country to country. | `94040` |
| `client.geo.region_iso_code` | string | Region ISO code ([ISO 3166-2](https://en.wikipedia.org/wiki/ISO_3166-2)). | `CA-QC` |
| `client.geo.region_name` | string | Region name. | `Quebec` |
| `client.geo.timezone` | string | The time zone of the location, such as [IANA time zone name](https://nodatime.org/TimeZones). | `America/Argentina/Buenos_Aires` |
| `client.port` | int | ![Stable](https://img.shields.io/badge/-stable-lightgreen)<br>Client port number. [3] | `65123` |

**[1]:** When observed from the server side, and when communicating through an intermediary, `client.address` SHOULD represent the client address behind any intermediaries, for example proxies, if it's available.

**[2]:** When observed from the server side, and when communicating through an intermediary, `client.port` SHOULD represent the client port behind any intermediaries, for example proxies, if it's available.
**[2]:** User-defined description of a location, at the level of granularity they care about. Could be the name of their data centers, the floor number, if this describes a local physical entity, city names. Not typically used in automated geolocation.

**[3]:** When observed from the server side, and when communicating through an intermediary, `client.port` SHOULD represent the client port behind any intermediaries, for example proxies, if it's available.

`client.geo.continent_code` MUST be one of the following:

| Value | Description |
|---|---|
| `AF` | Africa |
| `AN` | Antarctica |
| `AS` | Asia |
| `EU` | Europe |
| `NA` | North America |
| `OC` | Oceania |
| `SA` | South America |
<!-- endsemconv -->
30 changes: 28 additions & 2 deletions docs/attributes-registry/server.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,9 +15,35 @@ This also covers UDP network interactions where one side initiates the interacti
| Attribute | Type | Description | Examples |
|---|---|---|---|
| `server.address` | string | ![Stable](https://img.shields.io/badge/-stable-lightgreen)<br>Server domain name if available without reverse DNS lookup; otherwise, IP address or Unix domain socket name. [1] | `example.com`; `10.1.2.80`; `/tmp/my.sock` |
| `server.port` | int | ![Stable](https://img.shields.io/badge/-stable-lightgreen)<br>Server port number. [2] | `80`; `8080`; `443` |
| `server.geo.city_name` | string | City name. | `Montreal`; `Berlin` |
| `server.geo.continent_code` | string | Two-letter code representing continent’s name. | `AF` |
| `server.geo.continent_name` | string | Name of the continent. | `North America`; `Europe` |
| `server.geo.country_iso_code` | string | Two-letter ISO Country Code ([ISO 3166-1 alpha2](https://en.wikipedia.org/wiki/ISO_3166-1#Codes)). | `CA` |
| `server.geo.country_name` | string | Country name. | `Canada` |
| `server.geo.location.lat` | double | Latitude of the geo location in [WGS84](https://en.wikipedia.org/wiki/World_Geodetic_System#WGS84). | `45.505918` |
| `server.geo.location.lon` | double | Longitude of the geo location in [WGS84](https://en.wikipedia.org/wiki/World_Geodetic_System#WGS84). | `-73.61483` |
| `server.geo.name` | string | User-defined description of a location. [2] | `boston-dc` |
| `server.geo.postal_code` | string | Postal code associated with the location. Values appropriate for this field may also be known as a postcode or ZIP code and will vary widely from country to country. | `94040` |
| `server.geo.region_iso_code` | string | Region ISO code ([ISO 3166-2](https://en.wikipedia.org/wiki/ISO_3166-2)). | `CA-QC` |
| `server.geo.region_name` | string | Region name. | `Quebec` |
| `server.geo.timezone` | string | The time zone of the location, such as [IANA time zone name](https://nodatime.org/TimeZones). | `America/Argentina/Buenos_Aires` |
| `server.port` | int | ![Stable](https://img.shields.io/badge/-stable-lightgreen)<br>Server port number. [3] | `80`; `8080`; `443` |

**[1]:** When observed from the client side, and when communicating through an intermediary, `server.address` SHOULD represent the server address behind any intermediaries, for example proxies, if it's available.

**[2]:** When observed from the client side, and when communicating through an intermediary, `server.port` SHOULD represent the server port behind any intermediaries, for example proxies, if it's available.
**[2]:** User-defined description of a location, at the level of granularity they care about. Could be the name of their data centers, the floor number, if this describes a local physical entity, city names. Not typically used in automated geolocation.

**[3]:** When observed from the client side, and when communicating through an intermediary, `server.port` SHOULD represent the server port behind any intermediaries, for example proxies, if it's available.

`server.geo.continent_code` MUST be one of the following:

| Value | Description |
|---|---|
| `AF` | Africa |
| `AN` | Antarctica |
| `AS` | Asia |
| `EU` | Europe |
| `NA` | North America |
| `OC` | Oceania |
| `SA` | South America |
<!-- endsemconv -->
87 changes: 87 additions & 0 deletions model/registry/client.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,3 +26,90 @@ groups:
note: >
When observed from the server side, and when communicating through an intermediary, `client.port` SHOULD represent
the client port behind any intermediaries, for example proxies, if it's available.
- id: geo.city_name
type: string
brief: >
City name.
examples: [ 'Montreal', 'Berlin' ]
- id: geo.continent_code
brief: >
Two-letter code representing continent’s name.
type:
allow_custom_values: false
members:
- id: 'af'
value: 'AF'
brief: 'Africa'
- id: 'an'
value: 'AN'
brief: 'Antarctica'
- id: 'as'
value: 'AS'
brief: 'Asia'
- id: 'eu'
value: 'EU'
brief: 'Europe'
- id: 'na'
value: 'NA'
brief: 'North America'
- id: 'oc'
value: 'OC'
brief: 'Oceania'
- id: 'sa'
value: 'SA'
brief: 'South America'
- id: geo.continent_name
type: string
brief: >
Name of the continent.
examples: [ 'North America', 'Europe' ]
- id: geo.country_iso_code
type: string
brief: >
Two-letter ISO Country Code ([ISO 3166-1 alpha2](https://en.wikipedia.org/wiki/ISO_3166-1#Codes)).
examples: [ 'CA' ]
- id: geo.country_name
type: string
brief: >
Country name.
examples: [ 'Canada' ]
- id: geo.location.lon
type: double
brief: >
Longitude of the geo location in [WGS84](https://en.wikipedia.org/wiki/World_Geodetic_System#WGS84).
examples: [ -73.614830 ]
- id: geo.location.lat
type: double
brief: >
Latitude of the geo location in [WGS84](https://en.wikipedia.org/wiki/World_Geodetic_System#WGS84).
examples: [ 45.505918 ]
- id: geo.name
type: string
brief: >
User-defined description of a location.
note: >
User-defined description of a location, at the level of granularity they care about.
Could be the name of their data centers, the floor number, if this describes a local physical entity, city names.
Not typically used in automated geolocation.
examples: [ 'boston-dc' ]
- id: geo.postal_code
type: string
brief: >
Postal code associated with the location.
Values appropriate for this field may also be known as a postcode or ZIP code and will vary widely from country to country.
examples: [ '94040' ]
- id: geo.region_iso_code
type: string
brief: >
Region ISO code ([ISO 3166-2](https://en.wikipedia.org/wiki/ISO_3166-2)).
examples: [ 'CA-QC' ]
- id: geo.region_name
type: string
brief: >
Region name.
examples: [ 'Quebec' ]
- id: geo.timezone
type: string
brief: >
The time zone of the location, such as [IANA time zone name](https://nodatime.org/TimeZones).
examples: [ 'America/Argentina/Buenos_Aires' ]
87 changes: 87 additions & 0 deletions model/registry/server.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,3 +26,90 @@ groups:
When observed from the client side, and when communicating through an intermediary, `server.port` SHOULD represent
the server port behind any intermediaries, for example proxies, if it's available.
examples: [80, 8080, 443]
- id: geo.city_name
type: string
brief: >
City name.
examples: [ 'Montreal', 'Berlin' ]
- id: geo.continent_code
brief: >
Two-letter code representing continent’s name.
type:
allow_custom_values: false
members:
- id: 'af'
value: 'AF'
brief: 'Africa'
- id: 'an'
value: 'AN'
brief: 'Antarctica'
- id: 'as'
value: 'AS'
brief: 'Asia'
- id: 'eu'
value: 'EU'
brief: 'Europe'
- id: 'na'
value: 'NA'
brief: 'North America'
- id: 'oc'
value: 'OC'
brief: 'Oceania'
- id: 'sa'
value: 'SA'
brief: 'South America'
- id: geo.continent_name
type: string
brief: >
Name of the continent.
examples: [ 'North America', 'Europe' ]
- id: geo.country_iso_code
type: string
brief: >
Two-letter ISO Country Code ([ISO 3166-1 alpha2](https://en.wikipedia.org/wiki/ISO_3166-1#Codes)).
examples: [ 'CA' ]
- id: geo.country_name
type: string
brief: >
Country name.
examples: [ 'Canada' ]
- id: geo.location.lon
type: double
brief: >
Longitude of the geo location in [WGS84](https://en.wikipedia.org/wiki/World_Geodetic_System#WGS84).
examples: [ -73.614830 ]
- id: geo.location.lat
type: double
brief: >
Latitude of the geo location in [WGS84](https://en.wikipedia.org/wiki/World_Geodetic_System#WGS84).
examples: [ 45.505918 ]
- id: geo.name
type: string
brief: >
User-defined description of a location.
note: >
User-defined description of a location, at the level of granularity they care about.
Could be the name of their data centers, the floor number, if this describes a local physical entity, city names.
Not typically used in automated geolocation.
examples: [ 'boston-dc' ]
- id: geo.postal_code
type: string
brief: >
Postal code associated with the location.
Values appropriate for this field may also be known as a postcode or ZIP code and will vary widely from country to country.
examples: [ '94040' ]
- id: geo.region_iso_code
type: string
brief: >
Region ISO code ([ISO 3166-2](https://en.wikipedia.org/wiki/ISO_3166-2)).
examples: [ 'CA-QC' ]
- id: geo.region_name
type: string
brief: >
Region name.
examples: [ 'Quebec' ]
- id: geo.timezone
type: string
brief: >
The time zone of the location, such as [IANA time zone name](https://nodatime.org/TimeZones).
examples: [ 'America/Argentina/Buenos_Aires' ]
Loading