Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[etcd] Add metric type mapping to self datastream #8407

Closed
wants to merge 2 commits into from

Conversation

agithomas
Copy link
Contributor

@agithomas agithomas commented Nov 6, 2023

  • Enhancement

Proposed commit message

  • Added metric_type mapping to self datastream.
  • The change is related to the TSDB enablement of the package.

Checklist

  • I have reviewed tips for building integrations and this pull request is aligned with them.
  • I have verified that all data streams collect metrics or logs.
  • I have added an entry to my package's changelog.yml file.
  • I have verified that Kibana version constraints are current according to guidelines.

How to test this PR locally

  • elastic-package build
  • elastic-package stack up -v -d --services package-registry

Related issues

Screenshots

@agithomas
Copy link
Contributor Author

Index mappings
{
  "mappings": {
    "_meta": {
      "managed_by": "fleet",
      "managed": true,
      "package": {
        "name": "etcd"
      }
    },
    "_data_stream_timestamp": {
      "enabled": true
    },
    "dynamic_templates": [
      {
        "strings_as_keyword": {
          "match_mapping_type": "string",
          "mapping": {
            "ignore_above": 1024,
            "type": "keyword"
          }
        }
      }
    ],
    "date_detection": false,
    "properties": {
      "@timestamp": {
        "type": "date"
      },
      "agent": {
        "properties": {
          "ephemeral_id": {
            "type": "keyword",
            "ignore_above": 1024
          },
          "id": {
            "type": "keyword",
            "ignore_above": 1024
          },
          "name": {
            "type": "keyword",
            "ignore_above": 1024
          },
          "type": {
            "type": "keyword",
            "ignore_above": 1024
          },
          "version": {
            "type": "keyword",
            "ignore_above": 1024
          }
        }
      },
      "cloud": {
        "properties": {
          "account": {
            "properties": {
              "id": {
                "type": "keyword",
                "ignore_above": 1024
              }
            }
          },
          "availability_zone": {
            "type": "keyword",
            "ignore_above": 1024
          },
          "instance": {
            "properties": {
              "id": {
                "type": "keyword",
                "ignore_above": 1024
              },
              "name": {
                "type": "keyword",
                "ignore_above": 1024
              }
            }
          },
          "machine": {
            "properties": {
              "type": {
                "type": "keyword",
                "ignore_above": 1024
              }
            }
          },
          "project": {
            "properties": {
              "id": {
                "type": "keyword",
                "ignore_above": 1024
              }
            }
          },
          "provider": {
            "type": "keyword",
            "ignore_above": 1024
          },
          "region": {
            "type": "keyword",
            "ignore_above": 1024
          },
          "service": {
            "properties": {
              "name": {
                "type": "keyword",
                "ignore_above": 1024
              }
            }
          }
        }
      },
      "data_stream": {
        "properties": {
          "dataset": {
            "type": "constant_keyword",
            "value": "etcd.self"
          },
          "namespace": {
            "type": "constant_keyword",
            "value": "default"
          },
          "type": {
            "type": "constant_keyword",
            "value": "metrics"
          }
        }
      },
      "ecs": {
        "properties": {
          "version": {
            "type": "keyword",
            "ignore_above": 1024
          }
        }
      },
      "elastic_agent": {
        "properties": {
          "id": {
            "type": "keyword",
            "ignore_above": 1024
          },
          "snapshot": {
            "type": "boolean"
          },
          "version": {
            "type": "keyword",
            "ignore_above": 1024
          }
        }
      },
      "etcd": {
        "properties": {
          "api_version": {
            "type": "keyword",
            "ignore_above": 1024
          },
          "self": {
            "properties": {
              "id": {
                "type": "keyword",
                "ignore_above": 1024
              },
              "leaderinfo": {
                "properties": {
                  "leader": {
                    "type": "keyword",
                    "ignore_above": 1024
                  },
                  "start_time": {
                    "type": "keyword",
                    "ignore_above": 1024
                  },
                  "uptime": {
                    "type": "keyword",
                    "ignore_above": 1024
                  }
                }
              },
              "name": {
                "type": "keyword",
                "ignore_above": 1024
              },
              "recv": {
                "properties": {
                  "append_request": {
                    "properties": {
                      "count": {
                        "type": "long",
                        "time_series_metric": "counter"
                      }
                    }
                  },
                  "bandwidth_rate": {
                    "type": "scaled_float",
                    "scaling_factor": 1000,
                    "time_series_metric": "gauge"
                  },
                  "pkg_rate": {
                    "type": "scaled_float",
                    "scaling_factor": 1000,
                    "time_series_metric": "gauge"
                  }
                }
              },
              "send": {
                "properties": {
                  "append_request": {
                    "properties": {
                      "count": {
                        "type": "long",
                        "time_series_metric": "counter"
                      }
                    }
                  },
                  "bandwidth_rate": {
                    "type": "scaled_float",
                    "scaling_factor": 1000,
                    "time_series_metric": "gauge"
                  },
                  "pkg_rate": {
                    "type": "scaled_float",
                    "scaling_factor": 1000,
                    "time_series_metric": "gauge"
                  }
                }
              },
              "start_time": {
                "type": "keyword",
                "ignore_above": 1024
              },
              "state": {
                "type": "keyword",
                "ignore_above": 1024
              }
            }
          }
        }
      },
      "event": {
        "properties": {
          "agent_id_status": {
            "type": "keyword",
            "ignore_above": 1024
          },
          "dataset": {
            "type": "constant_keyword",
            "value": "etcd.self"
          },
          "duration": {
            "type": "long"
          },
          "ingested": {
            "type": "date",
            "format": "strict_date_time_no_millis||strict_date_optional_time||epoch_millis"
          },
          "module": {
            "type": "constant_keyword",
            "value": "etcd"
          }
        }
      },
      "host": {
        "properties": {
          "architecture": {
            "type": "keyword",
            "ignore_above": 1024
          },
          "containerized": {
            "type": "boolean"
          },
          "hostname": {
            "type": "keyword",
            "ignore_above": 1024
          },
          "id": {
            "type": "keyword",
            "ignore_above": 1024
          },
          "ip": {
            "type": "ip"
          },
          "mac": {
            "type": "keyword",
            "ignore_above": 1024
          },
          "name": {
            "type": "keyword",
            "ignore_above": 1024
          },
          "os": {
            "properties": {
              "codename": {
                "type": "keyword",
                "ignore_above": 1024
              },
              "family": {
                "type": "keyword",
                "ignore_above": 1024
              },
              "kernel": {
                "type": "keyword",
                "ignore_above": 1024
              },
              "name": {
                "type": "keyword",
                "ignore_above": 1024
              },
              "platform": {
                "type": "keyword",
                "ignore_above": 1024
              },
              "type": {
                "type": "keyword",
                "ignore_above": 1024
              },
              "version": {
                "type": "keyword",
                "ignore_above": 1024
              }
            }
          }
        }
      },
      "metricset": {
        "properties": {
          "name": {
            "type": "keyword",
            "ignore_above": 1024
          },
          "period": {
            "type": "long"
          }
        }
      },
      "service": {
        "properties": {
          "address": {
            "type": "keyword",
            "time_series_dimension": true
          },
          "type": {
            "type": "keyword",
            "ignore_above": 1024
          }
        }
      }
    }
  }
}

@agithomas agithomas mentioned this pull request Nov 6, 2023
6 tasks
@agithomas agithomas changed the title Add metric type mapping to self datastream [etcd] Add metric type mapping to self datastream Nov 6, 2023
@agithomas agithomas marked this pull request as ready for review November 6, 2023 11:26
@agithomas agithomas requested a review from a team as a code owner November 6, 2023 11:26
@elasticmachine
Copy link

elasticmachine commented Nov 6, 2023

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS
Pipeline View Test View Changes Artifacts preview preview

Expand to view the summary

Build stats

  • Start Time: 2023-11-06T11:15:11.331+0000

  • Duration: 17 min 16 sec

Test stats 🧪

Test Results
Failed 0
Passed 12
Skipped 0
Total 12

🤖 GitHub comments

Expand to view the GitHub comments

To re-run your PR in the CI, just comment with:

  • /test : Re-trigger the build.

@elasticmachine
Copy link

🌐 Coverage report

Name Metrics % (covered/total) Diff
Packages 100.0% (0/0) 💚
Files 100.0% (0/0) 💚
Classes 100.0% (0/0) 💚
Methods 64.286% (9/14) 👎 -35.714
Lines 100.0% (0/0) 💚 6.609
Conditionals 100.0% (0/0) 💚

@botelastic
Copy link

botelastic bot commented Dec 11, 2023

Hi! We just realized that we haven't looked into this PR in a while. We're sorry! We're labeling this issue as Stale to make it hit our filters and make sure we get back to it as soon as possible. In the meantime, it'd be extremely helpful if you could take a look at it as well and confirm its relevance. A simple comment with a nice emoji will be enough :+1. Thank you for your contribution!

@botelastic botelastic bot added the Stalled label Dec 11, 2023
@mrodm
Copy link
Contributor

mrodm commented Dec 21, 2023

Hi @agithomas, please update your branch with the latest contents from main branch. There was an important PR merged updating the CI pipelines. Thanks!

@botelastic botelastic bot removed the Stalled label Dec 21, 2023
@botelastic
Copy link

botelastic bot commented Jan 20, 2024

Hi! We just realized that we haven't looked into this PR in a while. We're sorry! We're labeling this issue as Stale to make it hit our filters and make sure we get back to it as soon as possible. In the meantime, it'd be extremely helpful if you could take a look at it as well and confirm its relevance. A simple comment with a nice emoji will be enough :+1. Thank you for your contribution!

@botelastic botelastic bot added the Stalled label Jan 20, 2024
@botelastic
Copy link

botelastic bot commented Feb 19, 2024

Hi! This PR has been stale for a while and we're going to close it as part of our cleanup procedure. We appreciate your contribution and would like to apologize if we have not been able to review it, due to the current heavy load of the team. Feel free to re-open this PR if you think it should stay open and is worth rebasing. Thank you for your contribution!

@botelastic botelastic bot closed this Feb 19, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants