-
Notifications
You must be signed in to change notification settings - Fork 22
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add support for Container insights on Windows #180
Add support for Container insights on Windows #180
Conversation
* Add pod level metric collection for Windows This PR defines code structure for metric provider which works on Windows. 1. Changed receiver.go in awscontainerinsights to run for Windows with metric provider. 2. Added summary API in kubeletclient 3. Add kubeletProvider to return metrics at different levels i.e. pod, contianer, node. 4. Updated hostInfo providers to run for Windows. 5. Updated ebsVolume Info provider to run for Windows. 1. Define correct ebsVolume Info provider for Windows 2. Change logic around k8s leader election to run for Windows # Conflicts: # receiver/awscontainerinsightreceiver/internal/cadvisor/extractors/cpu_extractor.go # receiver/awscontainerinsightreceiver/internal/cadvisor/extractors/diskio_extractor.go # receiver/awscontainerinsightreceiver/internal/cadvisor/extractors/extractor.go # receiver/awscontainerinsightreceiver/internal/cadvisor/extractors/fs_extractor.go # receiver/awscontainerinsightreceiver/internal/cadvisor/extractors/mem_extractor.go # receiver/awscontainerinsightreceiver/internal/cadvisor/extractors/net_extractor.go # receiver/awscontainerinsightreceiver/receiver.go
* Add CPU extractors from kubelet summary API 1. Make cadvisor helper func's public to be used in k8swindows extractor 2. Add CPU extractor and add utilization fields 3. Add unit test for CPU extractor. 4. Add unit test data for kubelet summary API 5. Add helper func to convert Pod and Node summary stats to RawMetric * Refactor code 1. Changed cExtractor to cextractors 2. Add nil checks to avoid panic during pointer deferences * Refactored code 1. Added missing HasValue func in extractors 2. Replaced cextractor with cExtractor 3. Corrected extractorhelper name with missing characters # Conflicts: # receiver/awscontainerinsightreceiver/internal/cadvisor/extractors/cpu_extractor.go # receiver/awscontainerinsightreceiver/internal/cadvisor/extractors/diskio_extractor.go # receiver/awscontainerinsightreceiver/internal/cadvisor/extractors/extractor.go # receiver/awscontainerinsightreceiver/internal/cadvisor/extractors/mem_extractor.go # receiver/awscontainerinsightreceiver/internal/k8swindows/kubelet.go
* Add memory extractor at pod and node level 1. Added memory extractor at pod and node level 2. Add unit tests for memory extractor 3. use cpu and memory extractor for k8s windows * Fix adding tags to collected metrics from extractors # Conflicts: # receiver/awscontainerinsightreceiver/internal/k8swindows/extractors/cpu_extractor_test.go # receiver/awscontainerinsightreceiver/internal/k8swindows/k8swindows.go # receiver/awscontainerinsightreceiver/internal/k8swindows/kubelet.go
* Define structs for CPU and Memory and stats 1. Create new structs to represent CPU and memory stats in RawMetric. This removes RawMetric dependency on Kubelet CPU and memory stats. 2. Refactor existing RawMetric struct to use new CPU and memory stats. 3. Add more unit tests for extractorhelpers * Refactor: Remove parameters passing by reference in extractors * Remove extra comments in error # Conflicts: # receiver/awscontainerinsightreceiver/internal/k8swindows/extractors/cpu_extractor.go # receiver/awscontainerinsightreceiver/internal/k8swindows/extractors/extractor.go # receiver/awscontainerinsightreceiver/internal/k8swindows/kubelet.go
…tributing#150) * Add container level CPU and memory metrics collection 1. Add metric collection at container level 2. Refactor existing kubelet code to make it unit testable. 3. Add units for kubelet to test pod, node and container level metric collection. * Refactor: rename port -> hostPort and summaryProvider -> kubeletSummaryProvider * Refactor: Change naming in kubelet 2. Remove pas by reference to extractors * Refactored: address naming inconsistencies * Refactor: remove extra GetClient * Fix merge conflict # Conflicts: # receiver/awscontainerinsightreceiver/internal/k8swindows/extractors/extractor.go # receiver/awscontainerinsightreceiver/internal/k8swindows/kubelet.go
…#151) * Add storage metrics for container and node level 1. Add storage extractors for container and node level 2. Add metric source for Windows metric collection 3. Refactor metric source for cadvisor 4. Add os label for windows * Address chad's and pooja's comments * Refactor: Address review comments * Refactor: remove extra add source func
# Conflicts: # receiver/awscontainerinsightreceiver/internal/k8swindows/k8swindows.go
…iner (amazon-contributing#153) * Enable awscontainerinsights receiver to run inside Host Process container 1. Added workaround to fix ServiceAccount token and cert path for kubelet account inside HPC. 2. Added workarond to fix above issue in k8s clientset. * Addressed chad's comments * Addrssed pooja's comments * Fix go.mod
…ntributing#154) * Add storage metrics for container and node level 1. Add storage extractors for container and node level 2. Add metric source for Windows metric collection 3. Refactor metric source for cadvisor 4. Add os label for windows # Conflicts: # internal/aws/containerinsight/const.go # receiver/awscontainerinsightreceiver/internal/k8swindows/extractors/extractor.go # receiver/awscontainerinsightreceiver/internal/k8swindows/extractors/extractorhelpers.go # receiver/awscontainerinsightreceiver/internal/k8swindows/extractors/fs_extractor.go # receiver/awscontainerinsightreceiver/internal/k8swindows/k8swindows.go # receiver/awscontainerinsightreceiver/internal/k8swindows/kubelet/kubelet.go # receiver/awscontainerinsightreceiver/internal/k8swindows/kubelet/kubelet_test.go # receiver/awscontainerinsightreceiver/internal/stores/utils.go # receiver/awscontainerinsightreceiver/internal/stores/utils_test.go * Add hcsshim API as source 1. Added hcshim API as alternative to kubelet for networking stats 2. Added unit tests for hcsshim provider 3. Add new fields to network extractor * Fix e2e test for docker build 1. Ran `go mod tidy` inside cmd/otelcontribcol # Conflicts: # receiver/awscontainerinsightreceiver/go.mod # receiver/awscontainerinsightreceiver/go.sum # Conflicts: # receiver/awscontainerinsightreceiver/internal/k8swindows/extractors/extractor.go # receiver/awscontainerinsightreceiver/internal/k8swindows/kubelet/kubelet.go
…on-contributing#156) # Conflicts: # receiver/awscontainerinsightreceiver/internal/k8swindows/hcsshim/hcsshim.go
…#161) * 1. Fix CPU cores in Windows 2. Sum fs usage field * fix fs type * fix fs issue * fix fs issue
* Added readme for awscontainerinsights for Windows * Removed Linux metrics from Windows metrics * Added back documentation removed by mistake
Removed completed todos # Conflicts: # receiver/awscontainerinsightreceiver/internal/k8swindows/kubelet/kubelet.go
1. Add windows build tag to fix building cw agent on Windows 2. Downgrade internal/aws/containerinsight from 0.92 0.89 3. Separate unit tests in util.go specific for Windows 4. Fix util unit tests applicable for Windows 5. Fix goporto issue 6. Fix lint issue 7. Fix regression in unit tests caused due to rebasing mainlin
839747f
to
c1bda62
Compare
|
||
var metricsExtractors = []extractors.MetricExtractor{} | ||
|
||
func New(logger *zap.Logger, decorator *stores.K8sDecorator, hostInfo host.Info) (*K8sWindows, error) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There are no tests for this file methods
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I added tests for this file. Unit test coverage is only limited to initializing k8sWindows object.
Adding unit test for GetMetrics will require some refactoring. Can follow up with adding unit test ater release.
1. Add unit test for kubelet client on Windows 2. Run DCGM scrapper only for CW agent on Linux
Codecov ReportAttention: Patch coverage is
❗ Your organization needs to install the Codecov GitHub app to enable full functionality. Additional details and impacted files@@ Coverage Diff @@
## aws-cwa-dev #180 +/- ##
==============================================
Coverage ? 82.44%
==============================================
Files ? 1772
Lines ? 165884
Branches ? 0
==============================================
Hits ? 136767
Misses ? 25131
Partials ? 3986 ☔ View full report in Codecov by Sentry. |
1. Fix k8s windows unit tests 2. Disable ebs volume unit test for Windows 3. Separate out node Volume unit tests for Windows
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
{
"Name": "node_cpu_limit"
},
and
{
"Name": "node_cpu_usage_total"
},
metrics are missing units. May I know why didn't we include the statsd and emf integ testing for windows EKS?Have you added additional unit tests from yesterday discussion for Extractor helpers class?
a7e0c68
into
amazon-contributing:aws-cwa-dev
The following metrics are reported in unit of millicores, and cloudwatch doesn't support it. |
Description:
This PR adds CI on Windows feature in awscontainerinsightsreceiver. This PR merges dev branch aws-ci-windows into aws-ci-dev
Ex. Adding a feature - Explain what this achieves.-->
This PR adds CI on Windows feature in awscontainerinsightsreceiver.
Testing:
https://github.com/aws/private-amazon-cloudwatch-agent-staging/actions/runs/8175470234/job/22352686273
Documentation:
This commit adds documentation - #166