Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Addon: expose /metrics endpoints for Prometheus #49

Open
wants to merge 27 commits into
base: master
Choose a base branch
from
Open

Addon: expose /metrics endpoints for Prometheus #49

wants to merge 27 commits into from

Commits on Nov 3, 2017

  1. Exposes /metrics endpoints for Prometheus scraping 

    This reverts commit 22a314a.
    @solsson
    solsson committed Nov 3, 2017
    Configuration menu
    Copy the full SHA
    41cdfd6 View commit details
    Browse the repository at this point in the history

  2. Adds pod that can be used to estimate resource limits 

    for jmx containers in kafka and zoo pods
    @solsson
    solsson committed Nov 3, 2017
    Configuration menu
    Copy the full SHA
    ffb89dd View commit details
    Browse the repository at this point in the history

  3. CPU limit on metrics export won't actually save any cycles 

    It'll just make the requests slower.
Dreadfully slow on Minikube (>30s even when limit is increased to 100m).
    @solsson
    solsson committed Nov 3, 2017
    Configuration menu
    Copy the full SHA
    51bbedb View commit details
    Browse the repository at this point in the history

  4. The test that caught the performance problem

    @solsson
    solsson committed Nov 3, 2017
    Configuration menu
    Copy the full SHA
    8ec2045 View commit details
    Browse the repository at this point in the history

  5. Demonstrates OOMKilled with current resource limits

    @solsson
    solsson committed Nov 3, 2017
    Configuration menu
    Copy the full SHA
    11feb28 View commit details
    Browse the repository at this point in the history

  6. Same base image as kafka and latest exporter source

    @solsson
    solsson committed Nov 3, 2017
    Configuration menu
    Copy the full SHA
    02c4b3e View commit details
    Browse the repository at this point in the history

  7. zoo is fast now, <0.02s compared to >1s for the others

    @solsson
    solsson committed Nov 3, 2017
    Configuration menu
    Copy the full SHA
    a563220 View commit details
    Browse the repository at this point in the history

  8. The 1s response time from kafka might be due to ... 

    that unlike zoo pods it actually exposes interesting data
    @solsson
    solsson committed Nov 3, 2017
    Configuration menu
    Copy the full SHA
    e8a61e7 View commit details
    Browse the repository at this point in the history

  9. interesting

    @solsson
    solsson committed Nov 3, 2017
    Configuration menu
    Copy the full SHA
    6d4ffc3 View commit details
    Browse the repository at this point in the history

  10. Adapts to Java 8+, but still guessing the numbers

    @solsson
    solsson committed Nov 3, 2017
    Configuration menu
    Copy the full SHA
    72cfc77 View commit details
    Browse the repository at this point in the history

  11. Don't touch Xss as it has <1MB defaults according to docs

    @solsson
    solsson committed Nov 3, 2017
    Configuration menu
    Copy the full SHA
    2b822b1 View commit details
    Browse the repository at this point in the history

  12. Let's focus on the two numbers that seem to matter

    @solsson
    solsson committed Nov 3, 2017
    Configuration menu
    Copy the full SHA
    6d3b935 View commit details
    Browse the repository at this point in the history

  13. Endpont works again, with similar scrape times as Xmx=80m without Met… 

    …aspace limit
    @solsson
    solsson committed Nov 3, 2017
    Configuration menu
    Copy the full SHA
    a4b51f4 View commit details
    Browse the repository at this point in the history

  14. Reverse order of containers to benefit from "Defaulting container nam… 

    …e to"
    @solsson
    solsson committed Nov 3, 2017
    Configuration menu
    Copy the full SHA
    8dbd7a1 View commit details
    Browse the repository at this point in the history

  15. Uses prometheus/jmx_exporter parent-0.10 tag

    @solsson
    solsson committed Nov 3, 2017
    Configuration menu
    Copy the full SHA
    d7d5044 View commit details
    Browse the repository at this point in the history

  16. Had 10 OOMKilled/hour with 100Mi so let's increase request, 

    and with 150Mi limit I got zero restarts in 48 hours.
    @solsson
    solsson committed Nov 3, 2017
    Configuration menu
    Copy the full SHA
    b6c85eb View commit details
    Browse the repository at this point in the history

  17. Uses JMX config from config map, so we can experiment

    @solsson
    solsson committed Nov 3, 2017
    Configuration menu
    Copy the full SHA
    db52a3c View commit details
    Browse the repository at this point in the history

  18. Scrape less, and improve scrape time further ... 

    through ssl=false and whitelist.

Thanks to @yacut, see #49
    @solsson
    solsson committed Nov 3, 2017
    Configuration menu
    Copy the full SHA
    37e58e9 View commit details
    Browse the repository at this point in the history

  19. For performance, again thanks to @yacut #49

    @solsson
    solsson committed Nov 3, 2017
    Configuration menu
    Copy the full SHA
    d4b95d2 View commit details
    Browse the repository at this point in the history

  20. Gets you JVM metrics from zoo, lots and lots of it

    @solsson
    solsson committed Nov 3, 2017
    Configuration menu
    Copy the full SHA
    0994950 View commit details
    Browse the repository at this point in the history

  21. Still not getting anything zookeeper-specific

    @solsson
    solsson committed Nov 3, 2017
    Configuration menu
    Copy the full SHA
    e35d077 View commit details
    Browse the repository at this point in the history

  22. Adds directives from kafka's rules, now for pzoo too. 

    But before this, how did the metrics container know which port to connect to?
    @solsson
    solsson committed Nov 3, 2017
    Configuration menu
    Copy the full SHA
    42d1b1a View commit details
    Browse the repository at this point in the history

  23. Zookeeper metrics conf contributed by @yacut #61

    @solsson
    solsson committed Nov 3, 2017
    Configuration menu
    Copy the full SHA
    253633f View commit details
    Browse the repository at this point in the history

  24. Merge pull request #61 from Yolean/metrics-jmx-zookeeper 

    Try to get meaningful metrics from Zookeeper
    @solsson
    solsson authored Nov 3, 2017
    Configuration menu
    Copy the full SHA
    b0e6145 View commit details
    Browse the repository at this point in the history

  25. Upgrade to jmx-exporter 0.1.0

    @solsson
    solsson committed Nov 3, 2017
    Configuration menu
    Copy the full SHA
    dc1c1da View commit details
    Browse the repository at this point in the history

  26. Upgrade test also to jmx-exporter 0.1.0

    @solsson
    solsson committed Nov 3, 2017
    Configuration menu
    Copy the full SHA
    4c35576 View commit details
    Browse the repository at this point in the history

  27. Adapts test instructions to debian based jre image

    @solsson
    solsson committed Nov 3, 2017
    Configuration menu
    Copy the full SHA
    6a26cf3 View commit details
    Browse the repository at this point in the history