Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

prometheus: Add a metrics family for Khepri's Ra counters #12190

Draft
wants to merge 1 commit into
base: main
Choose a base branch
from
Draft

prometheus: Add a metrics family for Khepri's Ra counters #12190

wants to merge 1 commit into from

Conversation

the-mikedavis
Copy link
Member

@the-mikedavis the-mikedavis commented Aug 30, 2024 •
edited
Loading

This exposes all of the metrics that Ra collects in counters for the node's Khepri server under a new metric family in the "detailed" registry.

Example output... 
$ curl -s "localhost:15692/metrics/detailed?family=khepri"
# TYPE rabbitmq_detailed_khepri_reserved_2 counter
# HELP rabbitmq_detailed_khepri_reserved_2 Reserved counter
rabbitmq_detailed_khepri_reserved_2 0
# TYPE rabbitmq_detailed_khepri_aux_commands counter
# HELP rabbitmq_detailed_khepri_aux_commands Total number of aux commands received
rabbitmq_detailed_khepri_aux_commands 0
# TYPE rabbitmq_detailed_khepri_checkpoints_promoted counter
# HELP rabbitmq_detailed_khepri_checkpoints_promoted Number of checkpoints promoted to snapshots
rabbitmq_detailed_khepri_checkpoints_promoted 0
# TYPE rabbitmq_detailed_khepri_command_flushes counter
# HELP rabbitmq_detailed_khepri_command_flushes Total number of low priority command batches written
rabbitmq_detailed_khepri_command_flushes 1
# TYPE rabbitmq_detailed_khepri_write_resends counter
# HELP rabbitmq_detailed_khepri_write_resends Total number of write resends
rabbitmq_detailed_khepri_write_resends 0
# TYPE rabbitmq_detailed_khepri_effective_machine_version gauge
# HELP rabbitmq_detailed_khepri_effective_machine_version The current effective version number of the machine.
rabbitmq_detailed_khepri_effective_machine_version 1
# TYPE rabbitmq_detailed_khepri_checkpoints counter
# HELP rabbitmq_detailed_khepri_checkpoints The number of checkpoint effects executed
rabbitmq_detailed_khepri_checkpoints 0
# TYPE rabbitmq_detailed_khepri_read_segment counter
# HELP rabbitmq_detailed_khepri_read_segment Total number of read segments
rabbitmq_detailed_khepri_read_segment 107
# TYPE rabbitmq_detailed_khepri_local_queries counter
# HELP rabbitmq_detailed_khepri_local_queries Total number of local queries
rabbitmq_detailed_khepri_local_queries 22
# TYPE rabbitmq_detailed_khepri_read_open_mem_tbl counter
# HELP rabbitmq_detailed_khepri_read_open_mem_tbl Total number of opened memory tables
rabbitmq_detailed_khepri_read_open_mem_tbl 0
# TYPE rabbitmq_detailed_khepri_rpcs_sent counter
# HELP rabbitmq_detailed_khepri_rpcs_sent Total number of rpcs, incl append_entries_rpcs
rabbitmq_detailed_khepri_rpcs_sent 0
# TYPE rabbitmq_detailed_khepri_dropped_sends counter
# HELP rabbitmq_detailed_khepri_dropped_sends Total number of message sends that return noconnect or nosuspend are dropped
rabbitmq_detailed_khepri_dropped_sends 0
# TYPE rabbitmq_detailed_khepri_forced_gcs counter
# HELP rabbitmq_detailed_khepri_forced_gcs Number of forced garbage collection runs
rabbitmq_detailed_khepri_forced_gcs 1
# TYPE rabbitmq_detailed_khepri_commands counter
# HELP rabbitmq_detailed_khepri_commands Total number of commands received by a leader
rabbitmq_detailed_khepri_commands 29
# TYPE rabbitmq_detailed_khepri_consistent_queries counter
# HELP rabbitmq_detailed_khepri_consistent_queries Total number of consistent query requests
rabbitmq_detailed_khepri_consistent_queries 0
# TYPE rabbitmq_detailed_khepri_open_segments gauge
# HELP rabbitmq_detailed_khepri_open_segments Number of open segments
rabbitmq_detailed_khepri_open_segments 0
# TYPE rabbitmq_detailed_khepri_term_and_voted_for_updates counter
# HELP rabbitmq_detailed_khepri_term_and_voted_for_updates Total number of updates of term and voted for
rabbitmq_detailed_khepri_term_and_voted_for_updates 1
# TYPE rabbitmq_detailed_khepri_aer_received_follower counter
# HELP rabbitmq_detailed_khepri_aer_received_follower Total number of append entries received by a follower
rabbitmq_detailed_khepri_aer_received_follower 0
# TYPE rabbitmq_detailed_khepri_fetch_term counter
# HELP rabbitmq_detailed_khepri_fetch_term Total number of terms fetched
rabbitmq_detailed_khepri_fetch_term 1
# TYPE rabbitmq_detailed_khepri_release_cursors counter
# HELP rabbitmq_detailed_khepri_release_cursors Total number of updates of the release cursor
rabbitmq_detailed_khepri_release_cursors 0
# TYPE rabbitmq_detailed_khepri_snapshots_written counter
# HELP rabbitmq_detailed_khepri_snapshots_written Total number of snapshots written
rabbitmq_detailed_khepri_snapshots_written 0
# TYPE rabbitmq_detailed_khepri_read_closed_mem_tbl counter
# HELP rabbitmq_detailed_khepri_read_closed_mem_tbl Total number of closed memory tables
rabbitmq_detailed_khepri_read_closed_mem_tbl 0
# TYPE rabbitmq_detailed_khepri_aer_replies_fail counter
# HELP rabbitmq_detailed_khepri_aer_replies_fail Total number of failed append entries
rabbitmq_detailed_khepri_aer_replies_fail 0
# TYPE rabbitmq_detailed_khepri_commit_index counter
# HELP rabbitmq_detailed_khepri_commit_index The current commit index.
rabbitmq_detailed_khepri_commit_index 136
# TYPE rabbitmq_detailed_khepri_checkpoint_index counter
# HELP rabbitmq_detailed_khepri_checkpoint_index The current checkpoint index.
rabbitmq_detailed_khepri_checkpoint_index 0
# TYPE rabbitmq_detailed_khepri_read_ops counter
# HELP rabbitmq_detailed_khepri_read_ops Total number of read ops
rabbitmq_detailed_khepri_read_ops 30
# TYPE rabbitmq_detailed_khepri_last_written_index counter
# HELP rabbitmq_detailed_khepri_last_written_index The last fully written and fsynced index of the log.
rabbitmq_detailed_khepri_last_written_index 136
# TYPE rabbitmq_detailed_khepri_elections counter
# HELP rabbitmq_detailed_khepri_elections Total number of elections
rabbitmq_detailed_khepri_elections 1
# TYPE rabbitmq_detailed_khepri_aer_received_follower_empty counter
# HELP rabbitmq_detailed_khepri_aer_received_follower_empty Total number of empty append entries received by a follower
rabbitmq_detailed_khepri_aer_received_follower_empty 0
# TYPE rabbitmq_detailed_khepri_snapshot_installed counter
# HELP rabbitmq_detailed_khepri_snapshot_installed Total number of snapshots installed
rabbitmq_detailed_khepri_snapshot_installed 0
# TYPE rabbitmq_detailed_khepri_last_index counter
# HELP rabbitmq_detailed_khepri_last_index The last index of the log.
rabbitmq_detailed_khepri_last_index 136
# TYPE rabbitmq_detailed_khepri_msgs_sent counter
# HELP rabbitmq_detailed_khepri_msgs_sent All messages sent (except messages sent to wal)
rabbitmq_detailed_khepri_msgs_sent 0
# TYPE rabbitmq_detailed_khepri_read_cache counter
# HELP rabbitmq_detailed_khepri_read_cache Total number of cache reads
rabbitmq_detailed_khepri_read_cache 29
# TYPE rabbitmq_detailed_khepri_commit_latency gauge
# HELP rabbitmq_detailed_khepri_commit_latency Approximate time taken from an entry being written to the log until it is committed.
rabbitmq_detailed_khepri_commit_latency 0
# TYPE rabbitmq_detailed_khepri_write_ops counter
# HELP rabbitmq_detailed_khepri_write_ops Total number of write ops
rabbitmq_detailed_khepri_write_ops 29
# TYPE rabbitmq_detailed_khepri_aer_replies_success counter
# HELP rabbitmq_detailed_khepri_aer_replies_success Total number of successful append entries received
rabbitmq_detailed_khepri_aer_replies_success 0
# TYPE rabbitmq_detailed_khepri_pre_vote_elections counter
# HELP rabbitmq_detailed_khepri_pre_vote_elections Total number of pre-vote elections
rabbitmq_detailed_khepri_pre_vote_elections 1
# TYPE rabbitmq_detailed_khepri_term counter
# HELP rabbitmq_detailed_khepri_term The current term.
rabbitmq_detailed_khepri_term 8
# TYPE rabbitmq_detailed_khepri_snapshot_index counter
# HELP rabbitmq_detailed_khepri_snapshot_index The current snapshot index.
rabbitmq_detailed_khepri_snapshot_index -1
# TYPE rabbitmq_detailed_khepri_reserved_1 counter
# HELP rabbitmq_detailed_khepri_reserved_1 Reserved counter
rabbitmq_detailed_khepri_reserved_1 0
# TYPE rabbitmq_detailed_khepri_checkpoint_bytes_written counter
# HELP rabbitmq_detailed_khepri_checkpoint_bytes_written Number of checkpoint bytes written
rabbitmq_detailed_khepri_checkpoint_bytes_written 0
# TYPE rabbitmq_detailed_khepri_send_msg_effects_sent counter
# HELP rabbitmq_detailed_khepri_send_msg_effects_sent Total number of send_msg effects executed
rabbitmq_detailed_khepri_send_msg_effects_sent 0
# TYPE rabbitmq_detailed_khepri_checkpoints_written counter
# HELP rabbitmq_detailed_khepri_checkpoints_written Total number of checkpoints written
rabbitmq_detailed_khepri_checkpoints_written 0
# TYPE rabbitmq_detailed_khepri_last_applied gauge
# HELP rabbitmq_detailed_khepri_last_applied The last applied index. Can go backwards if a ra server is restarted.
rabbitmq_detailed_khepri_last_applied 136
# TYPE rabbitmq_detailed_khepri_snapshots_sent counter
# HELP rabbitmq_detailed_khepri_snapshots_sent Total number of snapshots sent
rabbitmq_detailed_khepri_snapshots_sent 0
# TYPE rabbitmq_detailed_khepri_invalid_reply_mode_commands counter
# HELP rabbitmq_detailed_khepri_invalid_reply_mode_commands Total number of commands received with an invalid reply-mode
rabbitmq_detailed_khepri_invalid_reply_mode_commands 0
# TYPE rabbitmq_detailed_khepri_snapshot_bytes_written counter
# HELP rabbitmq_detailed_khepri_snapshot_bytes_written Number of snapshot bytes written (not installed)
rabbitmq_detailed_khepri_snapshot_bytes_written 0
# TYPE rabbitmq_build_info untyped
# HELP rabbitmq_build_info RabbitMQ & Erlang/OTP version info
rabbitmq_build_info{rabbitmq_version="4.1.0+alpha.178.g443be4e.dirty",prometheus_plugin_version="4.1.0+alpha.178.g443be4e.dirty",prometheus_client_version="4.11.0",erlang_version="26.2.1"} 1
# TYPE rabbitmq_identity_info untyped
# HELP rabbitmq_identity_info RabbitMQ node & cluster identity info
rabbitmq_identity_info{rabbitmq_node="rabbit@mango2",rabbitmq_cluster="rabbit@mango2",rabbitmq_cluster_permanent_id="rabbitmq-cluster-id-QuIiiEzWDeP4iD8i4VZkBw"} 1
# TYPE telemetry_scrape_encoded_size_bytes summary
# HELP telemetry_scrape_encoded_size_bytes Scrape size, encoded
# TYPE telemetry_scrape_size_bytes summary
# HELP telemetry_scrape_size_bytes Scrape size, not encoded
telemetry_scrape_size_bytes_count{registry="detailed",content_type="text/plain; version=0.0.4"} 1
telemetry_scrape_size_bytes_sum{registry="detailed",content_type="text/plain; version=0.0.4"} 9685
# TYPE telemetry_scrape_duration_seconds summary
# HELP telemetry_scrape_duration_seconds Scrape duration
telemetry_scrape_duration_seconds_count{registry="detailed",content_type="text/plain; version=0.0.4"} 1
telemetry_scrape_duration_seconds_sum{registry="detailed",content_type="text/plain; version=0.0.4"} 0.005331059

Closes #12142

@the-mikedavis
prometheus: Add a metrics family for Khepri's Ra counters
443be4e 
This exposes all of the metrics that Ra collects in counters for the
node's Khepri server under a new metric family in the "detailed"
registry.
@the-mikedavis the-mikedavis self-assigned this Aug 30, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@kjnilsson kjnilsson Awaiting requested review from kjnilsson

Assignees

@the-mikedavis the-mikedavis

Labels
backport-v4.0.x
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Khepri metrics are showing up without any tags (v4 beta 5)
1 participant
@the-mikedavis