Skip to content

Conversation

@mkuratczyk
Copy link
Contributor

This allows to differentiate between metrics with the same name, coming from the /metrics/ endpoint and those coming from /metrics/per-object (or from /metrics with prometheus.return_per_object_metrics = true).

The lack of such differentiation can easily lead (and leads in our Grafana dashboard) to some values being doubled if both endpoints are scraped (which is a reasonable thing to do - collect aggregated metrics often for a high-level overview and per-object metrics less often not to overload the system). This leads to metric series like these:

# aggregated metric
rabbitmq_queue_messages_ready 53
# per-object metrics
rabbitmq_queue_messages_ready{vhost="/",queue="qq-2"} 20
rabbitmq_queue_messages_ready{vhost="/",queue="qq-3"} 20
rabbitmq_queue_messages_ready{vhost="/",queue="qq-1"} 13

a PromQL query such as sum(rabbitmq_queue_messages_ready) returns 106, even though there are only 53 messages ready. With this change, you can write a query such as rabbitmq_queue_messages_ready * on (instance, job) rabbitmq_identity_info{rabbitmq_endpoint="aggregated"} so that per-object values are ignored

@michaelklishin michaelklishin merged commit 9f4853c into main Feb 11, 2025
268 checks passed
@michaelklishin michaelklishin deleted the identity-metric-endpoint-label branch February 11, 2025 21:13
michaelklishin added a commit that referenced this pull request Feb 11, 2025
Add rabbitmq_endpoint label to rabbitmq_identity_info (backport #13218)
michaelklishin added a commit that referenced this pull request Feb 11, 2025
Add rabbitmq_endpoint label to rabbitmq_identity_info (backport #13218) (backport #13237)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants