From eaf4c184ca71f2d4e2a1b867abc3f0220cb3817f Mon Sep 17 00:00:00 2001 From: Armin Ronacher Date: Thu, 2 Feb 2023 17:17:23 +0100 Subject: [PATCH 1/3] Added notes on metrics cardinality --- src/docs/services/metrics.mdx | 15 +++++++++++++++ 1 file changed, 15 insertions(+) diff --git a/src/docs/services/metrics.mdx b/src/docs/services/metrics.mdx index cc98874392..a4f2d15c8b 100644 --- a/src/docs/services/metrics.mdx +++ b/src/docs/services/metrics.mdx @@ -83,3 +83,18 @@ LOGGING['handlers']['console:metrics'] = { 'formatter': 'metrics', } ``` + +## Notes on Metrics + +When emitting metrics you need to be careful about cardinality. Particularly for +metrics that are supposed to be sent to Datadog, at our scale there can be large +server bills heading our way if a high cardinality tag is emitted. Examples for +high cardinality tags are: + +* `event_id`, `request_id` etc.: these are unique values, absolutely pointless and + if one were to emit them at out scale: very, very expensive. +* `project_id`, `org_id`: for common operations that happen across the entire user + base: still a very bad idea. Maybe they are acceptable for very rare circumstances + or things that only happen for a small segment of the user base (eg: when you are + tracking a beta feature with a small user base, but even in that case probably + a horrible idea). From 3d70fcb8178a0f8f98d6f1c1b0a1f5c6745b95e0 Mon Sep 17 00:00:00 2001 From: Armin Ronacher Date: Thu, 2 Feb 2023 17:22:42 +0100 Subject: [PATCH 2/3] Added acceptable tags --- src/docs/services/metrics.mdx | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/src/docs/services/metrics.mdx b/src/docs/services/metrics.mdx index a4f2d15c8b..4b067ad580 100644 --- a/src/docs/services/metrics.mdx +++ b/src/docs/services/metrics.mdx @@ -98,3 +98,9 @@ high cardinality tags are: or things that only happen for a small segment of the user base (eg: when you are tracking a beta feature with a small user base, but even in that case probably a horrible idea). + +Probably acceptable cardinality: + +* `platform` as in SDK platform: there is a finite number of them. +* `status` as in HTTP status code: not great, but we expect a finite number of them. +* `task_name`, `endpoint` etc. From ab49a8d8e55fdeeb1322512b4a0d430754267751 Mon Sep 17 00:00:00 2001 From: Armin Ronacher Date: Wed, 25 Oct 2023 13:49:09 +0200 Subject: [PATCH 3/3] Update src/docs/services/metrics.mdx Co-authored-by: Michael Warkentin --- src/docs/services/metrics.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/docs/services/metrics.mdx b/src/docs/services/metrics.mdx index 4b067ad580..e5c9e769f8 100644 --- a/src/docs/services/metrics.mdx +++ b/src/docs/services/metrics.mdx @@ -92,7 +92,7 @@ server bills heading our way if a high cardinality tag is emitted. Examples for high cardinality tags are: * `event_id`, `request_id` etc.: these are unique values, absolutely pointless and - if one were to emit them at out scale: very, very expensive. + if one were to emit them at our scale: very, very expensive. * `project_id`, `org_id`: for common operations that happen across the entire user base: still a very bad idea. Maybe they are acceptable for very rare circumstances or things that only happen for a small segment of the user base (eg: when you are