From 503933efecef0f305b22896ff19df3d5f1e0d288 Mon Sep 17 00:00:00 2001 From: Luca Forstner Date: Wed, 22 Jun 2022 11:35:17 +0200 Subject: [PATCH 01/28] Add spec for Dynamic Sampling Context --- src/components/sidebar.tsx | 1 + .../performance/dynamic-sampling-context.mdx | 124 ++++++++++++++++++ src/docs/sdk/performance/trace-context.mdx | 1 + 3 files changed, 126 insertions(+) create mode 100644 src/docs/sdk/performance/dynamic-sampling-context.mdx diff --git a/src/components/sidebar.tsx b/src/components/sidebar.tsx index c0aaaba22e..ca734e669b 100644 --- a/src/components/sidebar.tsx +++ b/src/components/sidebar.tsx @@ -208,6 +208,7 @@ export default () => { Trace Contexts Span Operations + Dynamic Sampling Context diff --git a/src/docs/sdk/performance/dynamic-sampling-context.mdx b/src/docs/sdk/performance/dynamic-sampling-context.mdx new file mode 100644 index 0000000000..b2def7cde6 --- /dev/null +++ b/src/docs/sdk/performance/dynamic-sampling-context.mdx @@ -0,0 +1,124 @@ +--- +title: "Dynamic Sampling Context (Experimental)" +--- + + + +This page is under active development. +Specifications are not final and subject to change. +Anything that sounds fishy probably is. + + + +Until now, traces sampling was only done through a `sample_rate` option in the SDKs. +This has quite a few drawbacks for users of Sentry SDKs: + +- Changing the sampling rate involved either redeploying applications (which is problematic in case of applications that are not updated automatically, i.e., mobile apps or physically distributed software) or building complex systems to dynamically fetch a sampling rate. +- Sampling only happened based on a factor of randomness. + Employing sampling rules, for example, based on event parameters, is currently very complex. + While writing rules for singular **transactions** is possible, enforcing them on an entire **trace** is infeasable. + +The solution for these problems is **Dynamic Sampling**. +Dynamic Sampling allows users to configure **sampling rules** directly in the Sentry interface. Important: Sampling rules may also be applied to **entire traces**. + +## High-Level Problem Statement + +### Ingest + +Implementing Dynamic Sampling comes with challenges, especially on the ingestion side of things. +For Dynamic Sampling, we want to make sampling decisions for entire traces. +However, to keep ingestion speedy, Relay only looks at singular transactions in isolation (as opposed to looking at whole traces). +This means that we need the exact same decision basis for all transactions belonging to a trace. +In other words, all transactions of a trace need to hold all of the information to make a sampling decision, and that information needs to be the same across all transactions of the trace. +We call the information we base sampling decisions on **"Dynamic Sampling Context"** or **"DSC"**. + +### SDKs + +SDKs are responsible for propagating **Dynamic Sampling Context** across all applications that are part of a trace. +This involves: + +1. Collecting the information that makes up the DSC **xor** extracting the DSC from incoming requests. +2. Propagating DSC to downstream SDKs. +3. Sending the DSC to Sentry via an envelope. + +Because there are quite a few things to keep in mind for DSC propagation and to avoid every SDK running into the same problems, we defined a [unified propagation mechanism](#unified-propagation-mechanism) (step-by-step instructions) that all SDK implementations should be able to follow. + +## Baggage + +We chose `baggage` as the propagation mechanism for DSC. ([w3c baggage spec](https://www.w3.org/TR/baggage/)) +Baggage is a standard HTTP header with URI encoded key-value pairs. + +For the propagation of DSC, SDKs first read the DSC from the baggage header of incoming requests/messages. +To propagate DSC to downstream SDKs/services, we create a baggage header (or modify an existing one) through HTTP request instrumentation. + + + +Other vendors might also be using the `baggage` header. +If a `baggage` header already exists on an outgoing request, SDKs should aim to be good citizens by only **appending** Sentry values to the header. +In the case that another vendor added Sentry values to an outgoing request, SDKs may overwrite those values. + +SDKs must not add other vendors' baggage from incoming requests to outgoing requests. +Sentry SDKs only concern themselves with Sentry baggage. + + + +The following is an example of what a baggage header containing Dynamic Sampling Context may look like: + +``` +baggage: other-vendor-value-1=foo;bar;baz, sentry-traceid=771a43a4192642f0b136d5159a501700, sentry-publickey=49d0f7386ad645858ae85020e393bef3; sentry-userid=Am%C3%A9lie, other-vendor-value-2=foo;bar; +``` + +See the [Payloads section](#payloads) for a complete list of key-value pairs that SDKs should propagate. + +## Payloads + +Dynamic Sampling Context is propagated via a baggage header and sent to Sentry via transaction envelope headers. + +### Baggage-Header + +SDKs may set the following key-value pairs on baggage headers. +While all of these values are optional, SDKs should make their best effort to add as many of them to the baggage header as possible when starting a trace. + +- `sentry-traceid` - The original trace ID as generated by the SDK +- `sentry-publickey` - Public key as defined by the user via the DSN in the SDK options +- `sentry-release` - The release as defined by the user in the SDK options +- `sentry-environment` - The environment as defined by the user in the SDK options +- `sentry-transaction` - The name of the trace's origin transaction in unparameterized (raw) format +- `sentry-userid` - User ID as set by the user with `scope.set_user` +- `sentry-usersegment` - User segment as set by the user with `scope.set_user` +- `sentry-samplerate` - Sample rate as defined by the user in the SDK options + +SDKs must set all of the keys in the form of "`sentry-[name]`". +The delimiter "`sentry-`" acts to identify key-value pairs set by Sentry SDKs. +Additionally, we chose `[name]` to be written in "snake case" without any underscore ( `_` ) characters. This naming convention is the most language agnostic. + +### Envelope Header + +Dynamic Sampling Context is transferred to Sentry through the transaction envelope headers, keyed by `trace`. +It corresponds directly to the definition of Trace Context. + +When a transaction is reported to Sentry, the Dynamic Sampling Context must be mapped to Trace Context in the following way: + +- `sentry-release` ➝ `release` +- `sentry-environment` ➝ `environment` +- `sentry-transaction` ➝ `transaction` +- `sentry-userid` ➝ `user.id` +- `sentry-usersegment` ➝ `user.segment` +- `sentry-samplerate` ➝ `sample_rate` +- `sentry-traceid` ➝ `trace_id` +- `sentry-publickey` ➝ `public_key` + +## Unified Propagation Mechanism + +Todo: + +- incoming request algorithm +- outgoing request algorithm + +## Considerations + +Todo: + +- Why baggage and not trace context https://www.w3.org/TR/trace-context/? +- Why must baggage be immutable before the second transaction has been started? +- Why can't we just make the decision for the whole trace in Relay after the trace is complete? diff --git a/src/docs/sdk/performance/trace-context.mdx b/src/docs/sdk/performance/trace-context.mdx index 1438cc93b8..6f4276b20a 100644 --- a/src/docs/sdk/performance/trace-context.mdx +++ b/src/docs/sdk/performance/trace-context.mdx @@ -22,6 +22,7 @@ Regardless of the transport mechanism, the trace context is a JSON object with t - `trace_id` (string, required) - UUID V4 encoded as a hexadecimal sequence with no dashes (e.g. `771a43a4192642f0b136d5159a501700`) that is a sequence of 32 hexadecimal digits. This must match the trace id of the submitted transaction item. - `public_key` (string, required) - Public key from the DSN used by the SDK. It allows Sentry to sample traces spanning multiple projects, by resolving the same set of rules based on the starting project. +- `sample_rate` (number, optional) - The sample rate as defined by the user on the SDK where the trace originated. - `release` (string, optional) - The release name as specified in client options, usually: `package@x.y.z+build`. _This should match the `release` attribute of the transaction event payload_.\* - `environment` - The environment name as specified in client options, for example `staging`. _This should match the `environment` attribute of the transaction event payload_.\* - `user` (object, optional) - A subset of the scope's user context containing the following fields: From 3f1302481d4cbbb0e364fdcf2fae34d8324ad297 Mon Sep 17 00:00:00 2001 From: Luca Forstner Date: Wed, 22 Jun 2022 12:28:00 +0200 Subject: [PATCH 02/28] Add Unified Propagation Mechanism --- .../performance/dynamic-sampling-context.mdx | 40 +++++++++++++++++-- 1 file changed, 37 insertions(+), 3 deletions(-) diff --git a/src/docs/sdk/performance/dynamic-sampling-context.mdx b/src/docs/sdk/performance/dynamic-sampling-context.mdx index b2def7cde6..5d0f6e3f58 100644 --- a/src/docs/sdk/performance/dynamic-sampling-context.mdx +++ b/src/docs/sdk/performance/dynamic-sampling-context.mdx @@ -110,10 +110,44 @@ When a transaction is reported to Sentry, the Dynamic Sampling Context must be m ## Unified Propagation Mechanism -Todo: +SDKs should follow these steps for any incoming and outgoing requests (in python pseudo-code for illustrative purposes): + +```python +def collect_dynamic_sampling_context(): + # Placeholder function that collects as many values for Dynamic Sampling Context + # as possible and returns a dict + +def on_incoming_request(request): + if has_header(request, "sentry-trace") and (not has_header(request, "baggage") or not has_sentry_value_in_baggage_header(request)): + # Request comes from an old SDK which doesn't support Dynamic Sampling Context yet + # --> we don't propagate baggage for this trace + transaction.baggage_locked = true + transaction.baggage = {} + elif has_header(request, "baggage") and has_sentry_value_in_baggage_header(request): + transaction.baggage_locked = true + transaction.baggage = baggage_header_to_dict(request.headers.baggage) + +def on_outgoing_request(request): + if not transaction.baggage_locked: + transaction.baggage_locked = true + if not transaction.baggage: + transaction.baggage = {} + transaction.baggage = merge_dicts(collect_dynamic_sampling_context(), transaction.baggage) + + if has_header(request, "baggage"): + outgoing_baggage_dict = baggage_header_to_dict(request.headers.baggage) + merged_baggage_dict = merge_dicts(outgoing_baggage_dict, transaction.baggage) + merged_baggage_header = dict_to_baggage_header(merged_baggage_dict) + set_header(request, "baggage", merged_baggage_header) + else: + baggage_header = dict_to_baggage_header(transaction.baggage) + set_header(request, "baggage", baggage_header) +``` -- incoming request algorithm -- outgoing request algorithm +While there is no strict necessity for the `transaction.baggage_locked` flag yet, there is a future use case where we need it: +We might want users to be able to set Dynamic Sampling Context values themselves. +The flag becomes relevant after the first propagation, where Dynamic Sampling Context becomes immutable. +When users attempt to set DSC afterwards, our SDKs should make this operation a noop. ## Considerations From 6cdc5699e8486b30c823cefc58fd215dfe38b1a7 Mon Sep 17 00:00:00 2001 From: Luca Forstner Date: Wed, 22 Jun 2022 12:59:26 +0200 Subject: [PATCH 03/28] Change `sample_rate` to `traces_sample_rate` --- src/docs/sdk/performance/dynamic-sampling-context.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/docs/sdk/performance/dynamic-sampling-context.mdx b/src/docs/sdk/performance/dynamic-sampling-context.mdx index 5d0f6e3f58..9730123d90 100644 --- a/src/docs/sdk/performance/dynamic-sampling-context.mdx +++ b/src/docs/sdk/performance/dynamic-sampling-context.mdx @@ -10,7 +10,7 @@ Anything that sounds fishy probably is. -Until now, traces sampling was only done through a `sample_rate` option in the SDKs. +Until now, traces sampling was only done through a `traces_sample_rate` option in the SDKs. This has quite a few drawbacks for users of Sentry SDKs: - Changing the sampling rate involved either redeploying applications (which is problematic in case of applications that are not updated automatically, i.e., mobile apps or physically distributed software) or building complex systems to dynamically fetch a sampling rate. From 082ca1b09c7bed8f4d5b1d25c5f1c6aaf979b3f5 Mon Sep 17 00:00:00 2001 From: Luca Forstner Date: Wed, 22 Jun 2022 13:05:44 +0200 Subject: [PATCH 04/28] Change "delimiter" to "prefix" Co-authored-by: Alexander Dinauer --- src/docs/sdk/performance/dynamic-sampling-context.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/docs/sdk/performance/dynamic-sampling-context.mdx b/src/docs/sdk/performance/dynamic-sampling-context.mdx index 9730123d90..0adf5ddbb7 100644 --- a/src/docs/sdk/performance/dynamic-sampling-context.mdx +++ b/src/docs/sdk/performance/dynamic-sampling-context.mdx @@ -89,7 +89,7 @@ While all of these values are optional, SDKs should make their best effort to ad - `sentry-samplerate` - Sample rate as defined by the user in the SDK options SDKs must set all of the keys in the form of "`sentry-[name]`". -The delimiter "`sentry-`" acts to identify key-value pairs set by Sentry SDKs. +The prefix "`sentry-`" acts to identify key-value pairs set by Sentry SDKs. Additionally, we chose `[name]` to be written in "snake case" without any underscore ( `_` ) characters. This naming convention is the most language agnostic. ### Envelope Header From ec30942000b971681c4767a8c4bed2aefa0f378e Mon Sep 17 00:00:00 2001 From: Luca Forstner Date: Wed, 22 Jun 2022 13:53:20 +0200 Subject: [PATCH 05/28] Clarify that DSC cannot be altered after first propagation --- src/docs/sdk/performance/dynamic-sampling-context.mdx | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/src/docs/sdk/performance/dynamic-sampling-context.mdx b/src/docs/sdk/performance/dynamic-sampling-context.mdx index 0adf5ddbb7..f6c8002bf6 100644 --- a/src/docs/sdk/performance/dynamic-sampling-context.mdx +++ b/src/docs/sdk/performance/dynamic-sampling-context.mdx @@ -29,8 +29,11 @@ Implementing Dynamic Sampling comes with challenges, especially on the ingestion For Dynamic Sampling, we want to make sampling decisions for entire traces. However, to keep ingestion speedy, Relay only looks at singular transactions in isolation (as opposed to looking at whole traces). This means that we need the exact same decision basis for all transactions belonging to a trace. -In other words, all transactions of a trace need to hold all of the information to make a sampling decision, and that information needs to be the same across all transactions of the trace. +In other words, all transactions of a trace need to hold all of the information to make a sampling decision, and that **information needs to be the same across all transactions of the trace**. We call the information we base sampling decisions on **"Dynamic Sampling Context"** or **"DSC"**. +As a mental model: +The head transaction in a trace determines the Dynamic Sampling Context for all following transactions in that trace. +No information can be changed, added or deleted after the first propagation. ### SDKs From 30d630cc463d1aee88ddcb705ccd995ce03b5b83 Mon Sep 17 00:00:00 2001 From: Luca Forstner Date: Wed, 22 Jun 2022 14:15:01 +0200 Subject: [PATCH 06/28] Specify that we're using the `trace` envelope header --- src/docs/sdk/performance/dynamic-sampling-context.mdx | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/src/docs/sdk/performance/dynamic-sampling-context.mdx b/src/docs/sdk/performance/dynamic-sampling-context.mdx index f6c8002bf6..c9c4ab8cf0 100644 --- a/src/docs/sdk/performance/dynamic-sampling-context.mdx +++ b/src/docs/sdk/performance/dynamic-sampling-context.mdx @@ -42,7 +42,7 @@ This involves: 1. Collecting the information that makes up the DSC **xor** extracting the DSC from incoming requests. 2. Propagating DSC to downstream SDKs. -3. Sending the DSC to Sentry via an envelope. +3. Sending the DSC to Sentry via the `trace` envelope header. Because there are quite a few things to keep in mind for DSC propagation and to avoid every SDK running into the same problems, we defined a [unified propagation mechanism](#unified-propagation-mechanism) (step-by-step instructions) that all SDK implementations should be able to follow. @@ -75,7 +75,7 @@ See the [Payloads section](#payloads) for a complete list of key-value pairs tha ## Payloads -Dynamic Sampling Context is propagated via a baggage header and sent to Sentry via transaction envelope headers. +Dynamic Sampling Context is propagated via a baggage header and sent to Sentry via the `trace` envelope header. ### Baggage-Header @@ -97,10 +97,10 @@ Additionally, we chose `[name]` to be written in "snake case" without any unders ### Envelope Header -Dynamic Sampling Context is transferred to Sentry through the transaction envelope headers, keyed by `trace`. -It corresponds directly to the definition of Trace Context. +Dynamic Sampling Context is transferred to Sentry through the `trace` envelope header. +The value of this header corresponds directly to the definition of Trace Context. -When a transaction is reported to Sentry, the Dynamic Sampling Context must be mapped to Trace Context in the following way: +When a transaction is reported to Sentry, the Dynamic Sampling Context must be mapped to the `trace` envelope header in the following way: - `sentry-release` ➝ `release` - `sentry-environment` ➝ `environment` From 0513094258f61cecaa3deddd390575e7d37a91b7 Mon Sep 17 00:00:00 2001 From: Luca Forstner Date: Wed, 22 Jun 2022 15:09:21 +0200 Subject: [PATCH 07/28] Clarify functionality of `has_sentry_value_in_baggage_header` --- src/docs/sdk/performance/dynamic-sampling-context.mdx | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/src/docs/sdk/performance/dynamic-sampling-context.mdx b/src/docs/sdk/performance/dynamic-sampling-context.mdx index c9c4ab8cf0..91eb3d2902 100644 --- a/src/docs/sdk/performance/dynamic-sampling-context.mdx +++ b/src/docs/sdk/performance/dynamic-sampling-context.mdx @@ -120,6 +120,10 @@ def collect_dynamic_sampling_context(): # Placeholder function that collects as many values for Dynamic Sampling Context # as possible and returns a dict +def has_sentry_value_in_baggage_header(request): + # Placeholder function that returns true when there is at least one key-value pair in the baggage + # header of `request`, where the key starts with "sentry-", otherwise it returns false. + def on_incoming_request(request): if has_header(request, "sentry-trace") and (not has_header(request, "baggage") or not has_sentry_value_in_baggage_header(request)): # Request comes from an old SDK which doesn't support Dynamic Sampling Context yet From 4a7508fdca382eb587fe7e2783257ed7fd064af4 Mon Sep 17 00:00:00 2001 From: Luca Forstner Date: Wed, 22 Jun 2022 15:26:50 +0200 Subject: [PATCH 08/28] Clarify what values are needed for Dynamic Sampling to work at all --- .../performance/dynamic-sampling-context.mdx | 27 ++++++++++--------- 1 file changed, 15 insertions(+), 12 deletions(-) diff --git a/src/docs/sdk/performance/dynamic-sampling-context.mdx b/src/docs/sdk/performance/dynamic-sampling-context.mdx index 91eb3d2902..a1c6b82f75 100644 --- a/src/docs/sdk/performance/dynamic-sampling-context.mdx +++ b/src/docs/sdk/performance/dynamic-sampling-context.mdx @@ -80,16 +80,19 @@ Dynamic Sampling Context is propagated via a baggage header and sent to Sentry v ### Baggage-Header SDKs may set the following key-value pairs on baggage headers. -While all of these values are optional, SDKs should make their best effort to add as many of them to the baggage header as possible when starting a trace. +While none of these values are strictly required, SDKs should make their best effort to add as many of them to the baggage header as possible when starting a trace. -- `sentry-traceid` - The original trace ID as generated by the SDK -- `sentry-publickey` - Public key as defined by the user via the DSN in the SDK options -- `sentry-release` - The release as defined by the user in the SDK options -- `sentry-environment` - The environment as defined by the user in the SDK options -- `sentry-transaction` - The name of the trace's origin transaction in unparameterized (raw) format -- `sentry-userid` - User ID as set by the user with `scope.set_user` -- `sentry-usersegment` - User segment as set by the user with `scope.set_user` -- `sentry-samplerate` - Sample rate as defined by the user in the SDK options +Some values are marked as "required for Dynamic sampling to work". +Nothing bad is going to happen when these values are missing, however, Dynamic Sampling will not work without them. + +- `sentry-traceid` - **Required for Dynamic Sampling to work** - The original trace ID as generated by the SDK. +- `sentry-publickey` - **Required for Dynamic Sampling to work** - Public key as defined by the user via the DSN in the SDK options. +- `sentry-samplerate` - **Required for Dynamic Sampling to work** - Sample rate as defined by the user in the SDK options. +- `sentry-release` - The release as defined by the user in the SDK options. +- `sentry-environment` - The environment as defined by the user in the SDK options. +- `sentry-transaction` - The name of the trace's origin transaction in unparameterized (raw) format. +- `sentry-userid` - User ID as set by the user with `scope.set_user`. +- `sentry-usersegment` - User segment as set by the user with `scope.set_user`. SDKs must set all of the keys in the form of "`sentry-[name]`". The prefix "`sentry-`" acts to identify key-value pairs set by Sentry SDKs. @@ -102,14 +105,14 @@ The value of this header corresponds directly to the definition of `tracesSampleRate` or `tracesSampler` options in the SDKs. This has quite a few drawbacks for users of Sentry SDKs: - Changing the sampling rate involved either redeploying applications (which is problematic in case of applications that are not updated automatically, i.e., mobile apps or physically distributed software) or building complex systems to dynamically fetch a sampling rate. From d848543826495a381abd1803da04afa96340dd77 Mon Sep 17 00:00:00 2001 From: Luca Forstner Date: Wed, 22 Jun 2022 15:34:19 +0200 Subject: [PATCH 10/28] Minor rewording --- src/docs/sdk/performance/dynamic-sampling-context.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/docs/sdk/performance/dynamic-sampling-context.mdx b/src/docs/sdk/performance/dynamic-sampling-context.mdx index 59a7eab235..4c69fd20d9 100644 --- a/src/docs/sdk/performance/dynamic-sampling-context.mdx +++ b/src/docs/sdk/performance/dynamic-sampling-context.mdx @@ -16,7 +16,7 @@ This has quite a few drawbacks for users of Sentry SDKs: - Changing the sampling rate involved either redeploying applications (which is problematic in case of applications that are not updated automatically, i.e., mobile apps or physically distributed software) or building complex systems to dynamically fetch a sampling rate. - Sampling only happened based on a factor of randomness. Employing sampling rules, for example, based on event parameters, is currently very complex. - While writing rules for singular **transactions** is possible, enforcing them on an entire **trace** is infeasable. + While writing rules for singular **transactions** is possible, enforcing them on entire **traces** is infeasable. The solution for these problems is **Dynamic Sampling**. Dynamic Sampling allows users to configure **sampling rules** directly in the Sentry interface. Important: Sampling rules may also be applied to **entire traces**. From 48f8ab9d2e3b20f2b36fe82e7b29c8209e875888 Mon Sep 17 00:00:00 2001 From: Luca Forstner Date: Wed, 22 Jun 2022 15:38:42 +0200 Subject: [PATCH 11/28] Clarify that DS allows users to sample traces AND transactions Co-authored-by: Abhijeet Prasad --- src/docs/sdk/performance/dynamic-sampling-context.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/docs/sdk/performance/dynamic-sampling-context.mdx b/src/docs/sdk/performance/dynamic-sampling-context.mdx index 4c69fd20d9..c24540bb9c 100644 --- a/src/docs/sdk/performance/dynamic-sampling-context.mdx +++ b/src/docs/sdk/performance/dynamic-sampling-context.mdx @@ -19,7 +19,7 @@ This has quite a few drawbacks for users of Sentry SDKs: While writing rules for singular **transactions** is possible, enforcing them on entire **traces** is infeasable. The solution for these problems is **Dynamic Sampling**. -Dynamic Sampling allows users to configure **sampling rules** directly in the Sentry interface. Important: Sampling rules may also be applied to **entire traces**. +Dynamic Sampling allows users to configure **sampling rules** directly in the Sentry interface. Important: Sampling rules may be applied to **entire traces** or to a **single transaction**. ## High-Level Problem Statement From 67b98b2f75474dd52572263f246a84af65600039 Mon Sep 17 00:00:00 2001 From: Luca Forstner Date: Thu, 23 Jun 2022 12:49:16 +0200 Subject: [PATCH 12/28] Clarify requiredness of values --- .../performance/dynamic-sampling-context.mdx | 21 +++++++++---------- 1 file changed, 10 insertions(+), 11 deletions(-) diff --git a/src/docs/sdk/performance/dynamic-sampling-context.mdx b/src/docs/sdk/performance/dynamic-sampling-context.mdx index c24540bb9c..17f1db5077 100644 --- a/src/docs/sdk/performance/dynamic-sampling-context.mdx +++ b/src/docs/sdk/performance/dynamic-sampling-context.mdx @@ -79,21 +79,20 @@ Dynamic Sampling Context is propagated via a baggage header and sent to Sentry v ### Baggage-Header -SDKs may set the following key-value pairs on baggage headers. -While none of these values are strictly required, SDKs should make their best effort to add as many of them to the baggage header as possible when starting a trace. +SDKs may set the following key-value pairs on baggage headers: -Some values are marked as "required for Dynamic sampling to work". -Nothing bad is going to happen when these values are missing, however, Dynamic Sampling will not work without them. - -- `sentry-traceid` - **Required for Dynamic Sampling to work** - The original trace ID as generated by the SDK. -- `sentry-publickey` - **Required for Dynamic Sampling to work** - Public key as defined by the user via the DSN in the SDK options. -- `sentry-samplerate` - **Required for Dynamic Sampling to work** - Sample rate as defined by the user in the SDK options. +- `sentry-traceid` - The original trace ID as generated by the SDK. +- `sentry-publickey` - Public key as defined by the user via the DSN in the SDK options. +- `sentry-samplerate` - Sample rate as defined by the user in the SDK options. - `sentry-release` - The release as defined by the user in the SDK options. - `sentry-environment` - The environment as defined by the user in the SDK options. - `sentry-transaction` - The name of the trace's origin transaction in unparameterized (raw) format. - `sentry-userid` - User ID as set by the user with `scope.set_user`. - `sentry-usersegment` - User segment as set by the user with `scope.set_user`. +All of these values are required in a sense, that when they are known to the head SDK at the time of propagation, they must also be propagated. +In any case, `sentry-traceid`, `sentry-publickey`, `sentry-samplerate` should always be known to the SDK, so these values are strictly required. + SDKs must set all of the keys in the form of "`sentry-[name]`". The prefix "`sentry-`" acts to identify key-value pairs set by Sentry SDKs. Additionally, we chose `[name]` to be written in "snake case" without any underscore ( `_` ) characters. This naming convention is the most language agnostic. @@ -105,9 +104,9 @@ The value of this header corresponds directly to the definition of Date: Thu, 23 Jun 2022 12:50:34 +0200 Subject: [PATCH 14/28] s/true/True/ --- src/docs/sdk/performance/dynamic-sampling-context.mdx | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/src/docs/sdk/performance/dynamic-sampling-context.mdx b/src/docs/sdk/performance/dynamic-sampling-context.mdx index f7dc19be7a..16b350a511 100644 --- a/src/docs/sdk/performance/dynamic-sampling-context.mdx +++ b/src/docs/sdk/performance/dynamic-sampling-context.mdx @@ -123,22 +123,22 @@ def collect_dynamic_sampling_context(): # as possible and returns a dict def has_sentry_value_in_baggage_header(request): - # Placeholder function that returns true when there is at least one key-value pair in the baggage - # header of `request`, for which the key starts with "sentry-". Otherwise, it returns false. + # Placeholder function that returns True when there is at least one key-value pair in the baggage + # header of `request`, for which the key starts with "sentry-". Otherwise, it returns False. def on_incoming_request(request): if has_header(request, "sentry-trace") and (not has_header(request, "baggage") or not has_sentry_value_in_baggage_header(request)): # Request comes from an old SDK which doesn't support Dynamic Sampling Context yet # --> we don't propagate baggage for this trace - transaction.baggage_locked = true + transaction.baggage_locked = True transaction.baggage = {} elif has_header(request, "baggage") and has_sentry_value_in_baggage_header(request): - transaction.baggage_locked = true + transaction.baggage_locked = True transaction.baggage = baggage_header_to_dict(request.headers.baggage) def on_outgoing_request(request): if not transaction.baggage_locked: - transaction.baggage_locked = true + transaction.baggage_locked = True if not transaction.baggage: transaction.baggage = {} transaction.baggage = merge_dicts(collect_dynamic_sampling_context(), transaction.baggage) From 35e91370b33d2dad9875d36d106d976d393cf163 Mon Sep 17 00:00:00 2001 From: Luca Forstner Date: Thu, 23 Jun 2022 13:03:54 +0200 Subject: [PATCH 15/28] Apply wording suggestions Co-authored-by: Philipp Hofmann --- src/docs/sdk/performance/dynamic-sampling-context.mdx | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/src/docs/sdk/performance/dynamic-sampling-context.mdx b/src/docs/sdk/performance/dynamic-sampling-context.mdx index 16b350a511..6662e42221 100644 --- a/src/docs/sdk/performance/dynamic-sampling-context.mdx +++ b/src/docs/sdk/performance/dynamic-sampling-context.mdx @@ -13,10 +13,10 @@ Anything that sounds fishy probably is. Until now, traces sampling was only done through the `tracesSampleRate` or `tracesSampler` options in the SDKs. This has quite a few drawbacks for users of Sentry SDKs: -- Changing the sampling rate involved either redeploying applications (which is problematic in case of applications that are not updated automatically, i.e., mobile apps or physically distributed software) or building complex systems to dynamically fetch a sampling rate. +- Changing the sampling rate involved either redeploying applications (which is problematic in case of applications that are not updated automatically, i.e., mobile apps or physically distributed software) or building complex systems to dynamically fetch a sample rate. - Sampling only happened based on a factor of randomness. - Employing sampling rules, for example, based on event parameters, is currently very complex. - While writing rules for singular **transactions** is possible, enforcing them on entire **traces** is infeasable. +- Employing sampling rules, for example, based on event parameters, is very complex. +- While writing rules for singular **transactions** is possible, enforcing them on entire **traces** is infeasible. The solution for these problems is **Dynamic Sampling**. Dynamic Sampling allows users to configure **sampling rules** directly in the Sentry interface. Important: Sampling rules may be applied to **entire traces** or to a **single transaction**. From 45b444c6166a980b22e685d3b08d899e6dc0acd2 Mon Sep 17 00:00:00 2001 From: Luca Forstner Date: Thu, 23 Jun 2022 13:08:10 +0200 Subject: [PATCH 16/28] Future-proof introduction Co-authored-by: Philipp Hofmann --- src/docs/sdk/performance/dynamic-sampling-context.mdx | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/src/docs/sdk/performance/dynamic-sampling-context.mdx b/src/docs/sdk/performance/dynamic-sampling-context.mdx index 6662e42221..35949b96f6 100644 --- a/src/docs/sdk/performance/dynamic-sampling-context.mdx +++ b/src/docs/sdk/performance/dynamic-sampling-context.mdx @@ -10,8 +10,7 @@ Anything that sounds fishy probably is. -Until now, traces sampling was only done through the `tracesSampleRate` or `tracesSampler` options in the SDKs. -This has quite a few drawbacks for users of Sentry SDKs: +Traces sampling done through the `tracesSampleRate` or `tracesSampler` options in the SDKs has quite a few drawbacks for users of Sentry SDKs: - Changing the sampling rate involved either redeploying applications (which is problematic in case of applications that are not updated automatically, i.e., mobile apps or physically distributed software) or building complex systems to dynamically fetch a sample rate. - Sampling only happened based on a factor of randomness. From 57a8f54bb0dcef6d3452825fc42a0b53005b4156 Mon Sep 17 00:00:00 2001 From: Luca Forstner Date: Thu, 23 Jun 2022 13:10:21 +0200 Subject: [PATCH 17/28] Open PRs plz --- src/docs/sdk/performance/dynamic-sampling-context.mdx | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/src/docs/sdk/performance/dynamic-sampling-context.mdx b/src/docs/sdk/performance/dynamic-sampling-context.mdx index 35949b96f6..a97c8927ac 100644 --- a/src/docs/sdk/performance/dynamic-sampling-context.mdx +++ b/src/docs/sdk/performance/dynamic-sampling-context.mdx @@ -6,7 +6,8 @@ title: "Dynamic Sampling Context (Experimental)" This page is under active development. Specifications are not final and subject to change. -Anything that sounds fishy probably is. +Anything that sounds fishy probably is - nothing is set in stone. +Opening PRs to improve this page is therefore highly encouraged! From d3ed0bd6b3e5e58a04644bccef13b7570c8e18fc Mon Sep 17 00:00:00 2001 From: Luca Forstner Date: Thu, 23 Jun 2022 13:22:41 +0200 Subject: [PATCH 18/28] Update considerations section --- src/docs/sdk/performance/dynamic-sampling-context.mdx | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/src/docs/sdk/performance/dynamic-sampling-context.mdx b/src/docs/sdk/performance/dynamic-sampling-context.mdx index a97c8927ac..df3fc7d95c 100644 --- a/src/docs/sdk/performance/dynamic-sampling-context.mdx +++ b/src/docs/sdk/performance/dynamic-sampling-context.mdx @@ -160,8 +160,11 @@ When users attempt to set DSC afterwards, our SDKs should make this operation a ## Considerations -Todo: +TODO - Add some sort of Q&A section on the following questions, after evaluating if they still need to be answered: -- Why baggage and not trace context https://www.w3.org/TR/trace-context/? - Why must baggage be immutable before the second transaction has been started? +- What are the consequences and impacts of the immutability of baggeg on Dynamic Sampling UX? - Why can't we just make the decision for the whole trace in Relay after the trace is complete? +- What is sample rate smoothing and how does it use `sample_rate` from the Dynamic Sampling Context? +- What are the differences between Dynamic Sampling on traces vs. transactions? +- Why did we choose baggage as propagation mechanism and not trace context https://www.w3.org/TR/trace-context/? From e8fca9adf32318354aae68f227233e06ac19e013 Mon Sep 17 00:00:00 2001 From: Luca Forstner Date: Thu, 23 Jun 2022 15:58:27 +0200 Subject: [PATCH 19/28] Update payload section to reflect discussion outcome --- .../performance/dynamic-sampling-context.mdx | 61 ++++++++++--------- 1 file changed, 32 insertions(+), 29 deletions(-) diff --git a/src/docs/sdk/performance/dynamic-sampling-context.mdx b/src/docs/sdk/performance/dynamic-sampling-context.mdx index df3fc7d95c..c8010189c6 100644 --- a/src/docs/sdk/performance/dynamic-sampling-context.mdx +++ b/src/docs/sdk/performance/dynamic-sampling-context.mdx @@ -68,50 +68,53 @@ Sentry SDKs only concern themselves with Sentry baggage. The following is an example of what a baggage header containing Dynamic Sampling Context may look like: ``` -baggage: other-vendor-value-1=foo;bar;baz, sentry-traceid=771a43a4192642f0b136d5159a501700, sentry-publickey=49d0f7386ad645858ae85020e393bef3; sentry-userid=Am%C3%A9lie, other-vendor-value-2=foo;bar; +baggage: other-vendor-value-1=foo;bar;baz, sentry-trace_id=771a43a4192642f0b136d5159a501700, sentry-public_key=49d0f7386ad645858ae85020e393bef3, sentry-sample_rate=0.01337, sentry-user_id=Am%C3%A9lie, other-vendor-value-2=foo;bar; ``` See the [Payloads section](#payloads) for a complete list of key-value pairs that SDKs should propagate. ## Payloads -Dynamic Sampling Context is propagated via a baggage header and sent to Sentry via the `trace` envelope header. +Dynamic Sampling Context is sent to Sentry via the `trace` envelope header and is propagated to downstream SDKs via a baggage header. -### Baggage-Header +All of the values in the payloads below are required (non-optional) in a sense, that when they are known to an SDK at the time a transaction envelope is sent to Sentry, or at the time a baggage header is propagated, they must also be included in said envelope or baggage. +In any case, `trace_id`, `public_key`, and `sample_rate` should always be known to an SDK, so these values are strictly required. -SDKs may set the following key-value pairs on baggage headers: +### Envelope Header -- `sentry-traceid` - The original trace ID as generated by the SDK. -- `sentry-publickey` - Public key as defined by the user via the DSN in the SDK options. -- `sentry-samplerate` - Sample rate as defined by the user in the SDK options. -- `sentry-release` - The release as defined by the user in the SDK options. -- `sentry-environment` - The environment as defined by the user in the SDK options. -- `sentry-transaction` - The name of the trace's origin transaction in unparameterized (raw) format. -- `sentry-userid` - User ID as set by the user with `scope.set_user`. -- `sentry-usersegment` - User segment as set by the user with `scope.set_user`. +Dynamic Sampling Context is transferred to Sentry through the `trace` envelope header. +The value of this envelope header is a JSON object with the following fields: -All of these values are required in a sense, that when they are known to the head SDK at the time of propagation, they must also be propagated. -In any case, `sentry-traceid`, `sentry-publickey`, `sentry-samplerate` should always be known to the SDK, so these values are strictly required. +- `trace_id` (string) - The original trace ID as generated by the SDK, UUID V4 encoded as a hexadecimal sequence with no dashes (e.g. `771a43a4192642f0b136d5159a501700`) that is a sequence of 32 hexadecimal digits. This must match the trace id of the submitted transaction item. +- `public_key` (string) - Public key from the DSN used by the SDK. It allows Sentry to sample traces spanning multiple projects, by resolving the same set of rules based on the starting project. +- `sample_rate` (string) - The sample rate as defined by the user on the SDK. +- `release` (string) - The release name as specified in client options`. +- `environment` (string) - The environment name as specified in client options. +- `user_id` (string) - User ID as set by the user with `scope.set_user`. +- `user_segment` (string) - User segment as set by the user with `scope.set_user`. +- `transaction` (string) - The transaction name set on the scope. + +### Baggage-Header + +SDKs may use the following keys to set entries on `baggage` HTTP headers: + +- `sentry-trace_id` +- `sentry-public_key` +- `sentry-sample_rate` +- `sentry-release` +- `sentry-environment` +- `sentry-user_id` +- `sentry-user_segment` +- `sentry-transaction` SDKs must set all of the keys in the form of "`sentry-[name]`". The prefix "`sentry-`" acts to identify key-value pairs set by Sentry SDKs. -Additionally, we chose `[name]` to be written in "snake case" without any underscore ( `_` ) characters. This naming convention is the most language agnostic. -### Envelope Header +All of the keys are defined in a way so their value directly corresponds to one of the fields on the `trace` envelope header. +**This allows SDKs to put all of the sentry key-value pairs from the `baggage` directly onto the envelope header, after stripping away the `sentry-` prefix.** -Dynamic Sampling Context is transferred to Sentry through the `trace` envelope header. -The value of this header corresponds directly to the definition of Trace Context. - -When a transaction is reported to Sentry, the Dynamic Sampling Context must be mapped to the `trace` envelope header in the following way: - -- `sentry-traceid` ➝ `trace_id` -- `sentry-publickey` ➝ `public_key` -- `sentry-samplerate` ➝ `sample_rate` -- `sentry-release` ➝ `release` -- `sentry-environment` ➝ `environment` -- `sentry-transaction` ➝ `transaction` -- `sentry-userid` ➝ `user.id` -- `sentry-usersegment` ➝ `user.segment` +Being able to simply copy key-value pairs from the baggage header onto the `trace` envelope header gives us the flexibility to provide dedicated API methods to propagate additional values using Dynamic Sampling Context. +This, in return, allows users to define their own values in the Dynamic Sampling Context so they can sample by those in the Sentry interface. ## Unified Propagation Mechanism From cdb1248aa41c531dc03f726abf205fecf1bf87ca Mon Sep 17 00:00:00 2001 From: Luca Forstner Date: Thu, 23 Jun 2022 16:34:40 +0200 Subject: [PATCH 20/28] Deprecate Trace Contexts --- src/docs/sdk/performance/trace-context.mdx | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/src/docs/sdk/performance/trace-context.mdx b/src/docs/sdk/performance/trace-context.mdx index 6f4276b20a..43a79857b6 100644 --- a/src/docs/sdk/performance/trace-context.mdx +++ b/src/docs/sdk/performance/trace-context.mdx @@ -1,10 +1,11 @@ --- -title: "Trace Contexts (Experimental)" +title: "Trace Contexts (Deprecated)" --- -This feature is under development and not required for all SDKs supporting Performance Monitoring, yet. Please consider the Performance Guidelines as reference documentation. Anything that contradicts it is a mistake (or an out of date detail) in this document. +This page and Trace Contexts is deprecated and will be superseded by Dynamic Sampling Context. +Please use that page instead to look up information on tracing context propagation. @@ -22,7 +23,6 @@ Regardless of the transport mechanism, the trace context is a JSON object with t - `trace_id` (string, required) - UUID V4 encoded as a hexadecimal sequence with no dashes (e.g. `771a43a4192642f0b136d5159a501700`) that is a sequence of 32 hexadecimal digits. This must match the trace id of the submitted transaction item. - `public_key` (string, required) - Public key from the DSN used by the SDK. It allows Sentry to sample traces spanning multiple projects, by resolving the same set of rules based on the starting project. -- `sample_rate` (number, optional) - The sample rate as defined by the user on the SDK where the trace originated. - `release` (string, optional) - The release name as specified in client options, usually: `package@x.y.z+build`. _This should match the `release` attribute of the transaction event payload_.\* - `environment` - The environment name as specified in client options, for example `staging`. _This should match the `environment` attribute of the transaction event payload_.\* - `user` (object, optional) - A subset of the scope's user context containing the following fields: From 85be8e75cd759041b5690b5786f39cd66ec462f0 Mon Sep 17 00:00:00 2001 From: Luca Forstner Date: Thu, 23 Jun 2022 16:46:38 +0200 Subject: [PATCH 21/28] Clarify format of `sample_rate` --- src/docs/sdk/performance/dynamic-sampling-context.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/docs/sdk/performance/dynamic-sampling-context.mdx b/src/docs/sdk/performance/dynamic-sampling-context.mdx index c8010189c6..d5bdd7d8a7 100644 --- a/src/docs/sdk/performance/dynamic-sampling-context.mdx +++ b/src/docs/sdk/performance/dynamic-sampling-context.mdx @@ -87,7 +87,7 @@ The value of this envelope header is a JSON object with the following fields: - `trace_id` (string) - The original trace ID as generated by the SDK, UUID V4 encoded as a hexadecimal sequence with no dashes (e.g. `771a43a4192642f0b136d5159a501700`) that is a sequence of 32 hexadecimal digits. This must match the trace id of the submitted transaction item. - `public_key` (string) - Public key from the DSN used by the SDK. It allows Sentry to sample traces spanning multiple projects, by resolving the same set of rules based on the starting project. -- `sample_rate` (string) - The sample rate as defined by the user on the SDK. +- `sample_rate` (string) - The sample rate as defined by the user on the SDK. This string should always be a number between (and including) 0 and 1 in basic float notation (`0.04242`) - no funky business like exponents or anything similar. - `release` (string) - The release name as specified in client options`. - `environment` (string) - The environment name as specified in client options. - `user_id` (string) - User ID as set by the user with `scope.set_user`. From 0c983c09b9bd259814cbd626d2aed4b750004c26 Mon Sep 17 00:00:00 2001 From: Luca Forstner Date: Thu, 23 Jun 2022 16:48:15 +0200 Subject: [PATCH 22/28] Remove Trace Context docs --- src/components/sidebar.tsx | 1 - src/docs/sdk/performance/trace-context.mdx | 275 --------------------- 2 files changed, 276 deletions(-) delete mode 100644 src/docs/sdk/performance/trace-context.mdx diff --git a/src/components/sidebar.tsx b/src/components/sidebar.tsx index ca734e669b..fe83170107 100644 --- a/src/components/sidebar.tsx +++ b/src/components/sidebar.tsx @@ -206,7 +206,6 @@ export default () => { Envelopes Rate Limiting - Trace Contexts Span Operations Dynamic Sampling Context diff --git a/src/docs/sdk/performance/trace-context.mdx b/src/docs/sdk/performance/trace-context.mdx deleted file mode 100644 index 43a79857b6..0000000000 --- a/src/docs/sdk/performance/trace-context.mdx +++ /dev/null @@ -1,275 +0,0 @@ ---- -title: "Trace Contexts (Deprecated)" ---- - - - -This page and Trace Contexts is deprecated and will be superseded by Dynamic Sampling Context. -Please use that page instead to look up information on tracing context propagation. - - - -In order to sample traces we need to pass along the call chain a trace id together with the necessary information for making a sampling decision, the so-called "trace context". - -## Protocol - -Trace information is passed between SDKs as an encoded `tracestate` header, which SDKs are expected to intercept and propagate. - -For event submission to sentry, the trace context is sent as JSON object embedded in an Envelope header with the key `trace`. - -### Trace Context - -Regardless of the transport mechanism, the trace context is a JSON object with the following fields: - -- `trace_id` (string, required) - UUID V4 encoded as a hexadecimal sequence with no dashes (e.g. `771a43a4192642f0b136d5159a501700`) that is a sequence of 32 hexadecimal digits. This must match the trace id of the submitted transaction item. -- `public_key` (string, required) - Public key from the DSN used by the SDK. It allows Sentry to sample traces spanning multiple projects, by resolving the same set of rules based on the starting project. -- `release` (string, optional) - The release name as specified in client options, usually: `package@x.y.z+build`. _This should match the `release` attribute of the transaction event payload_.\* -- `environment` - The environment name as specified in client options, for example `staging`. _This should match the `environment` attribute of the transaction event payload_.\* -- `user` (object, optional) - A subset of the scope's user context containing the following fields: - - `id` (string, optional) - The `id` attribute of the user context. - - `segment` (string, optional) - The value of a `segment` attribute in the user's data bag, if it exists. In the future, this field may be promoted to a proper attribute of the user context. -- `transaction` (string, optional) - The transaction name set on the scope. _This should match the `transaction` attribute of the transaction event payload_.\* - -\*_ See "Freezing the Context" for more information on consistency between the trace context and fields in the event payload._ - -**Example:** - -```json -{ - "trace_id": "771a43a4192642f0b136d5159a501700", - "public_key": "49d0f7386ad645858ae85020e393bef3", - "release": "myapp@1.1.2", - "environment": "production", - "user": { - "id": "7efa4978da177713df088f846f8c484d", - "segment": "vip" - }, - "transaction": "/api/0/project_details" -} -``` - -### Envelope Headers - -When sending transaction events to Sentry via Envelopes, the trace information must be set in the envelope headers under the `trace` field. - -Here's an example of a minimal envelope header containing the trace context (Although the header does not contain newlines, in the example below newlines were added for readability): - -```json -{ - "event_id": "12c2d058d58442709aa2eca08bf20986", - "trace": { - "trace_id": "771a43a4192642f0b136d5159a501700", - "public_key": "49d0f7386ad645858ae85020e393bef3" - // other trace attributes - } -} -``` - -### Tracestate Headers - -When propagating trace contexts to other SDKs, Sentry uses the [W3C `tracestate` header](https://www.w3.org/TR/trace-context/#trace-context-http-headers-format). See "Trace Propagation" for more information on how to propagate these headers to other SDKs. - -Tracestate headers contain several vendor-specific opaque data. As per HTTP spec, these multiple header values can be given in two ways, usually supported by HTTP libraries and framework out-of-the-box: - -- Joined by comma: - ``` - tracestate: sentry=,other= - ``` -- Repetition: - ``` - tracestate: sentry= - tracestate: other= - ``` - -To create contents of the tracestate header: - -1. Serialize the full trace context to JSON, including the trace_id. -2. Encode the resulting JSON string as UTF-8, if strings are represented differently on the platform. -3. Encode the UTF-8 string with base64. -4. Strip trailing padding characters (`=`), since this is a reserved character. -5. Prepend with `"sentry="`, resulting in `"sentry="`. -6. Join into the header as described above. - - - -By stripping trailing padding, default base64 parsers may detect an incomplete payload. Choose a parsing mode that either allows for missing `=` or allows truncated payloads. - - - -For example, the data - -```json -{ - "trace_id": "771a43a4192642f0b136d5159a501700", - "public_key": "49d0f7386ad645858ae85020e393bef3", - "release": "1.1.22", - "environment": "dev", - "user": { - "segment": "vip", - "id": "7efa4978da177713df088f846f8c484d" - } -} -``` - -would encode as - -``` -ewogICJ0cmFjZV9pZCI6ICI3NzFhNDNhNDE5MjY0MmYwYjEzNmQ1MTU5YTUwMTcwMCIsCiAgInB1YmxpY19rZXkiOiAiNDlkMGY3Mzg2YWQ2NDU4NThhZTg1MDIwZTM5M2JlZjMiLAogICJyZWxlYXNlIjogIjEuMS4yMiIsCiAgImVudmlyb25tZW50IjogImRldiIsCiAgInVzZXIiOiB7CiAgICAic2VnbWVudCI6ICJ2aXAiLAogICAgImlkIjogIjdlZmE0OTc4ZGExNzc3MTNkZjA4OGY4NDZmOGM0ODRkIgogIH0KfQ -``` - -and result in the header - -``` -tracestate: sentry=ewogIC...IH0KfQ,other=[omitted] -``` - -(Note the third-party entry at the end of the header; new or modified entries are always added to the lefthand side, so we put the `sentry=` value there. Also note that though the encoded value has been elided for legibilty here, in a real header the full value would be used.) - -## Implementation Guidelines - -An SDK supporting this header must: - -- Use scope information when creating a new trace context -- Add an envelope header with the trace context for envelopes containing transactions -- Add a `tracestate` HTTP header to outgoing HTTP requests for propagation -- Intercept incoming HTTP requests for `tracestate` HTTP headers where applicable and apply them to the local trace context - -### Background - -This is an extension of trace ID propagation covered by Performance Guidelines. According to the Unified API tracing spec, Sentry SDKs add an HTTP header `sentry-trace` to outgoing requests via integrations. Most importantly, this header contains the trace ID, which must match the trace id of the transaction event and also of the trace context below. - -The trace context shall be propagated in an additional `tracestate` header defined in [W3C traceparent header](https://www.w3.org/TR/trace-context/#trace-context-http-headers-format). Note that we must keep compatibility with the W3C spec as opposed to the proprietary `sentry-trace` header. The `tracestate` header also contains vendor-specific opaque data in addition to the contents placed by the Sentry SDK. - -### Client options - -While trace contexts are under development, they should be gated behind an internal `trace_sampling` boolean client option. The option defaults to `false` and should not be documented in Sentry docs. - -Based on platform naming guidelines, the option should be cased appropriately: - -- `trace_sampling` in snake case -- `traceSampling` in camel case -- `TraceSampling` in pascal case -- `setTraceSampling` for Java-style setters - -### Adding the Envelope Header - -The SDK should add the envelope header to outgoing envelopes under any of the following conditions: - -1. The envelope contains a transaction event. -2. The scope has a transaction bound. - -Specifically, this means even envelopes without transactions can contain the `trace` envelope header, -allowing Sentry to eventually sample attachments belonging to a transaction. When the envelope includes -a transaction and the scope has a bound transaction, the SDK should use the transaction of the envelope -to create the `trace` envelope header. - -### Freezing the Context - -To ensure fully consistent trace contexts for all transactions in a trace, the trace context cannot be changed once it is sent over the wire, even if scope or options change afterwards. That is, once computed the trace context is no longer updated. Even if the app calls `setRelease`, the old release remains in the context. - -To compensate for lazy calls to functions like `setTransaction` and `setUser`, the trace context can be thought to be in two states: _NEW_ and _SENT_ . Initially, the context is in the _NEW_ state and it is modifiable. Once sent for first time, it becomes _SENT_ and can no longer change. - -We recommend the trace context should be computed on-the-fly the first time it is needed in any of: - -- Creating an Envelope -- Propagation to an outgoing HTTP request - -The trace context must be retained until the user starts a new trace, at which point a new trace context must be computed by the SDK. - - - -It is recommended that SDKs log modifications of attributes that would result in trace context changes like `user.id` when the trace context is frozen, in order to simplify debugging of common dynamic sampling pitfalls. - - - -### Incoming Contexts - -Same as for intercepting trace IDs from inbound HTTP requests, SDKs should read `tracestate` headers and assume the Sentry trace context, if specified. Such a context is immediately frozen in the _SENT_ state and should no longer allow for modifications. - -## - -## Platform Specifics - -### Encoding in JavaScript - -As mentioned, we need to encode the JSON trace context using UTF-8 strings. JavaScript internally uses UTF16 so we need to work a bit to do the transformation. - -The basic idea is presented in [this](https://attacomsian.com/blog/javascript-base64-encode-decode) article (and in [other](https://stackoverflow.com/questions/30106476/using-javascripts-atob-to-decode-base64-doesnt-properly-decode-utf-8-strings) places). - -In short here's the function that converts a context into a base64 string that can be saved in `tracestate`. In the end we went with a simpler implementation, but the idea is the same: - -```jsx -// Compact form -function objToB64(obj) { - const utf16Json = JSON.stringify(obj); - const b64 = btoa( - encodeURIComponent(utf16Json).replace( - /%([0-9A-F]{2})/g, - function toSolidBytes(match, p1) { - return String.fromCharCode("0x" + p1); - } - ) - ); - const len = b64.length; - if (b64[len - 2] === "=") { - return b64.substr(0, len - 2); - } else if (b64[len - 1] === "=") { - return b64.substr(0, len - 1); - } - return b64; -} - -// Commented -function objToB64(obj) { - // object to JSON string - const utf16Json = JSON.stringify(obj); - // still utf16 string but with non ASCI escaped as UTF-8 numbers) - const encodedUtf8 = encodeURIComponent(utf16Json); - - // replace the escaped code points with utf16 - // in the first 256 code points (the most wierd part) - const b64 = btoa( - endcodedUtf8.replace(/%([0-9A-F]{2})/g, function toSolidBytes(match, p1) { - return String.fromCharCode("0x" + p1); - }) - ); - - // drop the '=' or '==' padding from base64 - const len = b64.length; - if (b64[len - 2] === "=") { - return b64.substr(0, len - 2); - } else if (b64[len - 1] === "=") { - return b64.substr(0, len - 1); - } - return b64; -} -// const test = {"x":"a-🙂-读写汉字 - 学中文"} -// objToB64(test) -// "eyJ4IjoiYS3wn5mCLeivu+WGmeaxieWtlyAtIOWtpuS4reaWhyJ9" -``` - -And here's the function that accepts a base64 string (with or without '=' padding) and returns an object - -```jsx -function b64ToObj(b64) { - utf16 = decodeURIComponent( - atob(b64) - .split("") - .map(function(c) { - return "%" + ("00" + c.charCodeAt(0).toString(16)).slice(-2); - }) - .join("") - ); - return JSON.parse(utf16); -} - -// b64ToObj("eyJ4IjoiYS3wn5mCLeivu+WGmeaxieWtlyAtIOWtpuS4reaWhyJ9") -// {"x":"a-🙂-读写汉字 - 学中文"} -``` - -## Base64 with Command Line Utils - -The GNU `base64` command line utility comes with a switch to wrap the encoded -string. This is not compatible with the `tracestate` header and should be -avoided. If the base64 implementations creates multiple lines, they must be -joined together. From 2fc84fd5586a9dcdfcd43757a8531cfdd095767b Mon Sep 17 00:00:00 2001 From: Abhijeet Prasad Date: Thu, 23 Jun 2022 14:38:53 -0400 Subject: [PATCH 23/28] add Considerations and Challenges section --- .../performance/dynamic-sampling-context.mdx | 55 +++++++++++++++++-- 1 file changed, 50 insertions(+), 5 deletions(-) diff --git a/src/docs/sdk/performance/dynamic-sampling-context.mdx b/src/docs/sdk/performance/dynamic-sampling-context.mdx index d5bdd7d8a7..d8d59e276e 100644 --- a/src/docs/sdk/performance/dynamic-sampling-context.mdx +++ b/src/docs/sdk/performance/dynamic-sampling-context.mdx @@ -11,11 +11,11 @@ Opening PRs to improve this page is therefore highly encouraged! -Traces sampling done through the `tracesSampleRate` or `tracesSampler` options in the SDKs has quite a few drawbacks for users of Sentry SDKs: +Traces sampling done through the `tracesSampleRate` or `tracesSampler` options in the SDKs has quite a few consquences for users of Sentry SDKs: - Changing the sampling rate involved either redeploying applications (which is problematic in case of applications that are not updated automatically, i.e., mobile apps or physically distributed software) or building complex systems to dynamically fetch a sample rate. -- Sampling only happened based on a factor of randomness. -- Employing sampling rules, for example, based on event parameters, is very complex. +- The sampling strategy leverages head-based simple random sampling. +- Employing sampling rules, for example, based on event parameters, is very complex. Sometimes this even requires users to have an in-depth understanding of the event schema. - While writing rules for singular **transactions** is possible, enforcing them on entire **traces** is infeasible. The solution for these problems is **Dynamic Sampling**. @@ -94,6 +94,8 @@ The value of this envelope header is a JSON object with the following fields: - `user_segment` (string) - User segment as set by the user with `scope.set_user`. - `transaction` (string) - The transaction name set on the scope. +It's important to note that at the current moment, only `release`, `environment`, `user_id`, `user_segment`, and `transaction` are used by the product for dynamic sampling functionality. The rest of the context attributes, `trace_id`, `public_key`, and `sample_rate`, are used by Relay for internal decisions (like transaction sample rate smoothing). + ### Baggage-Header SDKs may use the following keys to set entries on `baggage` HTTP headers: @@ -161,12 +163,55 @@ We might want users to be able to set Dynamic Sampling Context values themselves The flag becomes relevant after the first propagation, where Dynamic Sampling Context becomes immutable. When users attempt to set DSC afterwards, our SDKs should make this operation a noop. -## Considerations +## Considerations and Challenges + +This section details some open questions and considerations that need to be addressed for dynamic sampling and the usage of the baggage propogation mechanism. These are not blockers to the adoption of the spec, but instead are here as context for future developments of the dynamic sampling product and spec. + +### The Temporal Problem + +Unlike `environment` or `release`, which should always be known to an SDK at initialization time, `user_id`, `user_segment`, and `transaction` (name) are only known after SDK initialization time. This means that if a trace is propogated from a running transaction *BEFORE* the user/transaction attributes are set, you'll get a portion of transactions in a trace that have different Dynamic Sampling Context than other portions, leading to dynamic sampling not working as expected for users. + +Let's say we want to dynamically sample my browser application based on the `user_id`. In a typical single page application (SPA), the user information has to be requested from some backend service before it can be set with `Sentry.setUser` on the frontend. + +Here's an example of that flow: + +- Page starts loading +- Sentry initializes and starts `pageload` transaction +- Page makes HTTP request to user service to get user (propogates sentry-trace/baggage to user service) + - user service continues trace by automatically creating sampling transaction + - user service pings database service (propogates sentry-trace/baggage to database service) + - database service continues trace by automatically creating sampling transaction +- Page gets data from user service, calls `Sentry.setUser` and sets `user_id` +- Page makes HTTP requests to service A, service B, and service C (propogates sentry-trace/baggage to user service) + - traces propogated to service A, service B, and service C, so 3 child transactions +- Page finishes loading, finishing pageload transaction, which is sent to Sentry + +In this case, the baggage that is propogated to the user service and the downstream database service *does not* have the `user_id` value in it, because it was not yet set on the browser SDK. Therefore, when Relay tries to dynamically sample the user services and database services transactions based on `user_id`, it will not be able to. It will *ONLY* be able to dynamically sample the pageload transaction, as well as the transactions from service A, service B, and service C with a rule based on `user_id`. + +This problem exists for both `user_id` and `user_segment`, and it is because since we don't have the user information on some platforms/frameworks right as we initialize the SDK. + +For `transaction` name, the problem is similar, but it is because of paramaterization. As much as we can, the SDKs will try to paramaterize transaction names (for ex, turn `/teams/123/user/456` into `/teams/:id/user/:id`) so that similar transactions are grouped together in the UI. This improves both aggregate statistics for transactions and the general UX of using the product (setting alerts, checking measurements like web vitals, etc.). For some frameworks, for example React Router v4 - v6, we paramaterize the transaction name after the transaction has been started due to constraints with the framework itself. To illustrate this, let's look at another example: + +- Page starts loading +- Sentry initializes and starts `pageload` transaction (with transaction name `/teams/123/user/456` - based on window URL) +- Page makes HTTP request to service A (propogates sentry-trace/baggage to user service) +- Page renders with react router, triggering paramaterization of transaction name (`/teams/123/user/456` -> `/teams/:id/user/:id`). +- Page finishes loading, finishing pageload transaction, which is sent to Sentry + +When the pageload transaction shows up in Sentry, it'll be called `/teams/:id/user/:id`, but the Dynamic Sampling Context has `/teams/123/user/456` propogated, which means that the transaction from service A will not be affected by any trace-wide dynamic sampling rules put on the transaction name. This will be very confusing to users, as effectively the transaction name they thinking that works will not. + +### Choosing Baggage as the Propogation Mechanism + +For more information on this, check the [DACI around using baggage](https://www.notion.so/sentry/Trace-Context-vs-Baggage-f541525012344111921b6aa7bfd78dc4) (Sentry employees only). + +Previously, we were using the [w3c Trace Context spec](https://www.w3.org/TR/trace-context/) as the mechanism to propogate dynamic sampling context between SDKs. We switched to the [w3c Baggage spec](https://www.w3.org/TR/baggage) though because it was easier to use, required less code on the SDK side, and had much more [liberal size limits](https://www.w3.org/TR/baggage/#limits) than the trace context spec. To make sure that we were not colliding with user defined keys as baggage is an open standard, we decided to prefix all sentry related keys with `sentry-` as a namespace. This allowed us to use baggage as an open standard while only having to worry about Sentry data. + +### Other Items TODO - Add some sort of Q&A section on the following questions, after evaluating if they still need to be answered: - Why must baggage be immutable before the second transaction has been started? -- What are the consequences and impacts of the immutability of baggeg on Dynamic Sampling UX? +- What are the consequences and impacts of the immutability of baggage on Dynamic Sampling UX? - Why can't we just make the decision for the whole trace in Relay after the trace is complete? - What is sample rate smoothing and how does it use `sample_rate` from the Dynamic Sampling Context? - What are the differences between Dynamic Sampling on traces vs. transactions? From 5facd1e5da02e06f6f160bf712c73317474b510d Mon Sep 17 00:00:00 2001 From: Abhijeet Prasad Date: Thu, 23 Jun 2022 14:49:42 -0400 Subject: [PATCH 24/28] remove baggage todo --- src/docs/sdk/performance/dynamic-sampling-context.mdx | 1 - 1 file changed, 1 deletion(-) diff --git a/src/docs/sdk/performance/dynamic-sampling-context.mdx b/src/docs/sdk/performance/dynamic-sampling-context.mdx index d8d59e276e..5df6773d5a 100644 --- a/src/docs/sdk/performance/dynamic-sampling-context.mdx +++ b/src/docs/sdk/performance/dynamic-sampling-context.mdx @@ -215,4 +215,3 @@ TODO - Add some sort of Q&A section on the following questions, after evaluating - Why can't we just make the decision for the whole trace in Relay after the trace is complete? - What is sample rate smoothing and how does it use `sample_rate` from the Dynamic Sampling Context? - What are the differences between Dynamic Sampling on traces vs. transactions? -- Why did we choose baggage as propagation mechanism and not trace context https://www.w3.org/TR/trace-context/? From c2ec13e8fd64e7b557710cb6f42d963be4392b3f Mon Sep 17 00:00:00 2001 From: Luca Forstner Date: Fri, 24 Jun 2022 10:07:28 +0200 Subject: [PATCH 25/28] Put sentences in separate lines for better diffing --- .../performance/dynamic-sampling-context.mdx | 27 ++++++++++++++----- 1 file changed, 20 insertions(+), 7 deletions(-) diff --git a/src/docs/sdk/performance/dynamic-sampling-context.mdx b/src/docs/sdk/performance/dynamic-sampling-context.mdx index 5df6773d5a..f491fa0d60 100644 --- a/src/docs/sdk/performance/dynamic-sampling-context.mdx +++ b/src/docs/sdk/performance/dynamic-sampling-context.mdx @@ -165,13 +165,16 @@ When users attempt to set DSC afterwards, our SDKs should make this operation a ## Considerations and Challenges -This section details some open questions and considerations that need to be addressed for dynamic sampling and the usage of the baggage propogation mechanism. These are not blockers to the adoption of the spec, but instead are here as context for future developments of the dynamic sampling product and spec. +This section details some open questions and considerations that need to be addressed for dynamic sampling and the usage of the baggage propogation mechanism. +These are not blockers to the adoption of the spec, but instead are here as context for future developments of the dynamic sampling product and spec. ### The Temporal Problem -Unlike `environment` or `release`, which should always be known to an SDK at initialization time, `user_id`, `user_segment`, and `transaction` (name) are only known after SDK initialization time. This means that if a trace is propogated from a running transaction *BEFORE* the user/transaction attributes are set, you'll get a portion of transactions in a trace that have different Dynamic Sampling Context than other portions, leading to dynamic sampling not working as expected for users. +Unlike `environment` or `release`, which should always be known to an SDK at initialization time, `user_id`, `user_segment`, and `transaction` (name) are only known after SDK initialization time. +This means that if a trace is propogated from a running transaction _BEFORE_ the user/transaction attributes are set, you'll get a portion of transactions in a trace that have different Dynamic Sampling Context than other portions, leading to dynamic sampling not working as expected for users. -Let's say we want to dynamically sample my browser application based on the `user_id`. In a typical single page application (SPA), the user information has to be requested from some backend service before it can be set with `Sentry.setUser` on the frontend. +Let's say we want to dynamically sample my browser application based on the `user_id`. +In a typical single page application (SPA), the user information has to be requested from some backend service before it can be set with `Sentry.setUser` on the frontend. Here's an example of that flow: @@ -186,11 +189,17 @@ Here's an example of that flow: - traces propogated to service A, service B, and service C, so 3 child transactions - Page finishes loading, finishing pageload transaction, which is sent to Sentry -In this case, the baggage that is propogated to the user service and the downstream database service *does not* have the `user_id` value in it, because it was not yet set on the browser SDK. Therefore, when Relay tries to dynamically sample the user services and database services transactions based on `user_id`, it will not be able to. It will *ONLY* be able to dynamically sample the pageload transaction, as well as the transactions from service A, service B, and service C with a rule based on `user_id`. +In this case, the baggage that is propogated to the user service and the downstream database service _does not_ have the `user_id` value in it, because it was not yet set on the browser SDK. +Therefore, when Relay tries to dynamically sample the user services and database services transactions based on `user_id`, it will not be able to. +It will _ONLY_ be able to dynamically sample the pageload transaction, as well as the transactions from service A, service B, and service C with a rule based on `user_id`. This problem exists for both `user_id` and `user_segment`, and it is because since we don't have the user information on some platforms/frameworks right as we initialize the SDK. -For `transaction` name, the problem is similar, but it is because of paramaterization. As much as we can, the SDKs will try to paramaterize transaction names (for ex, turn `/teams/123/user/456` into `/teams/:id/user/:id`) so that similar transactions are grouped together in the UI. This improves both aggregate statistics for transactions and the general UX of using the product (setting alerts, checking measurements like web vitals, etc.). For some frameworks, for example React Router v4 - v6, we paramaterize the transaction name after the transaction has been started due to constraints with the framework itself. To illustrate this, let's look at another example: +For `transaction` name, the problem is similar, but it is because of paramaterization. +As much as we can, the SDKs will try to paramaterize transaction names (for ex, turn `/teams/123/user/456` into `/teams/:id/user/:id`) so that similar transactions are grouped together in the UI. +This improves both aggregate statistics for transactions and the general UX of using the product (setting alerts, checking measurements like web vitals, etc.). +For some frameworks, for example React Router v4 - v6, we paramaterize the transaction name after the transaction has been started due to constraints with the framework itself. +To illustrate this, let's look at another example: - Page starts loading - Sentry initializes and starts `pageload` transaction (with transaction name `/teams/123/user/456` - based on window URL) @@ -198,13 +207,17 @@ For `transaction` name, the problem is similar, but it is because of paramateriz - Page renders with react router, triggering paramaterization of transaction name (`/teams/123/user/456` -> `/teams/:id/user/:id`). - Page finishes loading, finishing pageload transaction, which is sent to Sentry -When the pageload transaction shows up in Sentry, it'll be called `/teams/:id/user/:id`, but the Dynamic Sampling Context has `/teams/123/user/456` propogated, which means that the transaction from service A will not be affected by any trace-wide dynamic sampling rules put on the transaction name. This will be very confusing to users, as effectively the transaction name they thinking that works will not. +When the pageload transaction shows up in Sentry, it'll be called `/teams/:id/user/:id`, but the Dynamic Sampling Context has `/teams/123/user/456` propogated, which means that the transaction from service A will not be affected by any trace-wide dynamic sampling rules put on the transaction name. +This will be very confusing to users, as effectively the transaction name they thinking that works will not. ### Choosing Baggage as the Propogation Mechanism For more information on this, check the [DACI around using baggage](https://www.notion.so/sentry/Trace-Context-vs-Baggage-f541525012344111921b6aa7bfd78dc4) (Sentry employees only). -Previously, we were using the [w3c Trace Context spec](https://www.w3.org/TR/trace-context/) as the mechanism to propogate dynamic sampling context between SDKs. We switched to the [w3c Baggage spec](https://www.w3.org/TR/baggage) though because it was easier to use, required less code on the SDK side, and had much more [liberal size limits](https://www.w3.org/TR/baggage/#limits) than the trace context spec. To make sure that we were not colliding with user defined keys as baggage is an open standard, we decided to prefix all sentry related keys with `sentry-` as a namespace. This allowed us to use baggage as an open standard while only having to worry about Sentry data. +Previously, we were using the [w3c Trace Context spec](https://www.w3.org/TR/trace-context/) as the mechanism to propogate dynamic sampling context between SDKs. +We switched to the [w3c Baggage spec](https://www.w3.org/TR/baggage) though because it was easier to use, required less code on the SDK side, and had much more [liberal size limits](https://www.w3.org/TR/baggage/#limits) than the trace context spec. +To make sure that we were not colliding with user defined keys as baggage is an open standard, we decided to prefix all sentry related keys with `sentry-` as a namespace. +This allowed us to use baggage as an open standard while only having to worry about Sentry data. ### Other Items From cbf805f2d7929369f03e2f366802bf493397ca34 Mon Sep 17 00:00:00 2001 From: Luca Forstner Date: Fri, 24 Jun 2022 14:25:29 +0200 Subject: [PATCH 26/28] Stress importance of "freezing" DSC --- .../performance/dynamic-sampling-context.mdx | 62 +++++++++++++------ 1 file changed, 44 insertions(+), 18 deletions(-) diff --git a/src/docs/sdk/performance/dynamic-sampling-context.mdx b/src/docs/sdk/performance/dynamic-sampling-context.mdx index f491fa0d60..1559efea39 100644 --- a/src/docs/sdk/performance/dynamic-sampling-context.mdx +++ b/src/docs/sdk/performance/dynamic-sampling-context.mdx @@ -31,9 +31,12 @@ However, to keep ingestion speedy, Relay only looks at singular transactions in This means that we need the exact same decision basis for all transactions belonging to a trace. In other words, all transactions of a trace need to hold all of the information to make a sampling decision, and that **information needs to be the same across all transactions of the trace**. We call the information we base sampling decisions on **"Dynamic Sampling Context"** or **"DSC"**. + As a mental model: The head transaction in a trace determines the Dynamic Sampling Context for all following transactions in that trace. No information can be changed, added or deleted after the first propagation. +Dynamic Sampling Context is bound to only one particular trace, and all the transactions that are part of this trace. +Multiple different traces can and should have different Dynamic Sampling Contexts. ### SDKs @@ -73,6 +76,29 @@ baggage: other-vendor-value-1=foo;bar;baz, sentry-trace_id=771a43a4192642f0b136d See the [Payloads section](#payloads) for a complete list of key-value pairs that SDKs should propagate. +## Freezing Dynamic Sampling Context + +As mentioned above, in order to be able to make sampling decisions for entire traces, Dynamic Sampling Context must be the same across all transactions of a trace. + +**What does this mean for SDKs?** + +When starting a new trace, SDKs are no longer allowed to alter the DSC for that trace as soon as this DSC leaves the boundaries of the SDK for the first time. +The DSC is then considered "frozen". +DSC leaves SDKs in two situations: + +- When an outgoing request with a `baggage` header, containing the DSC, is made. +- When a transaction envelope containing the DSC is sent to Sentry + +When an SDK receives an HTTP request that was "instrumented" or "traced" by a Sentry SDK, the receiving SDK should consider the incoming DSC as instantly frozen. +Any values on the DSC should be propagated "as is" - this includes values like "environment" or "release". + +SDKs should recognize incoming requests as "instrumented" or "traced" when at least one of the following applies: + +- The incoming request has a `sentry-trace` header +- The incoming request has a `baggage` header containing one or more keys starting with "`sentry-`" + +After the DSC of a particular trace has been frozen, API calls like `set_user` or `set_transaction` should have no effect on the DSC. + ## Payloads Dynamic Sampling Context is sent to Sentry via the `trace` envelope header and is propagated to downstream SDKs via a baggage header. @@ -132,33 +158,33 @@ def has_sentry_value_in_baggage_header(request): # header of `request`, for which the key starts with "sentry-". Otherwise, it returns False. def on_incoming_request(request): - if has_header(request, "sentry-trace") and (not has_header(request, "baggage") or not has_sentry_value_in_baggage_header(request)): + if request.has_header("sentry-trace") and (not request.has_header("baggage") or not has_sentry_value_in_baggage_header(request)): # Request comes from an old SDK which doesn't support Dynamic Sampling Context yet # --> we don't propagate baggage for this trace - transaction.baggage_locked = True - transaction.baggage = {} - elif has_header(request, "baggage") and has_sentry_value_in_baggage_header(request): - transaction.baggage_locked = True - transaction.baggage = baggage_header_to_dict(request.headers.baggage) + current_transaction.dynamic_sampling_context_frozen = True + elif request.has_header("baggage") and has_sentry_value_in_baggage_header(request): + current_transaction.dynamic_sampling_context_frozen = True + current_transaction.dynamic_sampling_context = baggage_header_to_dict(request.headers.baggage) def on_outgoing_request(request): - if not transaction.baggage_locked: - transaction.baggage_locked = True - if not transaction.baggage: - transaction.baggage = {} - transaction.baggage = merge_dicts(collect_dynamic_sampling_context(), transaction.baggage) + if not current_transaction.dynamic_sampling_context_frozen: + current_transaction.dynamic_sampling_context_frozen = True + current_transaction.dynamic_sampling_context = merge_dicts(collect_dynamic_sampling_context(), current_transaction.dynamic_sampling_context) + + if not current_transaction.dynamic_sampling_context: + # Make sure there is at least an empty DSC set on transaction + # This is independent of whether it is locked or not + current_transaction.dynamic_sampling_context = {} - if has_header(request, "baggage"): + if request.has_header("baggage"): outgoing_baggage_dict = baggage_header_to_dict(request.headers.baggage) - merged_baggage_dict = merge_dicts(outgoing_baggage_dict, transaction.baggage) - merged_baggage_header = dict_to_baggage_header(merged_baggage_dict) - set_header(request, "baggage", merged_baggage_header) + merged_baggage_dict = merge_dicts(outgoing_baggage_dict, current_transaction.dynamic_sampling_context) + request.set_header("baggage", dict_to_baggage_header(merged_baggage_dict)) else: - baggage_header = dict_to_baggage_header(transaction.baggage) - set_header(request, "baggage", baggage_header) + request.set_header("baggage", dict_to_baggage_header(current_transaction.dynamic_sampling_context)) ``` -While there is no strict necessity for the `transaction.baggage_locked` flag yet, there is a future use case where we need it: +While there is no strict necessity for the `current_transaction.dynamic_sampling_context_frozen` flag yet, there is a future use case where we need it: We might want users to be able to set Dynamic Sampling Context values themselves. The flag becomes relevant after the first propagation, where Dynamic Sampling Context becomes immutable. When users attempt to set DSC afterwards, our SDKs should make this operation a noop. From ee324c54528444dbde7cb14ce6b98bd105023c24 Mon Sep 17 00:00:00 2001 From: Abhijeet Prasad Date: Fri, 24 Jun 2022 15:54:19 -0400 Subject: [PATCH 27/28] Apply suggestions from code review Co-authored-by: Lukas Stracke --- src/docs/sdk/performance/dynamic-sampling-context.mdx | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/src/docs/sdk/performance/dynamic-sampling-context.mdx b/src/docs/sdk/performance/dynamic-sampling-context.mdx index 1559efea39..94f01a10f2 100644 --- a/src/docs/sdk/performance/dynamic-sampling-context.mdx +++ b/src/docs/sdk/performance/dynamic-sampling-context.mdx @@ -197,9 +197,9 @@ These are not blockers to the adoption of the spec, but instead are here as cont ### The Temporal Problem Unlike `environment` or `release`, which should always be known to an SDK at initialization time, `user_id`, `user_segment`, and `transaction` (name) are only known after SDK initialization time. -This means that if a trace is propogated from a running transaction _BEFORE_ the user/transaction attributes are set, you'll get a portion of transactions in a trace that have different Dynamic Sampling Context than other portions, leading to dynamic sampling not working as expected for users. +This means that if a trace is propagated from a running transaction _BEFORE_ the user/transaction attributes are set, you'll get a portion of transactions in a trace that have different Dynamic Sampling Context than other portions, leading to dynamic sampling not working as expected for users. -Let's say we want to dynamically sample my browser application based on the `user_id`. +Let's say we want to dynamically sample a browser application based on the `user_id`. In a typical single page application (SPA), the user information has to be requested from some backend service before it can be set with `Sentry.setUser` on the frontend. Here's an example of that flow: @@ -211,9 +211,9 @@ Here's an example of that flow: - user service pings database service (propogates sentry-trace/baggage to database service) - database service continues trace by automatically creating sampling transaction - Page gets data from user service, calls `Sentry.setUser` and sets `user_id` -- Page makes HTTP requests to service A, service B, and service C (propogates sentry-trace/baggage to user service) - - traces propogated to service A, service B, and service C, so 3 child transactions -- Page finishes loading, finishing pageload transaction, which is sent to Sentry +- Page makes HTTP requests to service A, service B, and service C (propogates sentry-trace/baggage to services A, B and C) + - DSC is propogated with baggage to service A, service B, and service C, so 3 child transactions +- Page finishes loading, finishing `pageload` transaction, which is sent to Sentry In this case, the baggage that is propogated to the user service and the downstream database service _does not_ have the `user_id` value in it, because it was not yet set on the browser SDK. Therefore, when Relay tries to dynamically sample the user services and database services transactions based on `user_id`, it will not be able to. From 700e02038cd52e3a0ad62fdf9196b55a0f8a3487 Mon Sep 17 00:00:00 2001 From: Abhijeet Prasad Date: Fri, 24 Jun 2022 16:12:19 -0400 Subject: [PATCH 28/28] Clarify trace behaviour in Temporal Problem --- src/docs/sdk/performance/dynamic-sampling-context.mdx | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/src/docs/sdk/performance/dynamic-sampling-context.mdx b/src/docs/sdk/performance/dynamic-sampling-context.mdx index 94f01a10f2..217f8dbc63 100644 --- a/src/docs/sdk/performance/dynamic-sampling-context.mdx +++ b/src/docs/sdk/performance/dynamic-sampling-context.mdx @@ -32,6 +32,8 @@ This means that we need the exact same decision basis for all transactions belon In other words, all transactions of a trace need to hold all of the information to make a sampling decision, and that **information needs to be the same across all transactions of the trace**. We call the information we base sampling decisions on **"Dynamic Sampling Context"** or **"DSC"**. ++Currently, we can dynamically sample in two ways. First, we can do dynamic sampling on single transactions. For this process, Relay looks at the incoming event payload to make decisions. Second, we can do dynamic sampling across an entire trace. For this process, Relay relies on a **Dynamic Sampling Context**. + As a mental model: The head transaction in a trace determines the Dynamic Sampling Context for all following transactions in that trace. No information can be changed, added or deleted after the first propagation. @@ -197,7 +199,7 @@ These are not blockers to the adoption of the spec, but instead are here as cont ### The Temporal Problem Unlike `environment` or `release`, which should always be known to an SDK at initialization time, `user_id`, `user_segment`, and `transaction` (name) are only known after SDK initialization time. -This means that if a trace is propagated from a running transaction _BEFORE_ the user/transaction attributes are set, you'll get a portion of transactions in a trace that have different Dynamic Sampling Context than other portions, leading to dynamic sampling not working as expected for users. +This means that if a trace is propagated from a running transaction _BEFORE_ the user/transaction attributes are set, you'll get a portion of transactions in a trace that have different Dynamic Sampling Context than other portions, leading to _dynamic sampling across a trace_ not working as expected for users. Let's say we want to dynamically sample a browser application based on the `user_id`. In a typical single page application (SPA), the user information has to be requested from some backend service before it can be set with `Sentry.setUser` on the frontend. @@ -217,7 +219,7 @@ Here's an example of that flow: In this case, the baggage that is propogated to the user service and the downstream database service _does not_ have the `user_id` value in it, because it was not yet set on the browser SDK. Therefore, when Relay tries to dynamically sample the user services and database services transactions based on `user_id`, it will not be able to. -It will _ONLY_ be able to dynamically sample the pageload transaction, as well as the transactions from service A, service B, and service C with a rule based on `user_id`. +In addition, since the DSC is frozen after it's been sent, the DSC sent to service A, service B, and service C will not have `user_id` on it either. This means it also will not be dynamically sampled properly if there is a trace-wide DS rule on `user_id`. This problem exists for both `user_id` and `user_segment`, and it is because since we don't have the user information on some platforms/frameworks right as we initialize the SDK.