You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: develop-docs/application-architecture/dynamic-sampling/fidelity-and-biases.mdx
+15-4Lines changed: 15 additions & 4 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -15,12 +15,19 @@ A sample rate is a number in the interval `[0.0, 1.0]` that will determine the l
15
15
16
16
At the core of Dynamic Sampling there is the concept of **fidelity**, which translates to an overall **target sample rate** that should be applied across all events of an organization.
17
17
18
-
### Target Sample Rate Adjustment: Automatic Mode and Manual Mode
18
+
### Dynamic Sampling Modes
19
19
There are two available modes to govern the target sample rates for Dynamic Sampling: Automatic Mode and Manual Mode.
20
20
-**Automatic mode** dynamically manages the target sample rate for each project based on the target sample rate for the organization, prioritizing lower volume projects to increase visibility.
21
21
-**Manual mode** allows the user to set static target sample rates on a per-project basis that serve as the baseline sample rate before applying the dynamic biases outlined below. Target sample rates are not adjusted by the system.
22
22
23
-
Within this target sample rate, Dynamic Sampling can create a **bias toward more meaningful data**. This is achieved by constantly updating and communicating special rules to Relay, via a project configuration, which then applies targeted sampling to every event.
23
+
Internally, Automatic Mode is called Organization Mode, while Manual Mode is called Project Mode. The settings around the mode and the sample rates are implemented using organization and project options. The [DynamicSamplingMode object](https://github.com/getsentry/sentry/blob/9b98be6b97323a78809a829e06dcbef26a16365c/src/sentry/dynamic_sampling/types.py#L7-L12) defines the available modes and their string representations to be set in the options. The dynamic sampling mode is set using the organization option `sentry:sampling_mode`.
24
+
25
+
If `sentry:sampling_mode` == `organization`, the **organization** option `sentry:target_sample_rate` defines the organization target sample rate.
26
+
If `sentry:sampling_mode` == `project`, the **project** option `sentry:target_sample_rate` defines the project target sample rate for each project.
27
+
28
+
On switching between modes, the current target sample rates are preserved unless changed by the user explicitly. For example, if the user switches from Automatic Mode to Manual Mode, the current target sample rate for the organization is preserved by setting the project options `project:target_sample_rate` to the project target sample rates calculated during automatic mode. Conversely, if the user switches from Manual Mode to Automatic Mode, the project target sample rates are recalculated based on the overall organization target sample rate.
29
+
30
+
The [target sample rates are periodically recalibrated](https://github.com/getsentry/sentry/blob/9b98be6b97323a78809a829e06dcbef26a16365c/src/sentry/dynamic_sampling/rules/biases/recalibration_bias.py#L11-L44) to ensure that the overall target sample rate is met. This recalibration is done on a project level or organization level, depending on the dynamic sampling mode. Within the target sample rate, Dynamic Sampling **biases towards more meaningful data**. This is achieved by constantly updating and communicating special rules to Relay, via a project configuration, which then applies targeted sampling to every event.
24
31
25
32

26
33
@@ -82,7 +89,7 @@ Since the adoption of a release is not constant, we created a system of _decayin
82
89
83
90

84
91
85
-
The latest release bias uses a decaying rule to interpolate between a starting sample rate and an ending sample rate over a time window that is statically defined for each platform (the list of time to adoptions is define [here](https://github.com/getsentry/sentry/blob/master/src/sentry/dynamic_sampling/rules/helpers/time_to_adoptions.py#L26-L26). For example, Android has a bigger time window than Javascript because on average Android apps take more time to get adopted by users.
92
+
The latest release bias uses a decaying rule to interpolate between a starting sample rate and an ending sample rate over a time window that is statically defined for each platform (the list of time to adoptions is define [here](https://github.com/getsentry/sentry/blob/9b98be6b97323a78809a829e06dcbef26a16365c/src/sentry/dynamic_sampling/rules/helpers/time_to_adoptions.py#L25). For example, Android has a bigger time window than Javascript because on average Android apps take more time to get adopted by users.
86
93
87
94
### Prioritize Dev Environments
88
95
@@ -109,7 +116,11 @@ For prioritizing dev environments, we use a sample rate of `1.0` (100%), which r
109
116
### Prioritize Low Volume Transactions
110
117
This bias is used to prioritize low-volume transactions that can be drowned out by high-volume transactions. The goal is to rebalance sample rates of the individual transactions so that low-volume transactions are more likely to have representative samples. The bias is of type trace, which means that the transaction considered for rebalancing will be the root transaction of the trace.
111
118
112
-
In order to rebalance transactions, the system computes the counts of the transactions for each project and runs an algorithm that, given the sample rate of the organization and the counts of each transaction, computes a new sample rate for each transaction assuming an ideal distribution of the counts.
119
+
Prioritization of low volume projects works slightly differently depending on the dynamic sampling mode:
120
+
- In **Automatic Mode** (`sentry:sampling_mode` == `organization`), the organization target sample rate is used as the base sample rate for the balancing algorithm.
121
+
- In **Manual Mode** (`sentry:sampling_mode` == `project`), the project target sample rate is used as the base sample rate for the balancing algorithm.
122
+
123
+
In order to rebalance transactions, the system retrieves the counts of the transactions for each project and calculates a new sample rate for each transaction.
Copy file name to clipboardExpand all lines: develop-docs/application-architecture/dynamic-sampling/the-big-picture.mdx
+2-2Lines changed: 2 additions & 2 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -18,7 +18,7 @@ Dynamic Sampling currently operates on either spans or transactions, based on th
18
18
19
19
Dynamic Sampling occurs at the edge of our ingestion pipeline, precisely in [Relay](https://github.com/getsentry/relay).
20
20
21
-
When events arrive, in a simplified model, they go through the following steps (some of which won't apply if you self-host Sentry):
21
+
When events arrive, in a simplified model, they go through the following steps:
22
22
23
23
1.**Inbound data filters**: every event runs through inbound data filters as configured in project settings, such as legacy browsers or denied releases. Events dropped here are not counted towards quota and are not included in "total events" data.
24
24
2.**Quota enforcement**: Sentry charges for all further events sent in, before they are passed on to dynamic sampling.
@@ -65,7 +65,7 @@ For example, if there is a trace from `Project A` to `Project B` and `Project B`
65
65
66
66
Clients have their own [traces sample rate](https://docs.sentry.io/platforms/javascript/tracing/#configure). The client sample rate is a number in the range `[0.0, 1.0]` (from 0% to 100%) that controls **how many events arrive at Sentry**. While documentation will generally suggest a sample rate of `1.0`, for some use cases it might be better to reduce it.
67
67
68
-
Dynamic Sampling further reduces how many events get stored internally. **While most graphs and numbers in Sentry are based on total events**, accessing spans and tags requires stored events. The sample rates apply on top of each other.
68
+
Dynamic Sampling further reduces how many events get stored internally. **While most graphs and numbers in Sentry are based on metrics**, accessing spans and tags requires stored events. The sample rates apply on top of each other.
69
69
70
70
An example of client side sampling and Dynamic Sampling starting from 100k events which results in 15k stored events is shown below:
0 commit comments