Merge branch 'master' into docs/span-metrics-examples

Author: codyde

develop-docs/application-architecture/dynamic-sampling/extrapolation.mdx

@@ -3,34 +3,27 @@ title: Extrapolation
 sidebar_order: 5
 ---
 
-Dynamic sampling reduces the amount of data ingested, for reasons of both performance and cost. When configured, a fraction of the data is ingested according to the specified sample rate of a project: if you sample at 10% and initially have 1000 requests to your site in a given timeframe, you will only see 100 spans in Sentry. Without making up for the sample rate, any metrics derived from these spans will misrepresent the true volume of the application. When different parts of the application have different sample rates, there will even be a bias towards some of them, skewing the total volume towards parts with higher sample rates. This bias especially impacts numerical attributes like latency, reducing their accuracy. To account for this fact, Sentry uses extrapolation to smartly combine the data to account for sample rates.
+Client and server-side sampling reduces the amount of data ingested, to help with both performance and cost. When configured, a fraction of the data is ingested according to the specified sample rates within a project. For example, if you sample 10% of 1000 requests to your site in a given timeframe, you will see 100 spans in Sentry. 
 
-### Accuracy & Expressiveness
-What happens during extrapolation, how does one handle this type of data, and when is extrapolated data accurate and expressive? Let's start with some definitions:
+Without accounting for the lower request volume due to the sample rate, any metrics derived from these spans will misrepresent the true volume of the application. Perhaps more importantly, when different parts of your application have different sample rates, attention may be skewed with a bias towards parts with higher sample rates. This bias especially impacts numerical attributes like latency, reducing their accuracy. To account for this, Sentry uses extrapolation to a) derive a "true" volume of each part of your application and b) combine the extrapolated data from different parts of the application to provide a more wholistic view of the application's performance.
 
-- **Accuracy** refers to data being correct. For example, the measured number of spans corresponds to the actual number of spans that were executed. As sample rates decrease, accuracy also goes down because minor random decisions can influence the result in major ways.
-- **Expressiveness** refers to data being able to express something about the state of the observed system. Expressiveness refers to the usefulness of the data for the user in a specific use case.
-
-### Modes
-Given these properties, there are two modes that can be used to view data in Sentry: default mode and sample mode.
-
-- **Default mode** extrapolates the ingested data as outlined below.
-- **Sample mode** does not extrapolate and presents exactly the data that was ingested.
+### Benefits of Extrapolation
+For high-volume organizations, sampling is a way to control costs and reduce volume, as well as reduce the amount of redundant data sent to Sentry. Extrapolation is a way to account for the lower request volume due to the sample rate, and to provide a more wholistic view of the application's performance. Here are some of the benefits of extrapolation:
 
-Depending on the context and the use case, one mode may be more useful than the other. Generally, default mode is useful for all queries that aggregate on a dataset of sufficient volume. As absolute sample size decreases below a certain limit, default mode becomes less and less expressive. There are scenarios where the user needs to temporarily switch between modes, for example, to examine the aggregate numbers first and dive into the number of samples for investigation. In both modes, the user may investigate single samples to dig deeper into the details.
+- **The numbers correspond to the real world**: When data is sampled, there is some math you need to do to infer what the real numbers are, e.g., when you have 1000 samples at 10% sample rate, there are 10000 requests to your application. With extrapolation, you don't have to know your sample rate to understand what your application is actually doing. Instead, while viewing charts, you see the real behavior without additional knowledge or math required on your end.
 
-### Benefits of Extrapolation
-At first glance, extrapolation may seem unnecessarily complicated. However, for high-volume organizations, sampling is a way to control costs and egress volume, as well as reduce the amount of redundant data sent to Sentry. Why don't we just show the user the data they send? We don't just extrapolate for fun; it actually has some major benefits for the user:
+- **Steady timeseries when sample rates change**: Whenever you change sample rates, both the count and possibly the distribution of the values will change in some way. When you switch the sample rate from 10% to 1% for whatever reason, there will be a sudden change in all associated metrics. Extrapolation corrects for this, so your graphs are steady, and your alerts track on the same data, regardless of the sample rate.
 
-- **The numbers correspond to the real world**: When data is sampled, there is some math you need to do to infer what the real numbers are, e.g., when you have 1000 samples at 10% sample rate, there are 10000 requests to your application. With extrapolation, you don't have to know your sample rate to understand what your application is actually doing. Instead, you get a view of the real behavior without additional knowledge or math required on your end.
+- **Combining different sample rates**: When your endpoints don't have the same sample rate, how are you supposed to know the true p90 when one of your endpoints is sampled at 1% and another at 100%, but all you get is the aggregate of the samples? Extrapolation calculates the true p90 by combining the data from all endpoints, weighted by the sample rate.
 
-- **Steady timeseries when sample rates change**: Whenever you change sample rates, both the count and possibly the distribution of the values will change in some way. When you switch the sample rate from 10% to 1% for whatever reason, there will be a sudden change in all associated metrics. Extrapolation corrects for this, so your graphs are steady, and your alerts don't fire when this happens.
-- **Combining different sample rates**: When your endpoints don't have the same sample rate, how are you supposed to know the true p90 when one of your endpoints is sampled at 1% and another at 100%, but all you get is the aggregate of the samples?
+**Note:** When a sample rate is too low, there may be a low confidence in the extrapolated data. When this is the case, you should consider increasing the sample rate, widening your time range or filter, or turning off extrapolation.  
 
 ## How Does Extrapolation Work?
+![extrapolation =1000x](./images/extrapolated_data_chart.png)
+
 ### Aggregates
 
-Sentry allows the user to aggregate data in different ways - the following aggregates are generally available, along with whether they are extrapolatable or not:
+Sentry allows you to aggregate data in different ways - the following aggregates are generally available, along with whether they are extrapolatable or not:
 
 | **Aggregate** | **Can be extrapolated?** |
 | --- | --- |
@@ -42,12 +35,12 @@ Sentry allows the user to aggregate data in different ways - the following aggre
 | max | no |
 | count_unique | no |
 
-Each of these aggregates has its own way of dealing with extrapolation, due to the fact that e.g., counts have to be extrapolated in a slightly different way from percentiles. While `min` and `max` are technically percentiles, we currently do not offer extrapolation due to the decreased stability of extreme aggregates when sampling. For example, the `p50` will also be more stable than the `p99`; the `min` and `max` are just extreme cases.
+Each of these aggregates has its own way of dealing with extrapolation, due to the fact that, for example, counts have to be extrapolated in a slightly different way from percentiles. While `min` and `max` are technically percentiles, we currently do not offer extrapolation due to the decreased stability of extreme aggregates when sampling. For example, the `p50` will also be more stable than the `p99`; the `min` and `max` are just extreme cases.
 
 ### Extrapolation for different aggregates
 To extrapolate, sampling weights are calculated as `1/sample rate`. The sampling weights of each row are then used in the following ways:
 
-- **Count**: Calculate a sum of the sampling weight
+- **Count**: Calculate a sum of the sampling weight.
 Example: the query `count()` becomes `round(sum(sampling_weight))`.
 - **Sum**: Multiply each value with `sampling_weight`.
 Example: the query `sum(foo)` becomes `sum(foo * sampling_weight)`
@@ -56,7 +49,7 @@ Example: the query `avg(foo)` becomes `avgWeighted(foo, sampling_weight)`
 - **Percentiles**: Calculate the weighted percentiles with sampling weight.
 Example: the query `percentile(0.95)(foo)` becomes `weightedPercentile(0.95)(foo, sampling_weight)`.
 
-As long as there are sufficient samples, the sample rate itself does not matter as much, but due to the extrapolation mechanism, what would be a fluctuation of a few samples may turn into a much larger absolute impact, e.g., in terms of the view count. Of course, when a site gets billions of visits, a fluctuation of 100,000 via the noise introduced by a sample rate of 0.00001 is not as critical.
+As long as there are sufficient samples, the sample rate itself does not matter as much, but due to the extrapolation mechanism, a fluctuation of a few samples in a small sample set may turn into a much larger absolute impact. When a site gets billions of visits, a fluctuation of 100,000 via the noise introduced by a sample rate of 0.00001 is not as critical.
 
 ## How to deal with extrapolation in the product?
 
@@ -72,20 +65,16 @@ In new product surfaces, the question of whether to use extrapolated vs. non-ext
 - Does the user care more about a truthful estimate of the aggregate data or about the actual events that happened?
     - Some scenarios, like visualizing metrics over time, are based on aggregates, whereas a case of debugging a specific user's problem hinges on actually seeing the specific events. The best mode depends on the intended usage of the product.
 
-### Switching to Sample Mode
-Sample mode is designed to help users investigate specific events. Here are two common scenarios where it makes the most sense:
-
-1. **When both sample rate and event volume are low**: Extrapolation becomes less reliable in these cases. You can either increase your sample rate to improve accuracy, or switch to sample mode to examine the actual events - both are valid approaches depending on the user's needs.
-2. **When you have a high sample rate but still see low event volumes**: In this case, increasing the sample rate won't help capture more data, and sample mode will give you a clearer picture of the events you do have.
+### When to Turn Off Extrapolation
+While extrapolation is usually helpful, if you are exclusively looking to confirm your sampling usage or look exclusively at sampled data, you can turn off extrapolation.
 
-### Opting Out of Extrapolation
-Users may want to opt out of extrapolation for different reasons. It is always possible to set the sample rate for specific events to 100% and therefore send all data to Sentry, implicitly opting out of extrapolation and behaving in the same way as sample mode. Depending on their configuration, users may need to change Dynamic Sampling settings or their SDK's traces sampler callback for this.
+You can always increase your sample rate to 100% to examine all events if traffic is too low to be otherwise useful via extrapolation or sampling.
 
 ### Confidence
-When users filter on data that has a very low count but also a low sample rate, yielding a highly extrapolated but low-sample dataset, developers and users should be careful with the conclusions they draw from the data. The storage platform provides confidence intervals along with the extrapolated estimates for the different aggregation types to indicate when there is elevated uncertainty in the data. These types of datasets are inherently noisy and may contain misleading information. When this is discovered, the user should either be very careful with the conclusions they draw from the aggregate data or switch to non-default mode for investigation of the individual samples.
+When there is not enough data to properly extrapolate, Sentry will indicate low confidence in the data. If this message is not present, Sentry has a high confidence in the data.
 
 ## **Conclusion**
 
 - Extrapolation offers benefits in many parts of the product but brings some inherent complexity.
-- Some aggregates can be extrapolated, others cannot - we may add the capability to additional aggregates in the future.
-- A lot of care should be taken about how to expose extrapolation and especially switching of the modes to the user.
+- Some aggregates can be extrapolated, others cannot.
+- A lot of care should be taken about how to expose extrapolation in the product, and especially in offering switching modes to a user.

develop-docs/application-architecture/dynamic-sampling/images/extrapolated_data_chart.png

@@ -97,7 +97,7 @@ While these [data categories](https://github.com/getsentry/relay/blob/master/rel
   - `profile_chunk`: Continuous Profiling chunks
   - `replay`: Session Replays
   - `feedback`: User Feedbacks
-  - `metric_bucket`: Sentry Metrics sent via the `statsd` or `metrics` items. The `namespaces` component of the *quota_limit* defines which namespace(s) will be affected.
+  - `trace_metric`: Metric type events
   - `internal`: a sentry/system internal event[^internal]
 
 - **Scope**: The unit / model in Sentry that quotas are enforced for.

develop-docs/sdk/expected-features/rate-limiting.mdx

@@ -396,7 +396,7 @@ A new data category for logs has been added to Relay, `log_item`. Both the `log`
 
 ### Buffering
 
-Logs should be buffered before being sent. SDKs should keep a buffer of logs on the client (so you can have logs from multiple traces in the buffer) that flushes out based on some kind of condition. We recommend to use follow the [batch processor specification outlined](/sdk/telemetry/spans/batch-processor/) in the develop docs, but you should choose the approach that works best for your platform. When starting initial development on the SDK you can choose a simple approach to buffer like flushing logs if the buffer length exceeds 100 items, or if 5 seconds have passed.
+Logs should be buffered before being sent. SDKs should keep a buffer of logs on the client (so you can have logs from multiple traces in the buffer) that flushes out based on some kind of condition. We recommend to follow the [telemetry buffer specification outlined](/sdk/telemetry/telemetry-buffer/) in the develop docs, but you should choose the approach that works best for your platform. When starting initial development on the SDK you can choose a simple approach to buffer like flushing logs if the buffer length exceeds 100 items, or if 5 seconds have passed.
 
 SDKS should NOT release logging capabilities to users if a buffering implementation has not been added to their SDK when adding logging APIs.
 

develop-docs/sdk/telemetry/logs.mdx

@@ -120,7 +120,7 @@ const checkoutSpan = Sentry.startSpan({ name: 'on-checkout-click', attributes: {
 const validationSpan = Sentry.startSpan({ name: 'validate-shopping-cart'})
 startFormValidation().then((result) => {
   validationSpan.setAttribute('valid-form-data', result.success);
-  processSpan.end();
+  validationSpan.end();
 })
 
 const processSpan = Sentry.startSpan({ name: 'process-order', parentSpan: checkoutSpan});
@@ -129,7 +129,7 @@ processOrder().then((result) => {
   processSpan.end();
 }).catch((error) => {
   processSpan.setStatus('error');
-  processSpan.setAttribute('error', error.message);
+  processSpan.setAttribute('order-processed', 'error');
   processSpan.end();
 });
 

develop-docs/sdk/telemetry/spans/span-api.mdx

@@ -52,7 +52,7 @@ The envelope item payload must contain an `items` array with span one and up to
       "span_id": "438f40bd3b4a41ee",
       "name": "GET /users",
       "status": "ok",
-      "is_remote": true,
+      "is_segment": true,
       "kind": "server",
       "start_timestamp": 1742921669.158209,
       "end_timestamp": 1742921669.180536,
@@ -114,7 +114,7 @@ The envelope item payload must contain an `items` array with span one and up to
       "span_id": "f1196292f76e45c0",
       "name": "app.handle",
       "status": "ok",
-      "is_remote": false,
+      "is_segment": false,
       "kind": "server",
       "start_timestamp": 1742921669.178306,
       "end_timestamp": 1742921669.180484,
@@ -148,7 +148,7 @@ The envelope item payload must contain an `items` array with span one and up to
 | `parent_span_id` | string | No | 16-character hexadecimal string (a valid uuid4 without dashes) |
 | `name` | string | Yes | A low cardinality description of what the span represents (e.g., "GET /users", "database.query") |
 | `status` | string | Yes | Status of the span operation. Either `"ok"` or `"error"` |
-| `is_remote` | boolean | Yes | Whether the SpanContext creating the span was received from somewhere else or locally generated |
+| `is_segment` | boolean | Yes | Whether the span is a segment span |
 | `kind` | string | Yes | The kind of span. Values: `"server"`, `"client"`, `"producer"`, `"consumer"`, `"internal"` |
 | `start_timestamp` | number | Yes | Unix timestamp (with fractional microseconds) when the span was started |
 | `end_timestamp` | number | Yes | Unix timestamp (with fractional microseconds) when the span was ended |

develop-docs/sdk/telemetry/spans/span-protocol.mdx

@@ -0,0 +1,7 @@
+---
+title: Backend Telemetry Buffer
+description: Detailed backend telemetry buffer design.
+sidebar_order: 1
+---
+
+To be defined — full spec lives here.

develop-docs/sdk/telemetry/telemetry-buffer/backend-telemetry-buffer.mdx

@@ -0,0 +1,7 @@
+---
+title: Browser Telemetry Buffer
+description: Detailed browser telemetry buffer design.
+sidebar_order: 2
+---
+
+To be defined — full spec lives here.

develop-docs/sdk/telemetry/telemetry-buffer/browser-telemetry-buffer.mdx

@@ -0,0 +1,7 @@
+---
+title: GDX Telemetry Buffer
+description: Detailed GDX telemetry buffer design.
+sidebar_order: 3
+---
+
+To be defined — full spec lives here.