Skip to content
Closed
Show file tree
Hide file tree
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions develop-docs/sdk/telemetry/spans/index.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
---
title: Spans
sidebar_order: 8
---

<PageGrid />
Empty file.
74 changes: 74 additions & 0 deletions develop-docs/sdk/telemetry/spans/span-api.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,74 @@
---
title: Span API
---

# Span API

Spans are measuring the duration of certain operations in an application.
The topmost member of a span tree is called the root span. This span has no parent span and groups together its children with a representative name for the entire operation, such as `GET /` in case of a request to a backend application.

### Creating a root span

The SDK must expose a method for creating a root span. The user must be able to set certain properties on this root span, such as its name, the type of operation (`op`) and others.

```js
span = sentry.tracing.startSpan()
->setName('GET /')
->setOp('http.server')

span.end()
```

### Creating nested spans

To create nested spans, the SDK must expose an explicit way for a user to perform this task.

Additionally, the SDK may expose alternative APIs to create nested spans, such as allowing a user to wrap an operation into a callback or apply a decorator to certain blocks. These alternative APIs must never create a root span and no-op if no parent span is present.

```js
childSpan = span.startChild()
->setName('authentication middleware')
->setOp('middleware.handle')

childSpan.end()
```

### Setting the span status

A span has two statuses, `ok` and `error`. By default, the status of a span is set to `ok`.
The SDK must allow a user to modify the status of a span.

```js
span.setStatus('error')
```

### Setting span attributes

The SDK must expose a method to allow a user to set data attributes onto a span.
These attributes should use pre-defined keys whenever possible.

```js
span.setAttribute(SpanAttributes.HTTP_METHOD, 'GET')
span.setAttribute(SpanAttributes.HTTP_RESPONSE_STATUS_CODE, 200)
```

### Receiving the trace parent

The SDK must expose a method to receive the baggage string.

```js
traceparent = span.getTraceparent()
```

### Receiving the baggage

The SDK must expose a method to receive the baggage string.

```js
baggage = span.getBaggage()
```

### Additional, optional span APIs

`span.setStartTimestamp()` - overwrite the span's start time
`span.setEndTimestamp()` - overwrites the span's end time
15 changes: 15 additions & 0 deletions develop-docs/sdk/telemetry/spans/span-properties.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
---
title: Span Propertires
---

# Span Propertires

Instead of spans containing tags, context, and data, we'll unify all these properties into a new “attributes” property.
Similar to OTel's semantic conventions, we'll add special meaning to certain attribute keys, such as `sentry.release`, `sentry.op`, etc.

```js
span.setAttribute('http.request.method', 'GET')
span.setAttribute('user.email', '[email protected]')
```

We'll map these attributes to their respective existing property in Relay to ease the work required for the product during the transition period.
63 changes: 63 additions & 0 deletions develop-docs/sdk/telemetry/spans/span-protocol.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,63 @@
---
title: Span Protocol
---

# Span protocol

We'll introduce a new “span” envelope item, which the SDK uses to emit a segment span and its children. And in the future, a batch of spans.
The payload of each envelope item follows the [OpenTelemetry Protocol](https://opentelemetry.io/docs/specs/otel/protocol/), which introduced typed attributes and will ease the conversion in our POtel SDKs.

```json
{
"event_id":"9ec79c33ec9942ab8353589fcb2e04dc"
}
{
"type": "span"
}
{
"traceId": "32d3c7cb501fbddbe3ce1016a72d50b5",
"spanId": "e91d37480970523b",
"name": "GET /",
"startTime": "1544712660",
"endTime": "1544712680",
"attributes": [
{
"key": "sentry.op",
"value": {
"stringValue": "http.sever",
}
},
{
"key": "http.response.status_code",
"value": {
"intValue": "200",
}
}
}
}
{
"type": "span"
}
{
"traceId": "32d3c7cb501fbddbe3ce1016a72d50b5",
"spanId": "6b22b3af586e777a",
"parentSpanId": "e91d37480970523b",
"name": "UserMiddleware",
"startTimeUnix": "1544712665",
"endTimeUnix": "1544712675",
"attributes": [
{
"key": "sentry.op",
"value": {
"stringValue": "middleware.handle",
}
},
{
"key": "user.email",
"value": {
"stringValue": "[email protected]",
}
}
}
}
```
150 changes: 150 additions & 0 deletions develop-docs/sdk/telemetry/spans/span-sampling.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,150 @@
---
title: Span Sampling
---

# Span Sampling

With the metrics product shifting to a sampling based solution, extrapolation is of the utter most importance to being able to display reliable metrics to our users. We want to account for client sampling in addition to server sampling.
This requires the SDKs to always report the correct sampling rates in each tracing related envelope send to Sentry.
Directional, the goal is to create complete traces by default and wherever possible. We will not optimise for spent-control.

We historically exposed many ways to our users to remove certain transactions or spans from being emitted to Sentry. This resulted in convoluted SDK APIs, weird edge cases in the product and an overall bad user experience. More importantly, these sampling controls will contribute to vastly wrong metrics being extracted from span attributes, hence we need to rework those:

- `beforeSendTransaction` and `beforeSendSpan` will be replaced with `beforeSendSpans`, which encourages users to *mutate* spans, but they cannot be dropped through this callback.
- All SDK integrations that create spans, need to be able to be turned off via a config flag for the purpose of noise reduction or via a new `ignoreSpans` options that accepts a glob pattern.
- Sampling happens exclusively via `tracesSampleRate` or `tracesSampler`. We need to make sure to always prefer the parent sampling decision, either via explicit docs or a new argument for the `tracesSampler` or SDK option.
- Trace propagation is aware of applications or at least organizations and prevents “leaking” traces across this boundary.

## `beforeSendSpans`

The primary use-case for this hook will be data scrubbing or mutating certain properties of spans.

We are likely only allow to mutate the span’s name, timestamps, status and most attributes. Trace ID, span ID, parent span ID are immutable, as well as certain span attributes, such as segment ID.

It is yet to be defined which arguments will be passed into the callback or how the hook behaves with transaction envelopes.

## Span configuration

To reduce noise, users might want to disable certain integrations creating spans. This should ideally be exposed as a global config or at an integration level. Additionally, a new `ignoreSpans` option will allow users to not emit certain spans based on their name & attributes.

```jsx
Sentry.init({
dsn: 'foo@bar',
ignoreSpans: [
'GET /about',
'events.signal *',
],
ignoreSpans: (name, attributes) {
if (
name === 'server.request' &&
attributes['server.address'] === 'https://sentry.io'
) {
return true
}
},
integrations: [
fsIntegration: {
ignoreSpans: [
'fs.read',
],
readSpans: true,
writeSpans: false,
}
]
})
```

## Parent Sampling Decision

In today's SDKs, a parent sampling decision received via a `sentry-trace` header or similar can be overruled by setting a `tracesSampler`.
As we need to optimize for trace completeness, we need to explicitly call out the impact of the sampler or change the behaviour to always use the parent’s decision unless explicitly opted-out.

```jsx
// Explict docs
Sentry.init({
tracesSampler: ({ name, attributes, parentSampled }) => {
// Continue trace decision, if there is any parentSampled information
// This is crucial for complete traces
if (typeof parentSampled === "boolean") {
return parentSampled;
}

// Else, use default sample rate (replacing tracesSampleRate)
return 0.5;
},
})

// Not chosen - New top level option
Sentry.init({
ignoreParentSamplingDecision: true,
tracesSampler: ({ name, attributes, parentSampled }) => {
// Do not sample health checks ever
if (name.includes("healthcheck")) {
// Drop this transaction, by setting its sample rate to 0%
return 0.0;
}

// Else, use default sample rate (replacing tracesSampleRate)
return 0.2;
},
})
```

## Parent Sampling Origins

In order to filter out unrelated 3rd party services that are making requests to a Sentry instrumented app containing a `sentry-trace` header, we’ll implement RFC https://github.com/getsentry/rfcs/pull/137. This feature might be enabled by default if the:

- SDK knows its org
- The incoming baggage header contains a `sentry-org` entry

## Sampling Seed in DSC

To increase the chance of capturing complete traces when users return a new sample rate `tracesSampler` in backend services, we propagate the random value used by the SDK for computing the sampling decision instead of creating a new random value in every service. Therefore, across a trace every SDK uses the *same* random value.

### Behavior

A user can also override the parent sample rate in traces sampler. For example, a backend service has a `tracesSampler` that overrides frontend traces. This leads to three scenarios:

- The new (backend) sample rate is lower than the parent’s (frontend): All traces captured in the backend are complete. There are additional partial traces for the frontend.
- The new (backend) sample rate is higher than the parent’s (fronted): All traces propagated from the frontend are complete. There are additional partial traces for the backend.
- Both sample rates are equal: All traces are complete, the sampling decision is fully inherited.

The behavior of the static `tracesSampleRate` without the use of `tracesSampler` does not change. We continue to fully inherit sampling decisions for propagated traces and create a new one for started traces. In the future, we might change the default behavior of `tracesSampleRate`, too.

### SDK Spec

- sentry baggage gains a new field `sentry-sample_rand`
- when a new trace is started, `sentry-sample_rand` is filled with a truly random number. this also applies when the trace’s sample rate is 1.0
- for inbound traces without a `sentry-sample_rand` (from old SDKs), the SDK inserts a new truly random number on-the-fly.
- sampling decisions in the SDK that currently compare `sentry-sample_rand` from the trace instead of `math.random()` with the sample rate.
- when traces sampler is invoked, this also applies to the return value of traces sampler. ie. `trace["sentry-sample_rand"] < tracesSampler(context)`
- otherwise, when the SDK is the head of a trace, this applies to sample decisions based on `tracesSampleRate` , i.e. ``trace["sentry-sample_rand"] < config.tracesSampleRate`
- There is no more `math.random()` directly involved in any sampling decision.
- in traces sampler, the most correct way to inherit parent sampling decisions is now to return the parent’s sample **rate** instead of the **decision** as float (`1.0`). This way, we can still extrapolate counts correctly.

```jsx
tracesSampler: ({ name, parentSampleRate }) => {
// Inherit the trace parent's sample rate if there is one. Sampling is deterministic
// for one trace, i.e. if the parent was sampled, we will be sampled too at the same
// rate.
if (typeof parentSampleRate === "number") {
return parentSampleRate;
}

// Else, use default sample rate (replacing tracesSampleRate).
return 0.5;
},

```

- if the `sentry-sample_rate` (`parentSampleRate`) is not available for any reason for an inbound trace, but the trace has the sampled flag set to `true`, the SDK injects `parentSampleRate: 1.0` into the callback.

## Baggage Freeze

---

We accept partial traces under the assumption that the transaction name is mostly changed early in the request cycle.

# External Resources

https://opentelemetry.io/docs/specs/otel/trace/tracestate-probability-sampling-experimental/
Loading