From 36d4d77a1b5e2972f5061f0f8d1690303a834100 Mon Sep 17 00:00:00 2001 From: Philipp Hofmann Date: Wed, 5 Nov 2025 09:48:34 +0100 Subject: [PATCH 1/2] feat(develop-docs): Add telemetry buffer page Add the WIP generic page for telemetry buffers and migrate the batch-processor content. --- develop-docs/sdk/telemetry/logs.mdx | 2 +- ...tch-processor.mdx => telemetry-buffer.mdx} | 46 ++++++++++++++++++- redirects.js | 4 ++ 3 files changed, 50 insertions(+), 2 deletions(-) rename develop-docs/sdk/telemetry/{spans/batch-processor.mdx => telemetry-buffer.mdx} (74%) diff --git a/develop-docs/sdk/telemetry/logs.mdx b/develop-docs/sdk/telemetry/logs.mdx index 320790bb9c0e3..76f9f56f60c59 100644 --- a/develop-docs/sdk/telemetry/logs.mdx +++ b/develop-docs/sdk/telemetry/logs.mdx @@ -396,7 +396,7 @@ A new data category for logs has been added to Relay, `log_item`. Both the `log` ### Buffering -Logs should be buffered before being sent. SDKs should keep a buffer of logs on the client (so you can have logs from multiple traces in the buffer) that flushes out based on some kind of condition. We recommend to use follow the [batch processor specification outlined](/sdk/telemetry/spans/batch-processor/) in the develop docs, but you should choose the approach that works best for your platform. When starting initial development on the SDK you can choose a simple approach to buffer like flushing logs if the buffer length exceeds 100 items, or if 5 seconds have passed. +Logs should be buffered before being sent. SDKs should keep a buffer of logs on the client (so you can have logs from multiple traces in the buffer) that flushes out based on some kind of condition. We recommend to follow the [telemetry buffer specification outlined](/sdk/telemetry/telemetry-buffer/) in the develop docs, but you should choose the approach that works best for your platform. When starting initial development on the SDK you can choose a simple approach to buffer like flushing logs if the buffer length exceeds 100 items, or if 5 seconds have passed. SDKS should NOT release logging capabilities to users if a buffering implementation has not been added to their SDK when adding logging APIs. diff --git a/develop-docs/sdk/telemetry/spans/batch-processor.mdx b/develop-docs/sdk/telemetry/telemetry-buffer.mdx similarity index 74% rename from develop-docs/sdk/telemetry/spans/batch-processor.mdx rename to develop-docs/sdk/telemetry/telemetry-buffer.mdx index 2d28342770699..bf395b6247350 100644 --- a/develop-docs/sdk/telemetry/spans/batch-processor.mdx +++ b/develop-docs/sdk/telemetry/telemetry-buffer.mdx @@ -1,5 +1,7 @@ --- -title: Batch Processor +title: Telemetry Buffer +redirect_from: + - /sdk/telemetry/spans/batch-processor/ --- @@ -10,6 +12,32 @@ title: Batch Processor This document uses key words such as "MUST", "SHOULD", and "MAY" as defined in [RFC 2119](https://www.ietf.org/rfc/rfc2119.txt) to indicate requirement levels. +The telemetry buffer sits between the client and the transport, temporarily storing telemetry data such as spans and logs. Its main goal is to efficiently batch data to reduce the number of outgoing HTTP requests and Sentry envelopes. Without buffering, each span or log would trigger its own request, quickly overwhelming our backends. + +```mermaid +flowchart LR + Client[Client] -->|capture telemetry data| Buffer[TelemetryBuffer] + Buffer -->|send envelope| Transport[Transport] +``` + +Because telemetry workloads and platform constraints vary widely, buffer requirements differ across environments. For example, backend SDKs need high throughput and backpressure management to handle large data volumes. Mobile SDKs have lower throughput and don't need to worry much about backpressure, but they do need to minimize data loss in the event of abnormal process termination. Browser and GDX SDKs also have different requirements. + +Therefore, we recommend implementing different types of telemetry buffers tailored to the platform's needs. As of Nov 5th, 2025, this page is under development, and we're currently refining the requirements for different platforms: + +* [BatchProcessor V0](#batchprocessor-v0): The BatchProcessor V0 outlines the initial requirements, primarily for logs, to avoid sending numerous HTTP requests. This section exists only as a placeholder until we fully spec out the other telemetry buffers. +* [Backend Telemetry Buffer](#backend-telemetry-buffer): To be defined +* [Mobile Telemetry Buffer](#mobile-telemetry-buffer): To be defined +* [Browser Telemetry Buffer](#browser-telemetry-buffer): To be defined +* [GDX Telemetry Buffer](#gdx-telemetry-buffer): To be defined + +# Common Requirements + +To be defined. Things like common API, client reports, etc. + +# BatchProcessor V0 + +## Overview + The BatchProcessor batches spans and logs into one envelope to reduce the number of HTTP requests. When an SDK implements span streaming or logs, it MUST use a BatchProcessor, which is similar to [OpenTelemetry's Batch Processor](https://github.com/open-telemetry/opentelemetry-collector/blob/main/processor/batchprocessor/README.md). The BatchProcessor holds logs and finished spans in memory and batches them together into envelopes. It uses a combination of time and size-based batching. When writing this, the BatchProcessor only handles spans and logs, but an SDK MAY use it for other telemetry data in the future. ## Specification @@ -95,3 +123,19 @@ Scenario: 1 span added application crashes And loses the spans in the BatchProcessor ``` + +# Backend Telemetry Buffer + +To be defined + +# Mobile Telemetry Buffer + +To be defined + +# Browser Telemetry Buffer + +To be defined + +# GDX Telemetry Buffer + +To be defined diff --git a/redirects.js b/redirects.js index a8a7135a1ca0e..eeab258d7c892 100644 --- a/redirects.js +++ b/redirects.js @@ -74,6 +74,10 @@ const developerDocsRedirects = [ source: '/sdk/replays/:path*', destination: '/sdk/telemetry/replays/:path*', }, + { + source: '/sdk/telemetry/spans/batch-processor/', + destination: '/sdk/telemetry/telemetry-buffer/', + }, { source: '/sdk/setup-wizards/:path*', destination: '/sdk/expected-features/setup-wizards/:path*', From 2ab0cc1c06671aad5c4702dae858e45ad34e113d Mon Sep 17 00:00:00 2001 From: Philipp Hofmann Date: Wed, 5 Nov 2025 10:12:49 +0100 Subject: [PATCH 2/2] improvements --- develop-docs/sdk/telemetry/telemetry-buffer.mdx | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/develop-docs/sdk/telemetry/telemetry-buffer.mdx b/develop-docs/sdk/telemetry/telemetry-buffer.mdx index bf395b6247350..ee5179f123a47 100644 --- a/develop-docs/sdk/telemetry/telemetry-buffer.mdx +++ b/develop-docs/sdk/telemetry/telemetry-buffer.mdx @@ -12,12 +12,13 @@ redirect_from: This document uses key words such as "MUST", "SHOULD", and "MAY" as defined in [RFC 2119](https://www.ietf.org/rfc/rfc2119.txt) to indicate requirement levels. -The telemetry buffer sits between the client and the transport, temporarily storing telemetry data such as spans and logs. Its main goal is to efficiently batch data to reduce the number of outgoing HTTP requests and Sentry envelopes. Without buffering, each span or log would trigger its own request, quickly overwhelming our backends. +The telemetry buffer sits between the client and the transport, temporarily buffering high-volume telemetry data such as spans and logs. The client SHOULD continue to pass low-volume telemetry data, such as events, directly to the transport. The telemetry buffer aims to efficiently batch data to reduce the number of outgoing HTTP requests and Sentry envelopes. Without buffering, each span or log would trigger its own request, quickly overwhelming our backends. ```mermaid flowchart LR - Client[Client] -->|capture telemetry data| Buffer[TelemetryBuffer] - Buffer -->|send envelope| Transport[Transport] + Client[Client] -->|high-volume telemetry data| Buffer[TelemetryBuffer] + Buffer -->|buffered telemetry data| Transport[Transport] + Client -->|low-volume telemetry data| Transport ``` Because telemetry workloads and platform constraints vary widely, buffer requirements differ across environments. For example, backend SDKs need high throughput and backpressure management to handle large data volumes. Mobile SDKs have lower throughput and don't need to worry much about backpressure, but they do need to minimize data loss in the event of abnormal process termination. Browser and GDX SDKs also have different requirements.