Merge branch 'master' into cursor/set-up-flutter-web-for-chrome-extensions-1df2

Author: buenaflor

develop-docs/integrations/index.mdx

@@ -4,4 +4,70 @@ description: How to setup Sentry integrations. Integrations connect the Sentry s
 sidebar_order: 80
 ---
 
-<PageGrid />
+<PageGrid/>
+
+
+
+## Integration Backup and Restore Scripts
+
+### Overview
+
+When working on integration development locally, your database contains important configuration that makes integrations work properly. If you run `make reset-db` or need to delete your local environment, you lose all this setup and have to configure integrations from scratch.
+There are two scripts that help you backup and restore your local Sentry integration configuration and setup data.
+
+These scripts allow you to:
+- **Backup**: Save your current integration state to a JSON file
+- **Restore**: Load the integration state back into a clean database
+
+### What Data is Backed Up
+
+The scripts handle the following Sentry models in the correct dependency order:
+- `IdentityProvider` - Authentication provider configurations
+- `Integration` - Integration instances and settings  
+- `Identity` - User identity mappings
+- `OrganizationIntegration` - Organization-specific integration configurations
+
+### Prerequisites
+
+- Sentry development environment set up locally
+- Python environment with Sentry dependencies installed
+- Access to your local Sentry database
+
+### Script Files
+
+There are two scripts (exist in `sentry` and `getsentry`):
+
+- `save_integration_data` - Backs up integration data
+- `load_integration_data` - Restores integration data
+
+### Usage Instructions
+
+#### Step 1: Save Your Integration Data
+
+Before running `make reset-db` or making any destructive changes, backup your current integration state:
+
+```bash
+# Navigate to your Sentry project directory
+cd /path/to/sentry # or /path/to/getsentry
+
+# Run the save script
+bin/save_integration_data --output-file integration_backup.json
+```
+
+#### Step 2: Restore Your Integration Data
+
+After your database is reset and ready, restore your integration configuration:
+
+```bash
+# Basic restore (preserves original organization IDs)
+bin/load_integration_data --input-file integration_backup.json
+```
+
+**Or, if you need to change the organization ID:**
+
+```bash
+# Restore and update organization ID for OrganizationIntegration objects
+bin/load_integration_data --input-file integration_backup.json --org-id 123
+```
+
+After restoring, all the previous integration data will be restored and you can start using the integration for local development again.

develop-docs/sdk/data-model/envelope-items.mdx

@@ -400,7 +400,7 @@ _None_
 
 Item type `"log"` contains an array of log payloads encoded as JSON. This allows for multiple log payloads to be sent in a single envelope item.
 
-Only a single log item is allowed per envelope. The `item_count` field in the envelope item header must match the amount of logs sent, it's not optional. A `content_type` field in the envelope item header must be set to `application/vnd.sentry.items.log+json`.
+Only a single log container is allowed per envelope. The `item_count` field in the envelope item header must match the amount of logs sent, it's not optional. A `content_type` field in the envelope item header must be set to `application/vnd.sentry.items.log+json`.
 
 It's okay to mix logs from different traces into the same log envelope item, but if you do, you MUST not attach a DSC (dynamic sampling context) to the envelope header.
 

develop-docs/sdk/telemetry/replays.mdx

@@ -172,3 +172,22 @@ The other sub-item is the replay recording's instructions set. This payload shou
 {"segment_id":0}
 /* gzipped JSON payload */
 ```
+
+## SDK Behavior
+
+### Session mode
+
+When an SDK records Session Replay in `session` mode (`sessionSampleRate` is specified), the recording should start when the SDK is initialized and should be continuously streamed to the Sentry servers. The SDK should send a replay envelope every 5 seconds. The maximum duration of the recording should not exceed 60 minutes (or 15 minutes without an activity on Web). After the hard limit has been reached, the SDK should stop recording, clear out the current `replay_id` and remove it from the Scope so all of the subsequent events don't get associated with it.
+
+For the SDKs that support disk cache, the recording should pause when there's no internet connection or the SDK is getting rate-limited. This is necessary to prevent overflowing the disk cache, which can potentially result in losing more critical envelopes. When internet connection is restored or rate limit is lifted, the recording should resume.
+
+### Buffer mode
+
+When an SDK records Session Replay in `buffer` mode (`onErrorSampleRate` is specified), the recording should start when the SDK is initialized and should be buffered in-memory (and to disk if the SDK supports disk cache) in a ring buffer for up to 30 seconds back. The capturing of the recording may be triggered when one of the following conditions is met:
+
+- A crash or error event occurs and is captured by the SDK
+- `flush` API has been called manually on the replay (for SDKs that support manual API)
+
+After the initial (buffered) segment has been captured, the SDK should continue recording in `session` mode. Note, however, that the `replay_type` field of the following segments should still be set to `buffer` to reflect the original `replay_type`.
+
+If the crash or error event has been dropped in `beforeSend`, the replay should **not** be captured.

develop-docs/sdk/telemetry/traces/modules/ai-agents.mdx

@@ -83,7 +83,7 @@ Additional attributes on the span:
 Describes a tool execution.
 
 - The spans `op` MUST be `"gen_ai.execute_tool"`.
-- The spans `name` SHOULD be `"gen_ai.execute_tool {gen_ai.tool.name}"`. (e.g. `"gen_ai.execute_tool query_database"`)
+- The spans `name` SHOULD be `"execute_tool {gen_ai.tool.name}"`. (e.g. `"execute_tool query_database"`)
 - The `gen_ai.tool.name` attribute SHOULD be set to the name of the tool. (e.g. `"query_database"`)
 - All [Common Span Attributes](#common-span-attributes) SHOULD be set (all `required` common attributes MUST be set).
 

develop-docs/self-hosted/index.mdx

@@ -40,7 +40,13 @@ Depending on your traffic volume, you may want to increase your system specifica
 
 If increasing the disk storage space isn't possible, you can migrate your storage to use external storage such as AWS S3 or Google Cloud Storage (GCS). Decreasing your `SENTRY_RETENTION_DAYS` environment variable to lower numbers will save some storage space from being full, at the cost of having shorter data retention period. See [Event Retention](/self-hosted/configuration#event-retention) section.
 
-There are known issues on installing self-hosted Sentry on RHEL-based Linux distros, such as CentOS, Rocky Linux, and Alma Linux. It is also not possible to install on an Alpine Linux distro. Most people succeed with Debian/Ubuntu-based distros. If you successfully install on another distro, please let us know on the [Sentry's Discord](https://discord.gg/sentry)!
+Below is a breakdown of self-hosted Sentry installation compatibility with various Linux distributions:
+* **Debian/Ubuntu-based** distros are preferred; most users succeed with them, and they're used on Sentry's dogfood instance.
+* **RHEL-based Linux** distributions (e.g., CentOS, Rocky Linux, Alma Linux) have known installation issues. While some users have made it work by disabling SELinux, this is highly discouraged.
+* **Amazon Linux 2023**, a Fedora Linux derivative, has seen one person successfully run self-hosted Sentry. This was achieved with SELinux enabled and by adding their user to the `docker` group.
+* **Alpine Linux** is unsupported due to install script compatibility.
+
+If you successfully install Sentry on a different distribution, please share your experience on the [Sentry's Discord](https://discord.gg/sentry)!
 
 ## Getting Started
 

develop-docs/self-hosted/troubleshooting/kafka.mdx

@@ -16,33 +16,89 @@ This happens where Kafka and the consumers get out of sync. Possible reasons are
 2. Having a sustained event spike that causes very long processing times, causing Kafka to drop messages as they go past the retention time
 3. Date/time out of sync issues due to a restart or suspend/resume cycle
 
+### Visualize
+
+You can visualize the Kafka consumers and their offsets by bringing an additional container, such as [Kafka UI](https://github.com/provectus/kafka-ui) or [Redpanda Console](https://github.com/redpanda-data/console) into your Docker Compose.
+
+Kafka UI:
+```yaml
+kafka-ui:
+  image: provectuslabs/kafka-ui:latest
+  restart: on-failure
+  environment:
+    KAFKA_CLUSTERS_0_NAME: "local"
+    KAFKA_CLUSTERS_0_BOOTSTRAPSERVERS: "kafka:9092"
+    DYNAMIC_CONFIG_ENABLED: "true"
+  ports:
+    - "8080:8080"
+  depends_on:
+    - kafka
+```
+
+Or, you can use Redpanda Console:
+```yaml
+redpanda-console:
+  image: docker.redpanda.com/redpandadata/console:latest
+  restart: on-failure
+  entrypoint: /bin/sh
+  command: -c "echo \"$$CONSOLE_CONFIG_FILE\" > /tmp/config.yml; /app/console"
+  environment:
+    CONFIG_FILEPATH: "/tmp/config.yml"
+    CONSOLE_CONFIG_FILE: |
+      kafka:
+        brokers: ["kafka:9092"]
+        sasl:
+          enabled: false
+      schemaRegistry:
+        enabled: false
+      kafkaConnect:
+        enabled: false
+  ports:
+    - "8080:8080"
+  depends_on:
+    - kafka
+```
+
+Ideally, you want to have zero lag for all consumer groups. If a consumer group has a lot of lag, you need to investigate whether it's caused by a disconnected consumer (e.g., a Sentry/Snuba container that's disconnected from Kafka) or a consumer that's stuck processing a certain message. If it's a disconnected consumer, you can either restart the container or reset the Kafka offset to 'earliest.' Otherwise, you can reset the Kafka offset to 'latest.'
+
 ### Recovery
 
-Note: These solutions may result in data loss when resetting the offset of the snuba consumers.
+<Alert level="warning" title="Warning">
+These solutions may result in data loss for the duration of your Kafka event retention (defaults to 24 hours) when resetting the offset of the consumers.
+</Alert>
 
 #### Proper solution
 
-The _proper_ solution is as follows ([reported](https://github.com/getsentry/self-hosted/issues/478#issuecomment-666254392) by [@rmisyurev](https://github.com/rmisyurev)):
+The _proper_ solution is as follows ([reported](https://github.com/getsentry/self-hosted/issues/478#issuecomment-666254392) by [@rmisyurev](https://github.com/rmisyurev)). This example uses `snuba-consumers` with `events` topic. Your consumer group name and topic name may be different.
 
-1. Receive consumers list:
+1. Shutdown the corresponding Sentry/Snuba container that's using the consumer group (You can see the corresponding containers by inspecting the `docker-compose.yml` file):
+   ```shell
+   docker compose stop snuba-errors-consumer snuba-outcomes-consumer snuba-outcomes-billing-consumer
+   ```
+2. Receive consumers list:
    ```shell
    docker compose run --rm kafka kafka-consumer-groups --bootstrap-server kafka:9092 --list
    ```
-2. Get group info:
+3. Get group info:
    ```shell
    docker compose run --rm kafka kafka-consumer-groups --bootstrap-server kafka:9092 --group snuba-consumers --describe
    ```
-3. Watching what is going to happen with offset by using dry-run (optional):
+4. Watching what is going to happen with offset by using dry-run (optional):
    ```shell
    docker compose run --rm kafka kafka-consumer-groups --bootstrap-server kafka:9092 --group snuba-consumers --topic events --reset-offsets --to-latest --dry-run
    ```
-4. Set offset to latest and execute:
+5. Set offset to latest and execute:
    ```shell
    docker compose run --rm kafka kafka-consumer-groups --bootstrap-server kafka:9092 --group snuba-consumers --topic events --reset-offsets --to-latest --execute
    ```
-
-<Alert title="Tip">
-You can replace <code>snuba-consumers</code> with other consumer groups or <code>events</code> with other topics when needed.
+6. Start the previously stopped Sentry/Snuba containers:
+   ```shell
+   docker compose start snuba-errors-consumer snuba-outcomes-consumer snuba-outcomes-billing-consumer
+   ```
+<Alert level="info" title="Tips">
+* You can replace <code>snuba-consumers</code> with other consumer groups or <code>events</code> with other topics when needed.
+* You can reset the offset to "earliest" instead of "latest" if you want to start from the beginning.
+* If you have Kafka UI or Redpanda Console, you can reset the offsets through the web UI instead of the CLI.
 </Alert>
 
 #### Another option

develop-docs/self-hosted/troubleshooting/sentry.mdx

@@ -15,6 +15,14 @@ CSRF_TRUSTED_ORIGINS = ["https://sentry.example.com", "http://10.100.10.10", "ht
 
 See [Django's documentation on CSRF](https://docs.djangoproject.com/en/4.2/ref/settings/#std-setting-CSRF_TRUSTED_ORIGINS) for further detail.
 
+## Containers taking too much CPU/RAM usage
+
+If you're seeing a higher incoming traffic load, then it's expected. If this is the case, you might want to increase your machine resources.
+
+However, if you have very low incoming traffic and the CPU/RAM constantly spikes, you might want to check on your `top` (or `htop`, or similar) command to see which process are taking the most resources. You can then track down the corresponding Docker container and see if there are any logs that might help you identify the issue.
+
+Usually, most containers that are not a dependency (like Postgres or Kafka) will consume some good amount of CPU & RAM during startup, specifically for catching up with backlogs. If they are experiencing a [bootloop](https://en.wikipedia.org/wiki/Booting#Bootloop), it usually comes down to invalid configuration or a broken dependency.
+
 ## `sentry-data` volume not being cleaned up
 
 You may see the `sentry-data` taking too much disk space. You can clean it manually (or putting the cleanup cronjob in place).

docs/concepts/otlp/index.mdx

@@ -0,0 +1,46 @@
+---
+title: OpenTelemetry Protocol (OTLP)
+sidebar_order: 400
+description: "Learn how to send OpenTelemetry trace data directly to Sentry from OpenTelemetry SDKs."
+keywords: ["otlp", "otel", "opentelemetry"]
+---
+
+<Include name="feature-available-alpha-otlp.mdx" />
+
+Sentry can ingest [OpenTelemetry](https://opentelemetry.io) traces directly via the  [OpenTelemetry Protocol](https://opentelemetry.io/docs/specs/otel/protocol/). If you have an existing OpenTelemetry trace instrumentation, you can configure your OpenTelemetry exporter to send traces to Sentry directly. Sentry's OTLP ingestion endpoint is currently in development, and has a few known limitations:
+
+- Span events are not supported. All span events are dropped during ingestion.
+- Span links are partially supported. We ingest and display span links, but they cannot be searched, filtered, or aggregated. Links are are shown in the [Trace View](https://docs.sentry.io/concepts/key-terms/tracing/trace-view/).
+- Array attributes are partially supported. We ingest and display array attributes, but they cannot be searched, filtered, or aggregated. Array attributes are shown in the [Trace View](https://docs.sentry.io/concepts/key-terms/tracing/trace-view/).
+- Sentry does not support ingesting OTLP metrics or OTLP logs.
+
+The easiest way to configure an OpenTelemetry exporter is with environment variables. You'll need to configure the trace endpoint URL, as well as the authentication headers. Set these variables on the server where your application is running.
+
+```bash {filename: .env}
+export OTEL_EXPORTER_OTLP_TRACES_ENDPOINT="___OTLP_TRACES_URL___""
+export OTEL_EXPORTER_OTLP_TRACES_HEADERS="x-sentry-auth=sentry sentry_key=___PUBLIC_KEY___"
+```
+
+Alternatively, you can configure the OpenTelemetry Exporter directly in your application code. Here is an example with the OpenTelemetry Node SDK:
+
+```typescript {filename: app.ts}
+import { NodeSDK } from "@opentelemetry/sdk-node";
+import { OTLPTraceExporter } from "@opentelemetry/exporter-trace-otlp-http";
+
+const sdk = new NodeSDK({
+  traceExporter: new OTLPTraceExporter({
+    url: "___OTLP_TRACES_URL___",
+    headers: {
+      "x-sentry-auth": "sentry sentry_key=___PUBLIC_KEY___",
+    },
+  }),
+});
+
+sdk.start();
+```
+
+You can find the values of Sentry's OTLP endpoint and public key in your Sentry project settings.
+
+1. Go to the [Settings > Projects](https://sentry.io/orgredirect/organizations/:orgslug/settings/projects/) page in Sentry.
+2. Select a project from the list.
+3. Go to the "Client Keys (DSN)" sub-page for this project under the "SDK Setup" heading.