You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: develop-docs/backend/application-domains/kafka.mdx
+28-7Lines changed: 28 additions & 7 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -3,15 +3,36 @@ title: Kafka consumers
3
3
sidebar_order: 60
4
4
---
5
5
6
-
## Creating a new consumer in Sentry
6
+
## Create a new Kafka topic
7
7
8
-
WIP! This is currently just a checklist.
8
+
1. Add the topic to the `KAFKA_TOPIC_TO_CLUSTER` in [src/sentry/conf/server.py](https://github.com/getsentry/sentry/blob/master/src/sentry/conf/server.py):
9
+
* e.g. `subscription-results-eap-items`
10
+
2. Add the topic to `Topic` in [src/sentry/conf/types/kafka_definition.py](https://github.com/getsentry/sentry/blob/master/src/sentry/conf/types/kafka_definition.py)
11
+
12
+
## Define or re-use a processing strategy
13
+
14
+
In most cases a [Streaming Factory](https://getsentry.github.io/arroyo/getstarted.html#create-a-streaming-consumer) is what you want to when defining a consumer (see next section). You can find examples of it in [Sentry's code base](https://github.com/search?q=repo%3Agetsentry%2Fsentry+%28ProcessingStrategyFactory&type=code).
15
+
16
+
## Define a new Kafka consumer
9
17
10
18
1. Add a new entry in the `KAFKA_CONSUMERS` key in
2. You may need optional properties (e.g. `click_options`, you will need to research them by looking at [ConsumerDefinition](https://github.com/getsentry/sentry/blob/master/src/sentry/conf/types/kafka_definition.py)'s code.
3. In the [Sentry Consumers metrics dashboard](https://www.notion.so/sentry/Kafka-e8b4f93595684c97b01fe831fbceb0dc?pvs=4#1fa8b10e4b5d8044859ecaf559463cb0), add a new saved view for your consumer.
Copy file name to clipboardExpand all lines: develop-docs/sdk/expected-features/setup-wizards/index.mdx
+13-6Lines changed: 13 additions & 6 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -53,15 +53,22 @@ For instance:
53
53
Related to this, wizards should [check the git status](#1-check-preconditions) of the project and warn users if they're not in a clean state.
54
54
This ensures that users can revert the changes easily if something goes wrong.
55
55
56
-
### Cater to the 80%
56
+
### Respect Users' decisions
57
+
58
+
Wizards should allow users to select the high-level Sentry features (e.g. tracing, replay, profiling) they want to install.
59
+
The SDK setup should be minimal, may be opinionated ("whatever makes sense for the platform") but
60
+
must respect users' decisions.
57
61
58
-
Wizards should be opinionated in which SDK options are set and cater to the 80% use case.
59
-
The SDK setup doesn't necessarily have to be minimal.
60
-
For example, it can include optional but recommended integrations (e.g. session replay or profiling) or options (e.g. sample rates).
62
+
Some examples:
63
+
- We should not enable a feature that users previously declined (e.g. set `tracesSampleRate` if users previously declined to enable tracing)
64
+
- We may set additional options, like the `tunnelRoute` option in Next.Js, but we should ask for consent whenever reasonable. In this example, consent is important because setting this option potentially increases users' traffic and hence their infrastructure bill.
65
+
- We may set additional options like `tracePropagationTargets` or `sendDefaultPii` as comments, if we think that these are options users should be aware of.
66
+
67
+
### Cater to the 80%
61
68
62
69
Given that most wizards will actively modify code or config files, there is a chance that they might fail or break something.
63
-
That's okay and expected as long as we're transparent about it and we tell users upfront that we're touching their files.
64
-
We cannot possibly cover all edge cases but we should try to cover the 80% so that typical projects can be set up with the wizard.
70
+
That's okay and expected as long as we're transparent about it and we tell users upfront that we're modifying their files.
71
+
We cannot possibly cover all edge cases but we should try to cover the 80% so that typical projects can be set up successfully.
Copy file name to clipboardExpand all lines: develop-docs/sdk/telemetry/logs.mdx
+89-17Lines changed: 89 additions & 17 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -268,33 +268,39 @@ An SDK should implement [Tracing without Performance](/sdk/telemetry/traces/trac
268
268
269
269
### Default Attributes
270
270
271
-
By default the SDK should set the following attributes:
271
+
By default the SDK should attach the following attributes to a log:
272
272
273
-
1.`sentry.message.template`: The parameterized template string
274
-
2.`sentry.message.parameter.X`: The parameters to the template string. X can either be the number that represent the parameter's position in the template string (`sentry.message.parameter.0`, `sentry.message.parameter.1`, etc) or the parameter's name (`sentry.message.parameter.item_id`, `sentry.message.parameter.user_id`, etc)
275
-
3.`sentry.environment`: The environment set in the SDK
276
-
4.`sentry.release`: The release set in the SDK
277
-
5.`sentry.trace.parent_span_id`: The span id of the span that was active when the log was collected. This should not be set if there was no active span.
278
-
6.`sentry.sdk.name`: The name of the SDK that sent the log
279
-
7.`sentry.sdk.version`: The version of the SDK that sent the log
280
-
8.[BACKEND SDKS ONLY]`server.address`: The address of the server that sent the log. Equivalent to `server_name` we attach to errors and transactions.
281
-
282
-
Example:
273
+
1.`sentry.environment`: The environment set in the SDK if defined.
274
+
2.`sentry.release`: The release set in the SDK if defined.
275
+
3.`sentry.trace.parent_span_id`: The span id of the span that was active when the log was collected. This should not be set if there was no active span.
276
+
4.`sentry.sdk.name`: The name of the SDK that sent the log
277
+
5.`sentry.sdk.version`: The version of the SDK that sent the log
283
278
284
279
```json
285
280
{
286
-
"sentry.message.template": "Adding item %s for user %s",
If the log was paramaterized the SDK should attach the following
290
+
291
+
1.`sentry.message.template`: The parameterized template string
292
+
2.`sentry.message.parameter.X`: The parameters to the template string. X can either be the number that represent the parameter's position in the template string (`sentry.message.parameter.0`, `sentry.message.parameter.1`, etc) or the parameter's name (`sentry.message.parameter.item_id`, `sentry.message.parameter.user_id`, etc)
293
+
294
+
```json
295
+
{
296
+
"sentry.message.template": "Adding item %s for user %s",
297
+
"sentry.message.parameter.0": "item_id",
298
+
"sentry.message.parameter.1": "user_id"
299
+
}
300
+
```
301
+
302
+
#### SDK Integration Attributes
303
+
298
304
If a log is generated by an SDK integration, the SDK should also set the `sentry.origin` attribute, as per the [Trace Origin](/sdk/telemetry/traces/trace-origin/) documentation. It is assumed that logs without a `sentry.origin` attribute are manually created by the user.
299
305
300
306
```json
@@ -303,7 +309,73 @@ If a log is generated by an SDK integration, the SDK should also set the `sentry
303
309
}
304
310
```
305
311
306
-
Beyond these attributes, we are exploring if the SDK should also send OS, user, and device information automatically (via reading the appropriate contexts from the scope). Given this behaviour can easily be added as a new feature to the SDK, it does not have to be part of the initial SDK implementation until we make a finalized decision.
312
+
#### User Attributes
313
+
314
+
If `sendDefaultPii`/`send_default_pii` is set to `true` in the SDK, the SDK should attach the following user data if available:
315
+
316
+
1.`user.id`: The user ID. Maps to `id` in the [User](/sdk/data-model/event-payloads/user/) payload.
317
+
2.`user.name`: The username. Maps to `username` in the [User](/sdk/data-model/event-payloads/user/) payload.
318
+
3.`user.email`: The email address. Maps to `email` in the [User](/sdk/data-model/event-payloads/user/) payload.
By default, Relay should parse the user agent attached to an incoming log envelope to parse `browser` and `os` information for logs. These attributes should be attached by Relay, but SDKs can attach them if they do not forward a user agent when sending logs to Sentry.
331
+
332
+
1.`browser.name`: Display name of the browser application. Maps to `name` in the [Contexts](/sdk/data-model/event-payloads/contexts/#browser-context) payload.
333
+
2.`browser.version`: Version string of the browser. Maps to `version` in the [Contexts](/sdk/data-model/event-payloads/contexts/#browser-context) payload.
334
+
335
+
```json
336
+
{
337
+
"browser.name": "Chrome",
338
+
"browser.version": "120.0"
339
+
}
340
+
```
341
+
342
+
#### Backend SDKs
343
+
344
+
For backend SDKs (Node.js, Python, PHP, etc.), the SDKs should attach the following:
345
+
346
+
1.`server.address`: The address of the server that sent the log. Equivalent to [`server_name`](sdk/data-model/event-payloads/#optional-attributes) we attach to errors and transactions.
347
+
348
+
```json
349
+
{
350
+
"server.address": "foo.example.com"
351
+
}
352
+
```
353
+
354
+
#### Mobile, Desktop, and Native SDKs
355
+
356
+
For mobile, desktop, and native SDKs (Android, Apple, Electron, etc.), the SDKs should attach the following:
357
+
358
+
1.`os.name`: The name of the operating system. Maps to `name` in the [Contexts](/sdk/data-model/event-payloads/contexts/#os-context) payload.
359
+
2.`os.version`: The version of the operating system. Maps to `version` in the [Contexts](/sdk/data-model/event-payloads/contexts/#os-context) payload.
360
+
3.`device.brand`: The brand of the device. Maps to `brand` in the [Contexts](/sdk/data-model/event-payloads/contexts/#device-context) payload.
361
+
4.`device.model`: The model of the device. Maps to `model` in the [Contexts](/sdk/data-model/event-payloads/contexts/#device-context) payload.
362
+
5.`device.family`: The family of the device. Maps to `family` in the [Contexts](/sdk/data-model/event-payloads/contexts/#device-context) payload.
363
+
364
+
```json
365
+
{
366
+
"os.name": "iOS",
367
+
"os.version": "17.0",
368
+
"device.brand": "Apple",
369
+
"device.model": "iPhone 15 Pro Max",
370
+
"device.family": "iPhone"
371
+
}
372
+
```
373
+
374
+
#### Future Default Attributes
375
+
376
+
The SDKs should aim to minimize the number of default attributes attached to a log. Logs are intended to be lightweight, and we want to try to keep the average byte size of a log as small as possible as users will be charged per byte size of logs sent.
377
+
378
+
We are trying to settle on a balance of debuggability vs. smaller byte size for logs which is why new default attributes should only be added after significant feedback from users and discussion internally with the SDK and ingest teams. There is no hard rule about what exact attributes are allowed, every proposed new attribute will be evaluated on a case-by-case basis.
0 commit comments