You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: develop-docs/backend/application-domains/kafka.mdx
+31-1Lines changed: 31 additions & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -3,4 +3,34 @@ title: Kafka consumers
3
3
sidebar_order: 60
4
4
---
5
5
6
-
Visit https://github.com/getsentry/ops/tree/master/shared_config/kafka for a full, in-depth step-by-step guide.
6
+
## Create a new Kafka topic
7
+
8
+
1. Add the topic to the `KAFKA_TOPIC_TO_CLUSTER` in [src/sentry/conf/server.py](https://github.com/getsentry/sentry/blob/master/src/sentry/conf/server.py):
9
+
* e.g. `subscription-results-eap-items`
10
+
2. Add the topic to `Topic` in [src/sentry/conf/types/kafka_definition.py](https://github.com/getsentry/sentry/blob/master/src/sentry/conf/types/kafka_definition.py)
11
+
12
+
## Define or re-use a processing strategy
13
+
14
+
In most cases a [Streaming Factory](https://getsentry.github.io/arroyo/getstarted.html#create-a-streaming-consumer) is what you want to when defining a consumer (see next section). You can find examples of it in [Sentry's code base](https://github.com/search?q=repo%3Agetsentry%2Fsentry+%28ProcessingStrategyFactory&type=code).
15
+
16
+
## Define a new Kafka consumer
17
+
18
+
1. Add a new entry in the `KAFKA_CONSUMERS` key in
2. You may need optional properties (e.g. `click_options`, you will need to research them by looking at [ConsumerDefinition](https://github.com/getsentry/sentry/blob/master/src/sentry/conf/types/kafka_definition.py)'s code.
29
+
30
+
3. Make sure you can run it: `sentry run consumer <your_topic>`
31
+
4. You may need to add some devserver options [here](https://github.com/getsentry/sentry/blob/master/src/sentry/runner/commands/devserver.py).
32
+
4. Add tests for your consumer
33
+
34
+
## Deployment
35
+
36
+
Visit the Ops repo and search for `shared_config/kafka/README.md` for a full, in-depth step-by-step guide.
0 commit comments