You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: develop-docs/sdk/telemetry/spans/batch-processor.mdx
+23-6Lines changed: 23 additions & 6 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -134,15 +134,15 @@ As the BatchProcessor MUST prevent data loss during flushing, it uses a double-b
134
134
The SDK SHOULD store the BatchProcessor files in a folder that is a sibling of the `envelopes` or `replay` folder, named `batch-processor`. This folder is scoped per DSN, so SDKs ensure not mixing up data for different DSNs. The `batch-processor` folder MAY contain the following files:
135
135
136
136
-`async-io-cache-1` and `async-io-cache-2` - The async IO cache files.
137
-
-`detected-termination-x` - The file containing items from a previous detected abnormal termination.
137
+
-`detected-termination-x` - The file containing items from the crash-safe list from a previous detected abnormal termination.
138
138
-`envelope-x` - The envelope that the BatchProcessor is about to move to the envelopes cache folder, so the SDK can send it to Sentry, where `x` is the an increasing index of the file starting from 0.
139
139
140
140
141
141
#### Receiving Items
142
142
143
143
The BatchProcessor has two lists `crash-safe-list-1` and `crash-safe-list-2` and two files `async-io-cache-1` and `async-io-cache-2`. When it receives items, it performs the following steps:
144
144
145
-
1. Put the item into the crash-safe `crash-safe-list1` on the calling thread.
145
+
1. Put the item into the crash-safe `crash-safe-list-1` on the calling thread.
146
146
2. On a background thread, store the item in the `async-io-cache-1`.
147
147
148
148
#### Flushing
@@ -161,16 +161,33 @@ The BatchProcessor stores the `envelope-x` not directly in the envelope cache fo
161
161
162
162
When SDKs detect an abnormal process termination, they MUST write the items in both `crash-safe-list-1` and `crash-safe-list-2` to the `detected-termination-x` file where `x` is the an increasing index of the file starting from 0.
163
163
164
-
When the process terminates abnormally and the SDKs can't detect it, the SDKs lose items in the crash safe lists, which we accept over blocking the calling thread that could be the main thread.
164
+
When the process terminates abnormally and the SDKs can't detect it, the SDKs lose items from the crash-safe lists, which we consider preferable to blocking the calling thread, which could be the main thread. However, the SDKs don't lose items from the async IO cache.
165
+
166
+
So the SDK MAY lose items due to undetectable abnormal process terminations that occur immediately after receiving an item, but it won't lose items due to detectable abnormal process terminations.
167
+
168
+
```
169
+
// SDKs keep this log
170
+
Sentry.logger.info("We might crash now.");
171
+
172
+
crash();
173
+
```
174
+
175
+
```
176
+
// SDKs might lose this log
177
+
Sentry.logger.info("We are going to allocate loads of memory now.");
178
+
179
+
// Something allocating a lot of memory leading to a watchdog termination due to out of memory.
180
+
allocateAllMemory();
181
+
```
165
182
166
183
#### SDK Initialization
167
184
168
-
Whenever the SDKs initialize, they must check if there is any data in the batch processor folder that needs to be recovered. SDKs MUST perform the following steps when initializing:
185
+
Whenever the SDKs initialize, they MUST check if there is any data in the batch processor folder that needs to be recovered. They MUST perform the following steps when initializing:
169
186
170
-
1. Load all items from `async-io-cache-1`, `async-io-cache-2` and `detected-termination-x`files into memory if there are any. If the application terminates normally, these files don't exist.
187
+
1. Load all items from `async-io-cache-1`, `async-io-cache-2` and `detected-termination-x` into memory if they exist. When the application terminates normally, these files don't exist.
171
188
2. Deduplicate the items based on the IDs of the items and store the deduplicated items in the `envelope-x` file.
172
189
3. Create new `async-io-cache-1` and `async-io-cache-2` files, and delete the `detected-termination-x` file.
173
-
4. Now the BatchProcessor can start receiving new items again.
190
+
4. Now the BatchProcessor can start receiving new items.
174
191
5. Move the `envelope-x` to the envelopes cache folder.
0 commit comments