Skip to content

Limit log batching to maximum 1000 logs per batchΒ #6769

@github-actions

Description

@github-actions

Note

The pull request "Limit log batching to maximum 1000 logs per batch" was created by @denrase but did not reference an issue. Therefore this issue was created for better visibility in external tools like Linear.

πŸ“œ Description

This took some digging. I was testing sending logs in a tight loop and got rejected by the backend.

[Sentry] [debug] [1763038665.309155] [SentryRequestOperation:36] Request status: 400
[Sentry] [debug] [1763038665.309175] [SentryRequestOperation:42] Request response: {"detail":"envelope exceeded size limits for type 'log' (https://develop.sentry.dev/sdk/envelopes/#size-limits)"}

So I checked replay code. The issue is that it has 1000 logs MAX item limit, which is not in any documentation we used when implementing the log batching.

max_log_count: 1000

Another issue I saw during testing is that envelopes persisted with > 1000 log items are being re-tried, so this is something that clients out there may be experiencing.

@philipphofmann We need to update the spec and also inform other SDK maintainers. Also, we need to check if we can/should do anything in regard to persisted envelopes with > 1000 log items.

The BatchProcessor MUST forward all items to the transport after the SDK when containing spans or logs exceeding 1MiB in size. The SDK MAY choose a different value for the max batch size keeping the envelope max sizes in mind.

This part is also somewhat ambiguous. What the backend does is check that all log items do not exceed 3 MB and the individual items do not exceed 1MB on average.

πŸ’‘ Motivation and Context

Log envelopes dropped by the backend.

πŸ’š How did you test it?

Unit tests. Sample app.

πŸ“ Checklist

You have to check all boxes before merging:

  • I added tests to verify the changes.
  • No new PII added or SDK only sends newly added PII if sendDefaultPII is enabled.
  • I updated the docs if needed.
  • I updated the wizard if needed.
  • Review from the native team if needed.
  • No breaking change or entry added to the changelog.
  • No breaking change for hybrid SDKs or communicated to hybrid SDKs.

Metadata

Metadata

Assignees

Labels

No labels
No labels

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions