How to configure out_stackdriver batch size #9283
Unanswered
rafal-dudek
asked this question in
Q&A
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
Hello,
I was checking stackdriver output configuration: https://docs.fluentbit.io/manual/pipeline/outputs/stackdriver
And didn't find an option to configure batch size.
Configuring that might be important for performance and quota limitations for high logs traffic:
https://cloud.google.com/logging/docs/reference/v2/rest/v2/entries/write
Here are the limits for entries.write: https://cloud.google.com/logging/quotas#api-limits
Size of an entries.write request | 10 MB
Number of entries.write requests | 120,000 per minute, per Google Cloud project
I've checked it briefly and I think the batch size depends on the chunk size in fluent-bit.
https://docs.fluentbit.io/manual/administration/buffering-and-storage#chunks
Is there any way we can influence this size? I only found an option to change the buffer size for all chunks, not the single chunk size.
Beta Was this translation helpful? Give feedback.
All reactions