Ability to add additional options into a writeRequest

[bcorbett6]

Use Case

For customers who need to tailor their write client based on the below config:

• batchSize
• flushInterval
• bufferLimit
• jitterInterval
• retryInterval

This is useful for developing a productionised and durable write-client

Expected behavior

The ability to use the specified config options to tailor how data is sent to InfluxDB

Actual behavior

I don't believe these write options are yet included

Additional info

No response

[bednar]

@bcorbett6, thank you for your proposal. We are currently prioritizing adding support for the web environment. Once that is accomplished, we can then focus on implementing support for batching writes.

[guoxiangke]

batching writes needed, thanks.

[bednar]

Hi @guoxiangke,

Thank you for your interest in our client.

Regarding batching writes, what is your preferred approach? Would you like a background service that writes data asynchronously on a separate thread? Or would you prefer a straightforward writer that divides data into batches, writes asynchronously, and then returns the result?

Regards

[anfreita]

@bednar

For lack of response to your earlier questions, I think some simple retry logic in the existing node implementation would be fantastic. Something to mitigate for potential 413s and 429s when loading larger data sets.

You maybe have more insight into this than I, but I think that ideally it might be nice to maintain parity with the telegraf influx plugin here: https://github.com/influxdata/telegraf/blob/master/plugins/outputs/influxdb_v2/http.go

[bednar]

Hi @anfreita,

Thank you for your feedback and suggestions.

I wanted to update you that we have added the following enhancements to our development backlog for the v3 clients:

1. Proper handling of the HTTP 429 status (Too Many Requests) - We will implement a retry mechanism that intelligently manages rate limiting issues by retrying requests after an appropriate delay.

2. Proper handling of the HTTP 413 status (Payload Too Large) - We plan to enhance the client to automatically split requests that exceed the server's size limits, ensuring that large payloads are handled more efficiently.

These enhancements are aimed at improving the robustness and usability of our clients. We'll be looking into these as part of our next set of enhancements.

Your input is invaluable, and we appreciate your contribution to making our tools better. We will keep you updated on our progress and when you can expect these features to be implemented.

Best regards.

[suyashcjoshi]

Hi @bednar community has asked again for bath writes, any update on that ? Thanks!

[bednar]

Hi @suyashcjoshi,
this improvement is currently not one of our priorities. Could you please summarize the community requests here for better visibility and clarity of the requirements?
Thanks!
Jakub

[suyashcjoshi]

Hi @bednar appreciate the quick reply. I believe @bcorbett6 has summed the feature request pretty well so that will be great when it's prioritised. Basically more options to write data just like we have in Python SDK specially batch option. Please let me know if anything else.