Introduce merge and split experimental APIs for bulk indexers

[lahsivjar]

The merge and split APIs will allow Elasticsearch exporter to size the requests to ES using the new batch queue based on the bulk request sizes.

[axw]

@lahsivjar this is only going to be used by the otel elasticsearch exporter right? I wonder if we should expose some lower level functionality from go-docappender, and keep the split/merge logic in the exporter?

Maybe one option for that would be to introduce a new "BulkIndexerBuffer" type that contains the Add method. It could have a Split method, or it could perhaps just have a method for iterating over items in the buffer and we can leave it to the caller to copy those to a new buffer.

[lahsivjar]

@lahsivjar this is only going to be used by the otel elasticsearch exporter right? I wonder if we should expose some lower level functionality from go-docappender, and keep the split/merge logic in the exporter?

Maybe one option for that would be to introduce a new "BulkIndexerBuffer" type that contains the Add method. It could have a Split method, or it could perhaps just have a method for iterating over items in the buffer and we can leave it to the caller to copy those to a new buffer.

Good idea! I will look into it.