fix(splunk_hec sink): Force compression disabled in HEC indexer ack queries #23560
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
The Splunk HEC sinks run indexer ack querying on a separate spawned task, reusing the sink's original
HttpRequestBuilder
. However, the serialized indexer ack query bodies are raw bystreams, uncompressed. This leads to a situation where the sink, and thus the sharedHttpRequestBuilder
, are configured for compression, but the indexer ack query task sends uncompressed requests. This PR is a fix to clone theHttpRequestBuilder
and force compression disabled, so noContent-Encoding
header is sent.A more comprehensive fix would be to pass the sink's configured
Writer
instance from the sink to the indexer ack task. The clonedHttpRequestBuilder
in this PR would be removed, returning to using the original sink'sHttpRequestBuilder
. The indexer ack query request building would be reworked to compress the bytestream (if necessary) by the Writer before submitting that bytestream to theHttpRequestBuilder
. Thus properly compressing the indexer ack query with the sink's configured compression.Vector configuration
How did you test this PR?
The change was tested by running against an emulated HEC via
nc
and manually verifying that the indexer ack query sent to the/services/collector/ack
endpoint did not contain aContent-Encoding
header.Change Type
Is this a breaking change?
Does this PR include user facing changes?
no-changelog
label to this PR.References
splunk_hec_logs
sink indexer ack queries fail when using compression #23559splunk_hec_logs
sink indexer ack queries fail when using compression #23559Notes
@vectordotdev/vector
to reach out to us regarding this PR.pre-push
hook, please see this template.cargo fmt --all
cargo clippy --workspace --all-targets -- -D warnings
cargo nextest run --workspace
(alternatively, you can runcargo test --all
)git merge origin master
andgit push
.Cargo.lock
), pleaserun
cargo vdev build licenses
to regenerate the license inventory and commit the changes (if any). More details here.