Problems with aws auth in http_client and prometheus_scrape sources

[T-Sh]

According to docs, prometheus_scrape and http_client sources support aws auth in outcoming request. I've tried next configurations:

sources:
  prometheus-source:
    type: prometheus_scrape
    endpoints:
     - http://127.0.0.1/metrics/
    scrape_interval_secs: 30
    scrape_timeout_secs: 20
    auth:
      strategy: aws
      service: vector
      auth:
        secret_access_key: <access_key>
        access_key_id: <key_id>
        region: us-east-1
  http-source:
    type: http_client
    endpoint: http://127.0.0.1/metrics/
    auth:
      strategy: aws
      service: vector
      auth:
        secret_access_key: <access_key>
        access_key_id: <key_id>
        region: us-east-1

But both sources don't send auth headers in requests (based on vector debug logs and actual request in polled service). Debug logs from vector:

DEBUG source{component_kind="source" component_id=http-source component_type=http_client}:http: vector::internal_events::http_client: Sending HTTP request. uri=http://127.0.0.1/metrics/ method=GET version=HTTP/1.1 headers={"accept": "text/plain", "user-agent": "Vector/0.47.0 (x86_64-unknown-linux-gnu 3d5af22 2025-05-20 13:53:41.638057046)", "accept-encoding": "identity"} body=[empty]

DEBUG source{component_kind="source" component_id=prometheus-source component_type=prometheus_scrape}:http: vector::internal_events::http_client: Sending HTTP request. uri=http://127.0.0.1/metrics/ method=GET version=HTTP/1.1 headers={"accept": "text/plain", "user-agent": "Vector/0.47.0 (x86_64-unknown-linux-gnu 3d5af22 2025-05-20 13:53:41.638057046)", "accept-encoding": "identity"} body=[empty]

@johannesfloriangeiger I assume you added this feature in #22744. Can you help what I am doing wrong and how to make it work?

[johannesfloriangeiger]

I haven't touched the HTTP source, #22744 is only about the sink - might it be possible the documentation for the http source reuses some of the general HTTP auth documentation and falsely advertises the HTTP source can also use AWS auth?

[T-Sh]

The issue is not about documentation only, vector accepts configs with aws auth in those sources as valid (but doesn't use it). If the configuration was invalid, vector wouldn't start at all.

Maybe it happens because src/http.rs is reused in both sinks and sources, but in defferent ways? That might explains why vector accepts auth.auth param in sources.

Unfortunately can't say more about code, I'm not rust programmer :)

[johannesfloriangeiger]

Well that's what I meant with "I haven't touched the HTTP source, #22744 is only about the sink" - if the source logic isn't actively using the AWS auth part I don't see how it should work.

[johannesfloriangeiger]

I think I've got an idea how to add the AWS auth also to the HTTP source, I'm running into a few syntax issues though when trying to re-use the existing SigV4 functionality (how the bodies of the HTTP requests differ).
I try to raise a PR soon to proof it's working in general and how to proceed from that!

[johannesfloriangeiger]

Raised ticket #23338, PR will follow soon.