Dynamic headers/authentication for HTTP-based sources (using the http_client source)

[d3vzer0]

Question

Hiya everyone,

Just wanted to say I'm super happy with the latest Vector feature to support templating in the http sink; this solved a usecases I'm working on for a particular API that uses a dynamic authentication method 🥳 The API expects a HMAC for each of the requests based on several components in the message, such as the time, optional body and the full url+query. In a previous transform I can generate the HMAC using VRL's built-in hmac function and save the result (base64 encoded) inside the metadata field of an event and use this in the sink with templates.

I'm also trying to achieve something similar but using the API as a source. As far as I know, the http_client source only supports vrl as part of the query configuration but not for headers or auth. Alternatively, using a custom secrets script won't work since the HMAC has to be based on the message itself. Has anyone worked with a similar usecase? Are there any plans to add support for VRL on other components of the http_client source, such as the headers or auth configuration? Otherwise I can take a look at the source and open a PR :)

Vector Config

No response

Vector Logs

No response

[pront]

Hi @d3vzer0, I am glad the new http features are useful!

Are there any plans to add support for VRL on other components of the http_client source, such as the headers or auth configuration?

No plans at the moment. I would encourage you to create one or two feature requests to get the ball rolling on this. PRs are always welcome.

For the authentication we can potentially draw inspiration from:
https://vector.dev/docs/reference/configuration/sources/http_server/#authorization-configuration