HttpPull stream - authenticate using oauth client credentials flow

[ZarkoPetrovicZuhlke]

Hi,

I am trying to use httppull stream to connect to rest endpoint secured by oauth. I am trying to use client credentials flow with Microsoft Entra (Azure) as identity provider.

I need to make POST request to obtain token as described here: https://learn.microsoft.com/en-us/entra/identity-platform/v2-oauth2-client-creds-grant-flow#get-a-token

I have populated url with https://login.microsoftonline.com/<TENANT_ID>/oauth2/v2.0/token and body with url encoded values but I am getting error 400 in response:

eKuiper log (I have replaced tenant id with <tenant>:

time="2025-05-05T14:16:58Z" level=info msg="Try to get access token from [https://login.microsoftonline.com/<tenant>/oauth2/v2.0/token"⁠](https://login.microsoftonline.com/<tenant>/oauth2/v2.0/token%22) file="http/client.go:208"


time="2025-05-05T14:16:59Z" level=info msg="rule device_vpn_connection_off transit to state stopped by error" Rule=device_vpn_connection_off file="rule/state.go:160"


time="2025-05-05T14:16:59Z" level=info msg="Rule device_vpn_connection_off start failed: fail to authorize by oAuth: response code error: 400" file="server/server.go:241"

I am assuming that error 400 might come from the fact that I didn't specify header Content-Type: application/x-www-form-urlencoded anywhere.

I have read documentation about using oauth in httppull stream but I don't see if that header is passed to token url and also I don't understand how to extract access token if the response was successful and use it as bearer token in httppull request.

[ZarkoPetrovicZuhlke]

I was able to fix issue locally with a hack.

The problem is that oAuth token request needs to have header Content-Type: application/x-www-form-urlencoded. I am using eKuiper Manager 1.9.5-plus and there is no way in the UI to specify that header, so the token request fails with 400 status when eKuiper runs the rule. eKuiper api will accept 'headers' property under oauth.access in the request body, so I have added that property and reissued request to save http pull configuration. After that, eKuiper was able to obtain token and use it in authorization header - the HttpPull stream started working correclty.

Do you know if there is a way to modify eKuiper Manager or make it open source so that header can be added to oAuth request?

Perhaps another solution would be to include Content-Type: application/x-www-form-urlencoded header by default when making token request since it is required per oAuth documentation (https://www.rfc-editor.org/rfc/rfc6749#section-4.1.1)

[ngjaying]

@ZarkoPetrovicZuhlke eKuiper supports to set header in the oAuth accessToken config since v2.1.0. eKuiper manager is not maintained now, so you need to set it from REST API.

[ZarkoPetrovicZuhlke]

Is there a way to define httppull configuration in yaml file as specific configuration key and specify headers there? Then that configuration can be used when creating streams.