feat: allow to use environment variables for openid-connect plugin #11451

darkSheep404 · 2024-07-31T08:40:29Z

Description

feat: allow to use environment variables for openid-connect plugin

Checklist

I have explained the need for this PR and the problem it solves
I have explained the changes or the new features added to this PR
I have added tests corresponding to this change
I have updated the documentation to reflect this change
I have verified that this change is backward compatible (If not, please discuss on the APISIX mailing list first)

darkSheep404 · 2024-09-24T01:17:13Z

hi good morning @shreemaan-abhishek clould you kindly review this PR for me ?

shreemaan-abhishek

please write a test case that uses vault as well.

apisix/plugins/openid-connect.lua

shreemaan-abhishek · 2024-09-25T08:21:02Z

apisix/plugins/openid-connect.lua


-function _M.check_schema(conf)
+function _M.check_schema(plugin_conf)
+    local conf = fetch_secrets(plugin_conf)


this is not needed.

this is not needed.↳

This is needed when someone puts a non-string value such as a Boolean into env var, otherwise the type inconsistency will fail the check

This line is giving me some issues after apisix reload.
If line fetch_secrets is done in check_schema, the route fails showing me the following error:

2024/11/12 17:12:51 [error] 240#240: *13728 lua entry thread aborted: runtime error: ...isix/custom-plugins/apisix/plugins/openid-connect.lua:478: attempt to compare nil with number stack traceback: coroutine 0: ...isix/custom-plugins/apisix/plugins/openid-connect.lua: in function 'phase_func' /usr/local/apisix/apisix/plugin.lua:1166: in function 'run_plugin' /usr/local/apisix/apisix/init.lua:689: in function 'http_access_phase' access_by_lua(nginx.conf:310):2: in main chunk, client: 10.89.2.37, server: _, request: "GET /private/anything HTTP/2.0", host: "XXXX"

If I remove fetch_secrets from check_schema, the route work as expected but the following warning is shown at startup:
[warn] 187#187: *8391 [lua] utils.lua:418: find_and_log(): Using openid-connect discovery with no TLS is a security risk, context: init_worker_by_lua*
I assume that without the fetch_secrets the value of discovery is not resolved and openid checks https for it.

So should we remove fetch_secrets from check_schema to avoid this?
By contrast, putting a Boolean value in a secret is not a particularly common case in this plugin. Typically, only string urls and secret keys will be placed in valut

pushed this change

shreemaan-abhishek

please write a test case that uses vault as well.

shreemaan-abhishek · 2024-09-25T08:22:50Z

t/fips/openid-connect.t

please add new test cases in t/plugin/openid-connect

Co-authored-by: Shreemaan Abhishek <[email protected]>

darkSheep404 · 2024-09-26T05:42:53Z

please write a test case that uses vault as well.↳
hi @shreemaan-abhishek
I tried to add a test for valut but I am not sure if it is correct, may need your help to have a look

shreemaan-abhishek

the test cases for secret resource seems correct to me, please resolve the conflicts with master so that the tests can run

alberto-corrales · 2024-10-07T16:14:30Z

Great contribution!! I'm looking forward to this fix, as we are using AWS Secrets Manager as a secrets provider and we need to configure the secret as an environment variable. I hope this fix is merged and released soon 🙏

# Conflicts: # t/fips/openid-connect.t

darkSheep404 · 2024-10-08T02:09:11Z

the test cases for secret resource seems correct to me, please resolve the conflicts with master so that the tests can run↳

hi @shreemaan-abhishek
The conflict has been resolved. Please help approve it when you are free so that can run the test

brmejia

This is exactly what I was looking for. I've tested this config on my local environment, and it works as expected.

client_id: "$env://APISIX_OIDC_CLIENT_ID"
client_secret: "$env://APISIX_OIDC_CLIENT_SECRET"
discovery: $env://APISIX_OIDC_DISCOVERY_URL

I hope this will be merged soon

alberto-corrales · 2024-12-17T13:38:30Z

Are there any updates on this? At the moment we have to keep a secret in our repo because of the bug that it is fixing this PR, and we need to move it to environment variables. So it would be awesome if we could have this fixed in a patch soon

darkSheep404 · 2024-12-17T13:42:50Z

Are there any updates on this? At the moment we have to keep a secret in our repo because of the bug that it is fixing this PR, and we need to move it to environment variables. So it would be awesome if we could have this fixed in a patch soon

We need another good Samaritan to approve this change, then this PR will have three approve and be merged

darkSheep404 · 2024-12-17T14:12:23Z

Are there any updates on this? At the moment we have to keep a secret in our repo because of the bug that it is fixing this PR, and we need to move it to environment variables. So it would be awesome if we could have this fixed in a patch soon

We need another good Samaritan to approve this change, then this PR will have three approve and be merged

@shreemaan-abhishek @nic-6443

Could any of you help approve this PR at your convenience? It has been pending here for a long time and only needs the last approver

darkSheep404 · 2024-12-24T14:35:52Z

Are there any updates on this? At the moment we have to keep a secret in our repo because of the bug that it is fixing this PR, and we need to move it to environment variables. So it would be awesome if we could have this fixed in a patch soon

We need another good Samaritan to approve this change, then this PR will have three approve and be merged

@shreemaan-abhishek @nic-6443

Could any of you help approve this PR at your convenience? It has been pending here for a long time and only needs the last approver

hi @kayx23 @membphis @Revolyssup Could any of you help?

shreemaan-abhishek · 2024-12-24T16:44:14Z

I have left a comment: https://github.com/apache/apisix/pull/11451/files#r1896868940

darkSheep404 · 2025-07-29T06:11:34Z

Hi @darkSheep404, please merge the master branch code, and the failed test will be fixed.

merged

membphis

LGTM

SkyeYoung · 2025-08-04T07:45:03Z

@nic-6443 @bzp2010 @Revolyssup @AlinsRan pls review this PR.

darkSheep404 · 2025-08-05T03:30:47Z

@nic-6443 @bzp2010 @Revolyssup @AlinsRan pls review this PR.

seems can merge now, need someone help to click "merge" button or need to wait all those members approve it, then it will be merged automatic ?

bzp2010

only a minor issue

bzp2010 · 2025-08-05T06:52:59Z

apisix/plugins/openid-connect.lua


 function _M.rewrite(plugin_conf, ctx)
-    local conf = core.table.clone(plugin_conf)
+    local conf = fetch_secrets(plugin_conf, true, plugin_conf, "")


Please still clone the table.
While it is true that fetch_secrets does imply this fact, it cannot be inferred from the name and is not a feature of any mandatory guarantee.
As soon as someone modifies fetch_secrets in the future and doesn't clone there anymore, something will go wrong with your code.

Therefore, it's best to explicitly copy this table here anyway to ensure that, no matter how much external conditions change, the logic of this plugin doesn't break.

Please still clone the table. While it is true that fetch_secrets does imply this fact, it cannot be inferred from the name and is not a feature of any mandatory guarantee. As soon as someone modifies fetch_secrets in the future and doesn't clone there anymore, something will go wrong with your code.

Therefore, it's best to explicitly copy this table here anyway to ensure that, no matter how much external conditions change, the logic of this plugin doesn't break.

hi @Baoyuantop please advice

I previously referred to the implementation of other plugin codes:

https://github.com/apache/apisix/blob/master/apisix/plugins/authz-keycloak.lua#L767

https://github.com/apache/apisix/blob/master/apisix/plugins/limit-count.lua#L37

There are many other locations, so I think there is no problem with the current PR from the implementation point of view.

As soon as someone modifies fetch_secrets in the future and doesn't clone there anymore, something will go wrong with your code.

If this happens, many plugin codes will need to be adjusted. If my understanding is wrong, please correct me.

hi @Baoyuantop already 5 approval more than 3 approval required now, cloud you help to merge this PR?

bzp2010 · 2025-08-22T04:58:50Z

apisix/plugins/openid-connect.lua


 function _M.rewrite(plugin_conf, ctx)
-    local conf = core.table.clone(plugin_conf)
+    local conf = fetch_secrets(plugin_conf, true, plugin_conf, "")


I previously referred to the implementation of other plugin codes:

https://github.com/apache/apisix/blob/master/apisix/plugins/authz-keycloak.lua#L767

https://github.com/apache/apisix/blob/master/apisix/plugins/limit-count.lua#L37

There are many other locations, so I think there is no problem with the current PR from the implementation point of view.

As soon as someone modifies fetch_secrets in the future and doesn't clone there anymore, something will go wrong with your code.

If this happens, many plugin codes will need to be adjusted. If my understanding is wrong, please correct me.

I do not agree with this conclusion.

The examples you listed are insufficient to prove this point. The reason we need to clone conf is that this plugin may temporarily and locally modify fields in conf.

Lua tables use reference passing (pointer), and modifying the conf within the plugin would have side effects on the global configuration cache, which is unacceptable.

You can check the two examples you provided; they do not involve any scenarios where the conf is modified, so there is no need to explicitly clone the conf. Whether fetch_secrets clones the conf internally has no impact on this.

Looking at the openid-connect plugin, there are modifications to conf everywhere, so it undoubtedly needs to know whether conf has been cloned. This is a necessary measure to ensure that the global cache is not polluted.
However, we cannot rely on external behavior (fetch_secrets) to guarantee this, as it violates the principle of low coupling.

https://github.com/darkSheep404/apisix/blob/ec2974aa3e1d486b8416e5010f3d1046db093f37/apisix/plugins/openid-connect.lua#L562-L585

I still stand by this view, and unless this is modified, I will not merge this PR.

I do not believe that fetch_secrets implies that it will perform a clone and guarantee that this behavior will always be valid.

Performing an explicit clone (which is only a shallow clone) is inexpensive. At the same time, it ensures that it will conform to the logic before this modification rather than relying on external conditions for assurance.

Even if anybody does modify fetch_secrets in the future, the fact that conf is not cloned cannot be easily detected.
This will lead to unexpected behavior rather than an error.
I do not have the time to track all code changes in this area, and I do not want to wait until a bad situation occurs to discover it.

If this is still a matter of controversy, then let more maintainers express their opinions.

hi @bzp2010
just add a commit to clone plugin_conf before fetch_secrets

@bzp2010 may i ask what is the apisix PR merge flow? more than 3 approvals will merge it automatic
or need someone manually click merge button,or else ?

more than 3 approvals and manually click merge button.

There is also a CI error, but it doesn't seem to be related to this PR. I've seen similar errors in other PRs.

more than 3 approvals and manually click merge button.

There is also a CI error, but it doesn't seem to be related to this PR. I've seen similar errors in other PRs.

then could you help to merge it now ?🙂

Merged, thanks for your contribution.

Baoyuantop · 2025-08-25T03:25:14Z

Merge first, the failed CI will be fixed at #12530 (comment)

…pache#11451)

darksheep404 added 3 commits July 31, 2024 16:21

feat: allow to use environment variables for openid-connect

68d705c

feat: allow to use environment variables for openid-connect

2e7df95

feat: allow to use environment variables for openid-connect

5ab3f3e

pottekkat requested a review from shreemaan-abhishek August 7, 2024 12:31

darkSheep404 mentioned this pull request Aug 9, 2024

bug: openid-connect yml conf could not use $env to set plugin properties #11319

Closed

juzhiyuan mentioned this pull request Aug 13, 2024

The client secret for oidc plugin cannot be configured as environment variable #11353

Closed

shreemaan-abhishek reviewed Sep 25, 2024

View reviewed changes

t/fips/openid-connect.t

Copy link

Contributor

shreemaan-abhishek Sep 25, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please add new test cases in t/plugin/openid-connect

feat: allow to use environment variables for openid-connect plugin

dfd3005

Co-authored-by: Shreemaan Abhishek <[email protected]>

dosubot bot added the size:M This PR changes 30-99 lines, ignoring generated files. label Sep 25, 2024

darksheep404 added 2 commits September 26, 2024 13:39

feat: allow to use environment variables for openid-connect

cc39db4

feat: allow to use environment variables for openid-connect

980df99

dosubot bot added size:L This PR changes 100-499 lines, ignoring generated files. and removed size:M This PR changes 30-99 lines, ignoring generated files. labels Sep 26, 2024

shreemaan-abhishek reviewed Oct 3, 2024

View reviewed changes

darksheep404 added 2 commits October 8, 2024 10:03

Merge branch 'master' into feat-openid-connect-support-env

a4b9e06

# Conflicts: # t/fips/openid-connect.t

feat: allow to use environment variables for openid-connect

3fe0e00

brmejia approved these changes Nov 12, 2024

View reviewed changes

darkSheep404 requested a review from shreemaan-abhishek November 15, 2024 03:01

feat: allow to use environment variables for openid-connect

6c7a0b9

alberto-corrales approved these changes Nov 25, 2024

View reviewed changes

Baoyuantop removed the wait for update wait for the author's response in this issue/PR label Jul 29, 2025

Merge branch 'apache:master' into feat-openid-connect-support-env

ec2974a

Baoyuantop removed the user responded label Aug 1, 2025

Baoyuantop previously approved these changes Aug 1, 2025

View reviewed changes

Baoyuantop requested review from AlinsRan, Revolyssup, bzp2010, membphis and nic-6443 August 1, 2025 06:13

membphis previously approved these changes Aug 4, 2025

View reviewed changes

SkyeYoung previously approved these changes Aug 4, 2025

View reviewed changes

bzp2010 reviewed Aug 5, 2025

View reviewed changes

SkyeYoung requested a review from bzp2010 August 6, 2025 01:11

bzp2010 requested changes Aug 22, 2025

View reviewed changes

feat: clone openid-connect conf before fetch_secrets

463c47a

darkSheep404 dismissed stale reviews from SkyeYoung, membphis, and Baoyuantop via 463c47a August 22, 2025 05:31

bzp2010 approved these changes Aug 22, 2025

View reviewed changes

darkSheep404 requested a review from Baoyuantop August 22, 2025 06:01

SkyeYoung approved these changes Aug 22, 2025

View reviewed changes

Baoyuantop approved these changes Aug 25, 2025

View reviewed changes

Baoyuantop merged commit 54418e1 into apache:master Aug 25, 2025
38 of 42 checks passed

github-project-automation bot moved this from 👀 In review to ✅ Done in ⚡️ Apache APISIX Roadmap Aug 25, 2025

mikyll mentioned this pull request Sep 4, 2025

[Site]: Some PR links are broken in monthly reports apache/apisix-website#1952

Closed

jizhuozhi pushed a commit to jizhuozhi/apisix that referenced this pull request Oct 18, 2025

feat: allow to use environment variables for openid-connect plugin (a…

d2b7efa

…pache#11451)

feat: allow to use environment variables for openid-connect plugin #11451

feat: allow to use environment variables for openid-connect plugin #11451

Uh oh!

Conversation

darkSheep404 commented Jul 31, 2024

Description

Checklist

Uh oh!

darkSheep404 commented Sep 24, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

shreemaan-abhishek left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

shreemaan-abhishek left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

darkSheep404 commented Sep 26, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

shreemaan-abhishek left a comment

Choose a reason for hiding this comment

Uh oh!

alberto-corrales commented Oct 7, 2024

Uh oh!

darkSheep404 commented Oct 8, 2024

Uh oh!

brmejia left a comment

Choose a reason for hiding this comment

Uh oh!

alberto-corrales commented Dec 17, 2024

Uh oh!

darkSheep404 commented Dec 17, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

darkSheep404 commented Dec 17, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

darkSheep404 commented Dec 24, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

shreemaan-abhishek commented Dec 24, 2024

Uh oh!

darkSheep404 commented Jul 29, 2025

Uh oh!

membphis left a comment

Choose a reason for hiding this comment

Uh oh!

SkyeYoung commented Aug 4, 2025

Uh oh!

darkSheep404 commented Aug 5, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

bzp2010 left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

bzp2010 Aug 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

darkSheep404 commented Sep 24, 2024 •

edited

Loading

darkSheep404 commented Sep 26, 2024 •

edited

Loading

darkSheep404 commented Dec 17, 2024 •

edited

Loading

darkSheep404 commented Dec 17, 2024 •

edited

Loading

darkSheep404 commented Dec 24, 2024 •

edited

Loading

darkSheep404 commented Aug 5, 2025 •

edited

Loading

bzp2010 Aug 22, 2025 •

edited

Loading