SAML Auth is broken after the 4.19.2 upgrade

[OlegChuev]

problem

My application serves as a SAML Identity Provider (IdP) and reverse proxy for accessing the ACS.
After the recent update to version 4.19.2, the SAML integration has stopped working correctly — authorization now fails with a 401 Unauthorized response (please refer to the attached videos)

Flow Overview:

1. User tries to access CloudStack UI (e.g., /cloudstack/client/):
   • The request is routed through the app, which acts as a reverse proxy
   • If the user is not signed in, they see an unauthorized page with a link to sign in.
2. CloudStack redirects to IdP for authentication:
   • CloudStack (as SP) redirects the user to /saml/auth on the app, passing a SAMLRequest.
3. IdP handles the SAMLRequest:
   • My app validates the request and renders a form that auto-submits the SAMLRequest.
   • If the user is authenticated, app generates a SAMLResponse and posts it back to CloudStack’s Assertion Consumer Service (ACS) endpoint. If not authenticated, the user is prompted to log in.
4. CloudStack consumes the SAMLResponse:
   • CloudStack validates the SAMLResponse, logs the user in, and grants access to the UI.

User ---> My App (IdP) ←→ CloudStack (SP)
   |         |                        |
   |         |---- SAML Auth -------->|
   |         |<--- SAMLRequest -------|
   |         |---- SAMLResponse ----->|
   |         |                        |
   |---- Reverse Proxy to CloudStack -|

The change introduced in PR #10047, particularly this line:

resp.addHeader("SET-COOKIE", String.format("%s=%s;HttpOnly;Path=/client/api;%s", ApiConstants.SESSIONKEY, loginResponse.getSessionKey(), sameSite));

…seems to break the flow by setting the HttpOnly flag on the sessionkey cookie.
As a result, the sessionkey is no longer accessible to the frontend JavaScript (ui/src/api/index.js), which expects to be able to read it (as far as I understood at least).

This appears to conflict with an earlier change made in this commit, where it looks like the frontend relies on accessing the sessionKey via JS

---

Behavior with HttpOnly header:

4.19.2_with_header.mov

Behavior without HttpOnly header:

4.19.2_without_header.mov

---

I also have tried tweaking api.sessionkey.check.locations and api.sessionkey.cookie.samesite, but none of the options helped. I also attached an example of my SAML settings, perhaps that might help.

I’d really appreciate your thoughts on what might be the most suitable solution in this case.

versions

4.19.2.0

The steps to reproduce the bug

1. Use your application as SAML IdP
2. Authorize request in your application and redirect to the login page

Actual result: 401 unauthorized
Expected result: user is authorized as expected

What to do about it?

I tried removing the HttpOnly flag from resp.addHeader(...), and it seems to resolve the issue.
To be honest, I’m not entirely sure how to approach this properly since I'm not really familiar with java and ACS logic

[harikrishna-patnala]

@OlegChuev thanks for the details. May I ask few more details and a way to try out.

1. From which version you have upgraded to 4.19.2 ? and I assume before upgrade it used to work with the same setup, if not please let us know.
2. Can you try taking out the reverse proxy, just to see if it works without it.
3. Can you also try setting "api.sessionkey.cookie.samesite=Null"

[OlegChuev]

hey @harikrishna-patnala, thank you for the quick response!

1. We were upgrading from 4.19.1.1 to 4.19.2, and yes – everything was working smoothly with the same setup prior to the upgrade
2. I’ll give it a try, though I’m not entirely sure if it’s feasible in my current environment
3. I’ve tested various samesite options, but unfortunately, none of them resolved the issue :(

[rohityadavcloud]

Is this considered a blocker cc @Pearl1594 @DaanHoogland ?

@OlegChuev our QA server uses https://mocksaml.com/ for testing purposes, is that good enough or should we look at keycloak or some other IdP server? What IdP server are you using?

[weizhouapache]

@rg9975
Have you faced the same issue ?

It looks like you are using SAML, as you are the author of #10311

[weizhouapache]

Is this considered a blocker cc @Pearl1594 @DaanHoogland ?

@OlegChuev our QA server uses https://mocksaml.com/ for testing purposes, is that good enough or should we look at keycloak or some other IdP server? What IdP server are you using?

@rohityadavcloud
let's try to reproduce it

@OlegChuev
the old behaviour is same as

• api.sessionkey.check.locations=CookieOrParameter
• api.sessionkey.cookie.samesite=Null

can you re-test ?

for your information, below are configurations on qa cloud

[rg9975]

@rg9975 Have you faced the same issue ?

It looks like you are using SAML, as you are the author of #10311

We have not rev'd to 4.19.2 yet.

[OlegChuev]

Is this considered a blocker cc @Pearl1594 @DaanHoogland ?

@OlegChuev our QA server uses mocksaml.com for testing purposes, is that good enough or should we look at keycloak or some other IdP server? What IdP server are you using?

My application functions as the IdP server, there’s no external service involved — only the application itself. I believe mocksaml should be more than sufficient in this case

[OlegChuev]

Is this considered a blocker cc @Pearl1594 @DaanHoogland ?
@OlegChuev our QA server uses mocksaml.com for testing purposes, is that good enough or should we look at keycloak or some other IdP server? What IdP server are you using?

@rohityadavcloud let's try to reproduce it

@OlegChuev the old behaviour is same as

• api.sessionkey.check.locations=CookieOrParameter
• api.sessionkey.cookie.samesite=Null

can you re-test ?

for your information, below are configurations on qa cloud

I’ve tested with both CookieOrParameter and Null and can confirm that neither resolves the issue. I’m starting to suspect that the root cause might lie in an insufficient Nginx configuration. However, what’s most puzzling is that everything seems to work fine when the HttpOnly header is removed...

[weizhouapache]

I’ve tested with both CookieOrParameter and Null and can confirm that neither resolves the issue. I’m starting to suspect that the root cause might lie in an insufficient Nginx configuration. However, what’s most puzzling is that everything seems to work fine when the HttpOnly header is removed...

however, we cannot remove HttpOnly, it will cause severe security issue.

has the nginx config updated recently ? especially settings for cookie

[OlegChuev]

@weizhouapache I can confirm that neither the Nginx configs nor the network configuration have changed. We’ve temporarily rolled back to the previous working version (4.19.1.1)

Let’s keep this issue open in case anyone else runs into the same problem

[DaanHoogland]

@OlegChuev the behaviour had been changed because of sec issues but then some regressions were introduced. 4.19.2 already had a fix but obviously not for your issue. 4.19.3 and 4.20.1 contain #10311 which might address your issue.