Keycloak Upgrade by tylerpotts · Pull Request #3179 · nebari-dev/nebari

tylerpotts · 2025-11-05T22:07:24Z

Reference Issues or PRs

Fixes #2495

What does this implement/fix?

Put a x in the boxes that apply

Bug fix (non-breaking change which fixes an issue)
[] New feature (non-breaking change which adds a feature)
Breaking change (fix or feature that would cause existing features not to work as expected)
Documentation Update
Code style update (formatting, renaming)
Refactoring (no functional changes, no API changes)
Build related changes
Other (please describe):
Software update for Keycloak

Documentation

For new features or enhancements, a corresponding PR has been opened in the documentation repository (if applicable)
- Link to docs PR:

Testing

Did you test the pull request locally?
Did you add new tests?

How to test this PR?

Deploy a cluster based on the 2025.10.1 version
Add a user to keycloak (to verify that there is a user to be backed up)
Run nebari upgrade -c <config_file> to automatically generate a backup of keycloak
Run nebari deploy -c <config_file and the keycloak database will be restored to the new database

Any other comments?

viniciusdc · 2025-11-06T15:53:14Z

That error is handled by us here:

nebari/tests/tests_deployment/utils.py

Lines 47 to 62 in b63040b

    
           def create_jupyterhub_token(note): 
        
               session = get_jupyterhub_session() 
        
               try: 
        
                   # Retrieve the XSRF token from session cookies 
        
                   xsrf_token = session.cookies.get("_xsrf") 
        
               except requests.cookies.CookieConflictError: 
        
                   xsrf_token = session.cookies.get("_xsrf", path="/hub/") 
        
               if not xsrf_token: 
        
                   raise ValueError("XSRF token not found in session cookies.") 
        
               headers = { 
        
                   "Referer": f"https://{constants.NEBARI_HOSTNAME}/hub/token", 
        
                   "X-XSRFToken": xsrf_token, 
        
               }

I have a feeling the client API on the newer keycloak might have moved a few settings to other places, usually, I see those XSRF errors when there is a misconfiguration within the OAuth client. We are currently passing through that stage during deployment, but it might be beneficial to update the provider as well, depending on how the API is called under the hood.

I think something else is erroring out, and it ends up being picked by that try block. I would compare the keycloak clients configuration available on a deploy from this branch vs a current deployment to see if there is anything that pops up -- usually at the redirection URLs.

Since I see some errors with missing groups

tests/tests_deployment/test_conda_store_roles_loaded.py::test_conda_store_roles_loaded_from_keycloak[admin!namespace=analyst,developer!namespace=nebari-git-changed_scopes0] - KeyError: 'nebari-git/*'
FAILED tests/tests_deployment/test_conda_store_roles_loaded.py::test_conda_store_roles_loaded_from_keycloak[admin!namespace=analyst,developer!namespace=invalid-namespace-changed_scopes1] - KeyError: 'analyst/*'
FAILED

- Upgrade from keycloak chart (15.0.2) to keycloakx chart (7.1.3) - Move PostgreSQL from subchart to standalone deployment - Add Upgrade_2025_10_1 step with database backup prompt - Add UPGRADE_STEPS.md with detailed migration guide - Update RELEASE.md with breaking changes and upgrade instructions

src/_nebari/stages/kubernetes_keycloak/__init__.py

viniciusdc · 2026-02-11T00:45:48Z

uhm... when trying to run a deploy against the current version on main to upgrade to this branch, I ended up with two things that called my attention:

the name of the main PVC/PV used by the postgresql DB changed:

NAME↑                                      CAPACITY  ACCESS MODES   RECLAIM POLICY   STATUS   CLAIM                                                 STORAGECLASS
pvc-a842a83c-caf7-49af-ad02-1064491e39ea   8Gi       RWO            Delete           Bound    dev/data-keycloak-postgresql-0                        standard-rwo
pvc-e905d783-d8ec-442a-9817-5f7dd11ed515   8Gi       RWO            Delete           Bound    dev/data-keycloak-postgres-standalone-postgresql-0

Got an error with realm not found during the keycloak configuration stage:

[tofu]: Changes to Outputs:
[tofu]:   ~ realm_id                            = "nebari" -> (known after apply)
[tofu]: ╷
[tofu]: │ Error: error sending GET request to /auth/admin/realms/nebari/default-groups: 404 Not Found. Response body: {"error":"Realm not found."}
[tofu]: │
[tofu]: │   with keycloak_default_groups.default,
[tofu]: │   on main.tf line 84, in resource "keycloak_default_groups" "default":
[tofu]: │   84: resource "keycloak_default_groups" "default" {
[tofu]: │
[tofu]: ╵
Error: OpenTofu command failed

The first one makes sense with the direction we took for this upgrade:

# Deployed separately to allow safe upgrade from keycloak to keycloakx chart

viniciusdc · 2026-02-11T11:38:26Z

Disregard that: because I was using the main branch version, the upgrade migration workflow was not triggered, since the version only differed by the commit hash... Rookie mistake hahaha, will re-test now

viniciusdc · 2026-02-11T16:12:06Z

This is nice, but it does not include the command for the user to perform the backup after the deployment suceeds and what to do after it (e.g after everything works, and they want to remove the old instance for example):

❯ nebari upgrade -c nebari-config.yaml

---> Starting upgrade from 2025.10.2 to 2026.02.1

Setting nebari_version to 2026.02.1

 ⚠️  CRITICAL UPGRADE WARNING ⚠️

This version includes a major Keycloak upgrade from the keycloak chart (15.0.2)
to the keycloakx chart (7.1.3). This upgrade changes the underlying architecture
from JBoss/WildFly to Quarkus.

IMPORTANT: The old keycloak chart includes PostgreSQL as a subchart dependency,
but the new keycloakx chart does not. To prevent data loss, you MUST backup your
Keycloak PostgreSQL database BEFORE running nebari deploy.

Key changes:
- PostgreSQL moved from subchart to standalone deployment
- Keycloak service name changes from keycloak-headless to keycloak-keycloakx-http
- OAuth clients now require openid scope explicitly

After this upgrade step completes, you will need to run:
      nebari deploy -c nebari-config.yaml to apply the changes


Below are available kubectl contexts for kubernetes. Please select the context for your cluster
  1. ****
  ....
  22. gke_nebari-dev-a****  (current)
  23. Desired context not present

Select context number [1/2/3/4/5/6/7/8/9/10/11/12/13/14/15/16/17/18/19/20/21/22/23]: 22

✓ Using context: gke_nebari-dev-a***** -> gke_nebari-dev-a****

✓ Kubernetes configuration detected. Nebari can backup the database automatically.

Would you like Nebari to backup the Keycloak database for you now? [y/n] (y): y

Backing up Keycloak database using Kubernetes API...

Creating backup at: keycloak-backup.sql

✓ Backup successful! Saved to keycloak-backup.sql
  Backup size: 313.17 KB

✓ Database backup completed. You can now proceed with:
  nebari deploy -c nebari-config.yaml
Ready to upgrade to Nebari version 2026.02.1.



---> Starting upgrade from 2026.02.1 to 2026.2.2

Setting nebari_version to 2026.2.2


Saving new config file nebari-config.yaml ready for Nebari version 2026.2.2.dev0+ge4f6c5bf4.d20260211

viniciusdc · 2026-02-11T17:24:45Z

working beautifully well

[tofu]: Apply complete! Resources: 2 added, 2 changed, 0 destroyed.
[tofu]:
[tofu]: Outputs:
[tofu]:
[tofu]: keycloak_credentials = <sensitive>
[tofu]: keycloak_nebari_bot_password = <sensitive>

================================================================================
KEYCLOAK DATABASE RESTORE
================================================================================
Found backup file: /.../keycloak-upgrade/keycloak-backup.sql
Size: 313.17 KB

Step 0: Scaling down Keycloak statefulset 'keycloak-keycloakx' to 0 replicas...
  Scaled to 0 replicas
  Waiting for Keycloak pods to terminate...
  ✓ All Keycloak pods terminated
✓ Keycloak scaled down

Checking if pod 'keycloak-postgres-standalone-postgresql-0' exists in namespace 'dev'...
✓ Pod found

Getting database passwords from secret...
✓ Got database passwords

Step 1: Dropping database 'keycloak' (if exists)...
  Running: env PGPASSWORD=*** psql -U postgres -c 'DROP DATABASE IF EXISTS keycloak;'
DROP DATABASE
✓ Database dropped

Step 2: Creating fresh database 'keycloak'...
  Running: env PGPASSWORD=*** psql -U postgres -c 'CREATE DATABASE keycloak;'
CREATE DATABASE
✓ Database created

Step 3: Granting privileges to user 'keycloak'...
  Running: env PGPASSWORD=*** psql -U postgres -c 'GRANT ALL PRIVILEGES ON DATABASE keycloak TO keycloak;'
GRANT
  Running: env PGPASSWORD=*** psql -U postgres -d keycloak -c 'GRANT ALL ON SCHEMA public TO keycloak;'
GRANT
  Running: env PGPASSWORD=*** psql -U postgres -d keycloak -c 'ALTER DEFAULT PRIVILEGES IN SCHEMA public GRANT ALL ON TABLES TO keycloak;'
ALTER DEFAULT PRIVILEGES
✓ Privileges granted

Step 4: Copying backup file to pod...
  Copying keycloak-backup.sql to pod:keycloak-backup.sql
  File size: 313.17 KB
  ✓ File copied successfully
✓ Backup file copied to pod

Step 5: Restoring database from backup...
This may take a few moments. Output will be shown below:

Note: Warnings about 'public' schema permissions are expected and harmless.
================================================================================
  Running: env PGPASSWORD=*** psql -U keycloak -d keycloak --set ON_ERROR_STOP=off -f /tmp/keycloak-backup.sql
SET
SET
SET
SET
SET
 set_config
------------

(1 row)

SET
SET
SET
SET
SET
SET
CREATE TABLE
ALTER TABLE
CREATE TABLE
ALTER TABLE
CREATE TABLE
ALTER TABLE
CREATE TABLE
ALTER TABLE
CREATE TABLE
ALTER TABLE
CREATE TABLE
ALTER TABLE
CREATE TABLE
ALTER TABLE
CREATE TABLE
ALTER TABLE
CREATE TABLE
ALTER TABLE
CREATE TABLE
ALTER TABLE
CREATE TABLE
ALTER TABLE
CREATE TABLE
ALTER TABLE
CREATE TABLE
ALTER TABLE
CREATE TABLE
ALTER TABLE
CREATE TABLE
ALTER TABLE
CREATE TABLE
ALTER TABLE
CREATE TABLE
ALTER TABLE
CREATE TABLE
ALTER TABLE
CREATE TABLE
ALTER TABLE
CREATE TABLE
ALTER TABLE
CREATE TABLE
ALTER TABLE
CREATE TABLE
ALTER TABLE
CREATE TABLE
ALTER TABLE
CREATE TABLE
ALTER TABLE
CREATE TABLE
ALTER TABLE
CREATE TABLE
ALTER TABLE
CREATE TABLE
ALTER TABLE
CREATE TABLE
ALTER TABLE
CREATE TABLE
ALTER TABLE
CREATE TABLE
ALTER TABLE
CREATE TABLE
ALTER TABLE
CREATE TABLE
ALTER TABLE
CREATE TABLE
ALTER TABLE
CREATE TABLE
ALTER TABLE
CREATE TABLE
ALTER TABLE
CREATE TABLE
ALTER TABLE
CREATE TABLE
ALTER TABLE
CREATE TABLE
ALTER TABLE
CREATE TABLE
ALTER TABLE
CREATE TABLE
ALTER TABLE
CREATE TABLE
ALTER TABLE
CREATE TABLE
ALTER TABLE
CREATE TABLE
ALTER TABLE
CREATE TABLE
ALTER TABLE
CREATE TABLE
ALTER TABLE
CREATE TABLE
ALTER TABLE
CREATE TABLE
ALTER TABLE
CREATE TABLE
ALTER TABLE
CREATE TABLE
ALTER TABLE
CREATE TABLE
ALTER TABLE
CREATE TABLE
ALTER TABLE
CREATE TABLE
ALTER TABLE
CREATE TABLE
ALTER TABLE
CREATE TABLE
ALTER TABLE
CREATE TABLE
ALTER TABLE
CREATE TABLE
ALTER TABLE
CREATE TABLE
ALTER TABLE
CREATE TABLE
ALTER TABLE
CREATE TABLE
ALTER TABLE
CREATE TABLE
ALTER TABLE
CREATE TABLE
ALTER TABLE
CREATE TABLE
ALTER TABLE
CREATE TABLE
ALTER TABLE
CREATE TABLE
ALTER TABLE
CREATE TABLE
ALTER TABLE
CREATE TABLE
ALTER TABLE
CREATE TABLE
ALTER TABLE
CREATE TABLE
ALTER TABLE
CREATE TABLE
ALTER TABLE
CREATE TABLE
ALTER TABLE
CREATE TABLE
ALTER TABLE
CREATE TABLE
ALTER TABLE
CREATE TABLE
ALTER TABLE
CREATE TABLE
ALTER TABLE
CREATE TABLE
ALTER TABLE
CREATE TABLE
ALTER TABLE
CREATE TABLE
ALTER TABLE
CREATE TABLE
ALTER TABLE
CREATE TABLE
ALTER TABLE
CREATE TABLE
ALTER TABLE
CREATE TABLE
ALTER TABLE
CREATE TABLE
ALTER TABLE
CREATE TABLE
ALTER TABLE
CREATE TABLE
ALTER TABLE
CREATE TABLE
ALTER TABLE
CREATE TABLE
ALTER TABLE
CREATE TABLE
ALTER TABLE
CREATE TABLE
ALTER TABLE
CREATE TABLE
ALTER TABLE
CREATE TABLE
ALTER TABLE
CREATE TABLE
ALTER TABLE
CREATE TABLE
ALTER TABLE
COPY 78
COPY 0
COPY 96
COPY 41
COPY 4
COPY 4
COPY 0
COPY 17
COPY 24
COPY 0
COPY 0
COPY 0
COPY 18
COPY 48
COPY 128
COPY 2
COPY 0
COPY 0
COPY 0
COPY 0
COPY 0
COPY 0
COPY 24
COPY 72
COPY 76
COPY 6
COPY 104
COPY 3
COPY 18
COPY 2
COPY 0
COPY 0
COPY 0
COPY 0
COPY 0
COPY 0
COPY 0
COPY 0
COPY 0
COPY 0
COPY 21
COPY 0
COPY 0
COPY 0
COPY 0
COPY 8
COPY 91
COPY 1
COPY 0
COPY 0
COPY 0
COPY 72
COPY 376
COPY 2
COPY 67
COPY 2
COPY 0
COPY 3
COPY 0
COPY 2
COPY 0
COPY 0
COPY 11
COPY 0
COPY 14
COPY 0
COPY 0
COPY 0
COPY 0
COPY 0
COPY 0
COPY 0
COPY 0
COPY 0
COPY 10
COPY 2
COPY 0
COPY 0
COPY 0
COPY 0
COPY 9
COPY 0
COPY 0
COPY 0
COPY 0
COPY 16
COPY 0
COPY 17
COPY 0
COPY 0
COPY 0
COPY 3
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
CREATE INDEX
CREATE INDEX
CREATE INDEX
CREATE INDEX
CREATE INDEX
CREATE INDEX
CREATE INDEX
CREATE INDEX
CREATE INDEX
CREATE INDEX
CREATE INDEX
CREATE INDEX
CREATE INDEX
CREATE INDEX
CREATE INDEX
CREATE INDEX
CREATE INDEX
CREATE INDEX
CREATE INDEX
CREATE INDEX
CREATE INDEX
CREATE INDEX
CREATE INDEX
CREATE INDEX
CREATE INDEX
CREATE INDEX
CREATE INDEX
CREATE INDEX
CREATE INDEX
CREATE INDEX
CREATE INDEX
CREATE INDEX
CREATE INDEX
CREATE INDEX
CREATE INDEX
CREATE INDEX
CREATE INDEX
CREATE INDEX
CREATE INDEX
CREATE INDEX
CREATE INDEX
CREATE INDEX
CREATE INDEX
CREATE INDEX
CREATE INDEX
CREATE INDEX
CREATE INDEX
CREATE INDEX
CREATE INDEX
CREATE INDEX
CREATE INDEX
CREATE INDEX
CREATE INDEX
CREATE INDEX
CREATE INDEX
CREATE INDEX
CREATE INDEX
CREATE INDEX
CREATE INDEX
CREATE INDEX
CREATE INDEX
CREATE INDEX
CREATE INDEX
CREATE INDEX
CREATE INDEX
CREATE INDEX
CREATE INDEX
CREATE INDEX
CREATE INDEX
CREATE INDEX
CREATE INDEX
CREATE INDEX
CREATE INDEX
CREATE INDEX
CREATE INDEX
CREATE INDEX
CREATE INDEX
CREATE INDEX
CREATE INDEX
CREATE INDEX
CREATE INDEX
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
psql:/tmp/keycloak-backup.sql:5682: WARNING:  no privileges could be revoked for "public"
REVOKE
psql:/tmp/keycloak-backup.sql:5683: WARNING:  no privileges could be revoked for "public"
REVOKE
psql:/tmp/keycloak-backup.sql:5684: WARNING:  no privileges were granted for "public"
GRANT
psql:/tmp/keycloak-backup.sql:5685: WARNING:  no privileges were granted for "public"
GRANT
================================================================================

✓ Restore completed!

Step 6: Verifying restore by checking user count...
  Running: env PGPASSWORD=*** psql -U keycloak -d keycloak -c 'SELECT count(*) FROM user_entity;'
 count
-------
     9
(1 row)

✓ Verification complete

Step 7: Cleaning up temporary file in pod...
  Running: rm -f /tmp/keycloak-backup.sql
✓ Removed /tmp/keycloak-backup.sql

================================================================================
DATABASE RESTORE SUCCESSFUL!
================================================================================

Step 8: Scaling Keycloak statefulset back to 1 replicas...
✓ Keycloak scaled back to 1 replicas
  Waiting for Keycloak to be ready...
  Still waiting... (0/1 ready)
  Still waiting... (0/1 ready)
  Still waiting... (0/1 ready)
  Still waiting... (0/1 ready)
  Still waiting... (0/1 ready)
  Still waiting... (0/1 ready)
  Still waiting... (0/1 ready)
  Still waiting... (0/1 ready)
  Still waiting... (0/1 ready)
  ✓ Keycloak is ready (1/1 replicas)
  Keycloak pods are ready and connected to the restored database


✓ Renamed backup file to /.../keycloak-upgrade/keycloak-backup.sql.restored
================================================================================

Creating nebari-bot user in Keycloak master realm...
/Users/vinicius/miniconda3/envs/nebari-dev/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'vini-kc.nebari.dev'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings
  warnings.warn(
/Users/vinicius/miniconda3/envs/nebari-dev/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'vini-kc.nebari.dev'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings
  warnings.warn(
nebari-bot user already exists
/Users/vinicius/miniconda3/envs/nebari-dev/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'vini-kc.nebari.dev'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings
  warnings.warn(
Updated nebari-bot password to match expected value
/Users/vinicius/miniconda3/envs/nebari-dev/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'vini-kc.nebari.dev'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings
  warnings.warn(
/Users/vinicius/miniconda3/envs/nebari-dev/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'vini-kc.nebari.dev'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings
  warnings.warn(
nebari-bot user already has admin role
/Users/vinicius/miniconda3/envs/nebari-dev/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'vini-kc.nebari.dev'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings
  warnings.warn(
Attempt 1 succeeded connecting to keycloak master realm
Keycloak service successfully started

Requested changes have been implemented

viniciusdc

LGTM, tested on live deployments of both GCP and AWS, upgrade command steps were clear as well as the migration.

tylerpotts requested a review from a team as a code owner November 5, 2025 22:07

tylerpotts requested review from dcmcand and marcelovilla and removed request for a team November 5, 2025 22:07

github-project-automation bot added this to 🪴 Nebari Project Management Nov 5, 2025

github-project-automation bot moved this to New 🚦 in 🪴 Nebari Project Management Nov 5, 2025

tylerpotts changed the title ~~Keycloak Upgrade~~ [WIP] Keycloak Upgrade Nov 5, 2025

tylerpotts changed the title ~~[WIP] Keycloak Upgrade~~ Keycloak Upgrade Nov 13, 2025

tylerpotts mentioned this pull request Nov 13, 2025

Add keycloak tests #3157

Merged

11 tasks

Tyler Potts added 20 commits November 17, 2025 15:34

keycloak deploying. API endpoints not working

a2b5e37

startup script was deprecated. Add nebari-bot in python

8c0ca6c

nebari deploying fully now. Auth not working for apps

29348f6

add openid scope to fix jhub keycloak

ba57293

add openid to fix grafana

e2f142d

updates for conda-store to access keycloak

60976bd

add retry mechanism for nebari bot

954a0fd

remove force replacement that was used for dev

7e74f58

add database configuration into values for keycloak

3c4e90d

add standalone postgres because new keycloak doesn't bundle

cede549

bump helm provider version, add standalone postgres

042c736

add upgrade command leveraging python-kubernetes

8f24426

use python kubernetes for keycloak backup instead of kubectl

a165286

add code to automatically restore database in post_deploy

e1264fa

update upgrade step version

a4c2416

copy sql restore file as tar instead of standard in

2c56491

clean up unneeded notes and documentation

c4c1ead

add templating for keycloak external URL

d3cf6e3

Clean up step naming in restore function

c394eb5

Tyler Potts added 4 commits November 17, 2025 15:34

add playwright test updates for new keycloak logout flow

364ac3a

update all keycloak providers in Nebari

d4350a7

add context choices for upgrade step, sleep for keycloak restart

a3d4f96

update test_upgrade file for latest upgrade command

9b77ba9

tylerpotts force-pushed the 2495_rebased branch from 12e9815 to 9b77ba9 Compare November 17, 2025 22:36

Tyler Potts added 3 commits November 18, 2025 09:42

mock helm subprocess, only check for keycloak if version>2025.11.1

9828d4d

add proper scopes to forwardauth to fix dask gateway

0494ee2

clean up upgrade notice

8c26204

dcmcand previously requested changes Nov 25, 2025

View reviewed changes

github-project-automation bot moved this from New 🚦 to Changes requested 🧱 in 🪴 Nebari Project Management Nov 25, 2025

Tyler Potts added 6 commits November 26, 2025 10:44

add backup failure handling

cd09090

change stateful set to always be 1

ea07ef1

mask postgres passwords in console output

04e22c1

move imports to top of file

d47364b

add logic for finding postgres and keycloak podnames

167b85e

raise error on failure to get database password

dfb12dc

tylerpotts requested a review from dcmcand November 26, 2025 18:42

Merge branch 'main' into 2495_rebased

2f58538

viniciusdc added this to the 2026.02.1 milestone Jan 20, 2026

Merge branch 'main' into 2495_rebased

e4f6c5b

Merge branch 'main' into 2495_rebased

7057e10

viniciusdc approved these changes Mar 10, 2026

View reviewed changes

viniciusdc merged commit cbb5567 into main Mar 10, 2026
27 checks passed

viniciusdc deleted the 2495_rebased branch March 10, 2026 12:45

github-project-automation bot moved this from Changes requested 🧱 to Done 💪🏾 in 🪴 Nebari Project Management Mar 10, 2026

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Keycloak Upgrade#3179

Keycloak Upgrade#3179
viniciusdc merged 46 commits intomainfrom
2495_rebased

tylerpotts commented Nov 5, 2025

Uh oh!

viniciusdc commented Nov 6, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

viniciusdc commented Feb 11, 2026 •

edited

Loading

Uh oh!

viniciusdc commented Feb 11, 2026

Uh oh!

viniciusdc commented Feb 11, 2026

Uh oh!

viniciusdc commented Feb 11, 2026

Uh oh!

viniciusdc left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

tylerpotts commented Nov 5, 2025

Reference Issues or PRs

What does this implement/fix?

Documentation

Testing

How to test this PR?

Any other comments?

Uh oh!

viniciusdc commented Nov 6, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

viniciusdc commented Feb 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

viniciusdc commented Feb 11, 2026

Uh oh!

viniciusdc commented Feb 11, 2026

Uh oh!

viniciusdc commented Feb 11, 2026

Uh oh!

viniciusdc left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

viniciusdc commented Feb 11, 2026 •

edited

Loading