Conversation
…port - Detect if OIDC is enabled by checking for placeholders in config - Only validate OIDC credentials when OIDC is actually enabled - Fail fast with helpful error messages when OIDC enabled but credentials missing - Show troubleshooting steps with exact kubectl commands - Validate substitution worked (no remaining placeholders) - Check for empty client_id/client_secret that cause nil errors - Mask sensitive values in logs (show first 4 chars only) - Keep secrets as optional: true (OIDC is opt-in feature) Fixes customer issue where missing oidcGithubClientId/oidcGithubClientSecret in Kubernetes secret resulted in nil values and Kratos startup failures. The init container now: - Skips validation when OIDC is disabled - Provides clear guidance when OIDC is enabled but secrets are missing: 1. Which secret name to look for (templated with release name) 2. Required secret keys (oidcGithubClientId, oidcGithubClientSecret) 3. How to check ExternalSecret status 4. How to create the secret manually 5. How to disable OIDC if not needed Tested with: - OIDC enabled + missing credentials (fails with helpful errors) ✅ - OIDC enabled + empty credentials (fails with helpful errors) ✅ - OIDC enabled + partial credentials (fails with helpful errors) ✅ - OIDC enabled + valid credentials (succeeds with validation) ✅ - OIDC disabled + no credentials (succeeds, skips checks) ✅ Changes: - charts/kratos/templates/deployment-kratos.yaml: Updated init container logic - charts/kratos/Chart.yaml: Bump version 1.6.31 -> 1.6.32 - charts/judge/Chart.yaml: Bump version 1.8.40 -> 1.8.41, update kratos dependency Co-Authored-By: Claude <noreply@anthropic.com>
When OIDC credentials are missing, the error message now shows the complete list of required Kratos secret keys, not just the OIDC ones. Required secrets shown to customers: - dsn (Database connection string) - REQUIRED - secretsCookie (Cookie encryption) - REQUIRED - secretsCipher (Field encryption) - REQUIRED - oidcGithubClientId (GitHub OAuth) - Required only if OIDC enabled - oidcGithubClientSecret (GitHub OAuth) - Required only if OIDC enabled Optional secrets NOT shown (to avoid confusion): - secretsDefault - Auto-generated by Kratos if missing (verified via source) - smtpConnectionURI - Only needed if SMTP configured Also updated the example kubectl command to show how to create the complete secret with all required keys including random generation for cookie/cipher. Changes: - charts/kratos/templates/deployment-kratos.yaml: Enhanced error messaging - charts/kratos/Chart.yaml: Bump version 1.6.32 -> 1.6.33 - charts/judge/Chart.yaml: Bump version 1.8.41 -> 1.8.42, update dependency Co-Authored-By: Claude <noreply@anthropic.com>
…and -hex 16) Changed secret generation command from complex tr/urandom to simple openssl: - OLD: LC_ALL=C tr -dc 'A-Za-z0-9' < /dev/urandom | head -c 32 - NEW: openssl rand -hex 16 This produces exactly 32 hexadecimal characters, which: ✅ Meets secretsCipher requirement (MUST be exactly 32 chars) ✅ Meets secretsCookie recommendation (min 16, recommended 32) ✅ Matches existing README.md documentation ✅ Simpler and more portable than tr/urandom approach Updated error message details: - secretsCookie: min 16 chars, 32 recommended (per Kratos schema) - secretsCipher: MUST be exactly 32 chars (validated by Kratos) - Both use same command for consistency Verified against Kratos source code: - secretsCipher validation: config.go lines 831-849 - Filters out secrets that aren't exactly 32 bytes - secretsCookie minimum: config.schema.json (16 char minimum) Changes: - charts/kratos/templates/deployment-kratos.yaml: Updated secret generation - charts/kratos/Chart.yaml: Bump version 1.6.33 -> 1.6.34 - charts/judge/Chart.yaml: Bump version 1.8.42 -> 1.8.43, update dependency Co-Authored-By: Claude <noreply@anthropic.com>
Resolved conflicts by merging main changes with our OIDC validation fix. Version bumps to account for both sets of changes: - charts/kratos/Chart.yaml: 1.6.31 (main) + our changes → 1.6.35 - charts/judge/Chart.yaml: 1.8.40 (main) + our changes → 1.8.44 Changes from main (merged): - archivista 1.6.19 → 1.6.20 - judge-api 1.6.19 → 1.6.20 - kratos template improvements (_helpers.tpl, cleanup-cron-job, etc.) - judge manual secrets configuration Our changes (preserved): - Improved Kratos init container OIDC validation - Better error messages with all required secrets listed - Correct secret generation (openssl rand -hex 16) Rebuilt dependencies with kratos 1.6.35.
|
❌ Helm Dependencies Check Failed Stale Helm dependencies detected! This means .tgz files are older than source files. How to fix:
Why this matters: Stale dependencies cause ArgoCD to deploy outdated configs, leading to issues like missing Vault annotations. See the Makefile for more details. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes customer issue where missing OIDC credentials resulted in
nilvalues and Kratos startup failures with unhelpful error messages.Problem
Customer reported this error:
Root cause: The init container was replacing
${OIDC_GITHUB_CLIENT_ID}placeholders with empty strings when environment variables weren't set, producing nil values.Solution
Improved the Kratos init container to:
openssl rand -hex 16(32 hex chars exactly)Error Message Now Shows
Testing
Validated using TDD/TCR methodology with black-box tests:
Changes
Secret Length Requirements
Verified against Kratos source code:
secretsCipher: MUST be exactly 32 chars (validated at config.go:831-849)secretsCookie: Min 16 chars, 32 recommended (per config.schema.json)openssl rand -hex 16produces exactly 32 hex charactersCommits
feb360a- Main fix with conditional OIDC validatione8e86db- Show ALL required secret keys in error92070af- Use correct secret generation (openssl rand -hex 16)Co-Authored-By: Claude noreply@anthropic.com