Skip to content

Add CryptoEnvironmentProvider with AES-256-GCM encryption#675

Open
chhil wants to merge 20 commits intojpos:mainfrom
TransactilityInc:crypto-environment-provider
Open

Add CryptoEnvironmentProvider with AES-256-GCM encryption#675
chhil wants to merge 20 commits intojpos:mainfrom
TransactilityInc:crypto-environment-provider

Conversation

@chhil
Copy link
Contributor

@chhil chhil commented Feb 21, 2026

jPOS Cryptographic Implementation Guide

Overview

jPOS implements AES-256-GCM authenticated encryption for securing sensitive configuration data. This system allows encrypted values to be stored in configuration files while automatically decrypting them at runtime.

Key Components

1. SystemKeyManager

The SystemKeyManager is a singleton class that manages encryption keys. It supports multiple named keys for different encryption contexts.

  • Key Generation: Uses new SecureRandom() (OS-provided entropy) to generate cryptographically random 256-bit keys
  • AES-256: Keys are generated using the AES algorithm with 256-bit key size
  • Multiple Key Support: Can manage multiple named keys (default + custom names)
  • Stateless/No Caching: Keys are loaded directly from environment variables on every use, enabling immediate key rotation without restarting the application.

2. CryptoEnvironmentProvider

This class provides encryption/decryption services for configuration values. It uses the SystemKeyManager to retrieve keys and implements AES-256-GCM encryption.

  • Format: enc::[keyname]:[base64-encoded-iv+ciphertext]
  • Default key: enc::[base64-encoded-iv+ciphertext]
  • Named keys: enc::keyname:[base64-encoded-iv+ciphertext]

Configuration Setup

Step 1: Generate a Key

Use the CRYPTO CLI command to generate a key. Keys are generated but never cached in memory - you must save the output.

jpos crypto generate

This will output:

=== Key Generated ===
Key Name: default
Environment Variable: JPOS_ENCRYPTION_KEY
Key (Base64): [base64-encoded-key]
=====================

For named keys, use:

jpos crypto generate db

This will output:

=== Key Generated ===
Key Name: db
Environment Variable: JPOS_ENCRYPTION_KEY_DB
Key (Base64): [base64-encoded-key]
=====================

Step 2: Set Environment Variable

Set the environment variable with the Base64 output from the previous step. This is mandatory, as jPOS will read the key from the environment on every encryption/decryption operation.

export JPOS_ENCRYPTION_KEY=[base64-encoded-key]

For named keys, use:

export JPOS_ENCRYPTION_KEY_DB=[base64-encoded-key]

(Note: For development or testing, you can also set these as Java system properties: -DJPOS_ENCRYPTION_KEY_DB=[base64-encoded-key])

Step 3: Encrypt Configuration Values

Ensure the environment variable is set in your current session, then encrypt sensitive values using the CRYPTO command:

jpos crypto "my-database-password" db

This will output an encrypted value like:

enc::db:2w20mm7Nkx/d03oLGEgbmXkfPKZ9P9vCzp//1bgAsCD7qTEcj/F2ub5nwtLqHO10

Step 4: Use in Configuration Files

Add the encrypted value to your configuration files:

db.properties

hibernate.connection.password=enc::db:kCoa31d4jO1EBQhF7WOzAR7iwfS1RwDrrn/iWRiL1Y4grtPwyfTCwuDqipuBnq1Ffg==

default.properties

JPOS_ENCRYPTION_KEY_DB=$env{JPOS_ENCRYPTION_KEY_DB}

default.yml

JPOS_ENCRYPTION_KEY_DB: $env{JPOS_ENCRYPTION_KEY_DB}

Encryption Format Details

The encrypted format follows this pattern:

  • enc:: - Prefix indicating encrypted content
  • [keyname]: - Optional key name (for named keys)
  • [base64-encoded-iv+ciphertext] - Base64-encoded initialization vector + ciphertext

The IV (12 bytes) is prepended to the ciphertext for decryption.

Security Features

  • AES-256-GCM authenticated encryption (prevents tampering)
  • 128-bit authentication tag for integrity verification
  • Random 12-byte IV per encryption (different output each time for same input)
  • Strict Environment Loading (no internal caching, enabling secure, real-time key rotation)
  • Tamper detection via GCM tag (throws exception on modification)

Usage Examples

Using q2 CLI

# Start q2 shell
$ cd jpos
$ ./gradlew q2 -Dq2.home=$(pwd)

# Generate a named key for database
q2> crypto generate db
=== Key Generated ===
Key Name: db
Environment Variable: JPOS_ENCRYPTION_KEY_DB
Key (Base64): 5G+HWiKUhIn3aSeyf6YRJUn9vhPUA4RV6Y4+S9gkc90=
=====================

# IMPORTANT: You must exit q2, export the key, and restart q2, 
# OR set it as a system property when starting q2:
# ./gradlew q2 -Dq2.home=$(pwd) -DJPOS_ENCRYPTION_KEY_DB=5G+HWiKUhIn3aSeyf6YRJUn9vhPUA4RV6Y4+S9gkc90=

# Encrypt using the named key (requires the env var or property to be set)
q2> crypto "jdbc:mysql:acme:acme/jpos_acme?useSSL=false" db
enc::db:hXR64gSbru2XeXDbt2n9zaU/thS/k0CVsek3eqqAK24AfjsOagMC9Q+ATpvLr1UmOVSgSUaL7z3j2vZ5NVnD1nZka60tmcM=

# Encrypt a password
q2> crypto "my-database-password" db
enc::db:2w20mm7Nkx/d03oLGEgbmXkfPKZ9P9vCzp//1bgAsCD7qTEcj/F2ub5nwtLqHO10

Using Java Code

import org.jpos.core.CryptoEnvironmentProvider;

// Ensure the environment variable or system property is set before calling
// System.setProperty("JPOS_ENCRYPTION_KEY", "your-base64-key");
// System.setProperty("JPOS_ENCRYPTION_KEY_DB", "your-base64-db-key");

// Encrypt using the default key
String encryptedVal = CryptoEnvironmentProvider.encrypt("my-secret");

// Encrypt using a named key
String encryptedUrl = CryptoEnvironmentProvider.encrypt(
    "jdbc:mysql:acme:acme/jpos_acme?useSSL=false",
    "db"
);
// Returns: enc::db:hXR64gSbru2XeXDbt2n9zaU/thS/k0CVsek3eqqAK24AfjsOagMC9Q+ATpvLr1UmOVSgSUaL7z3j2vZ5NVnD1nZka60tmcM=

Configuration Files

cfg/db.properties (for jPOS-EE DB module):

# Key must be available in environment: JPOS_ENCRYPTION_KEY_DB
hibernate.connection.driver_class=com.mysql.cj.jdbc.Driver
hibernate.connection.url=enc::db:hXR64gSbru2XeXDbt2n9zaU/thS/k0CVsek3eqqAK24AfjsOagMC9Q+ATpvLr1UmOVSgSUaL7z3j2vZ5NVnD1nZka60tmcM=
hibernate.connection.username=enc::db:broVkGTAa91nWqamlUYeUQmRjisvsgO/pk29g3oywSMY
hibernate.connection.password=enc::db:2w20mm7Nkx/d03oLGEgbmXkfPKZ9P9vCzp//1bgAsCD7qTEcj/F2ub5nwtLqHO10
hibernate.dialect=org.hibernate.dialect.MySQL8Dialect
hibernate.hbm2ddl.auto=update

cfg/default.yml (for standard jPOS Environment):

# Key must be available in environment: JPOS_ENCRYPTION_KEY_DB
hibernate:
  connection:
    driver_class: com.mysql.cj.jdbc.Driver
    url: enc::db:hXR64gSbru2XeXDbt2n9zaU/thS/k0CVsek3eqqAK24AfjsOagMC9Q+ATpvLr1UmOVSgSUaL7z3j2vZ5NVnD1nZka60tmcM=
    username: enc::db:broVkGTAa91nWqamlUYeUQmRjisvsgO/pk29g3oywSMY
    password: enc::db:2w20mm7Nkx/d03oLGEgbmXkfPKZ9P9vCzp//1bgAsCD7qTEcj/F2ub5nwtLqHO10
  dialect: org.hibernate.dialect.MySQL8Dialect
  hbm2ddl.auto: update

How It Works

When jPOS loads configuration files, the Environment class automatically processes enc:: values:

  1. Load Configuration: Reads cfg/db.properties or cfg/default.yml
  2. Detect Prefix: Identifies values starting with enc:: via ServiceLoader
  3. Strip Prefix: Environment removes the enc:: prefix before calling get() (matching the ObfEnvironmentProvider pattern)
  4. Fetch Key: Looks up the key dynamically via System.getenv() (or System.getProperty())
  5. Decrypt: Calls CryptoEnvironmentProvider.get() with the stripped value to decrypt using SystemKeyManager
  6. Pass Value: Provides decrypted plaintext to the application component

This happens automatically for:

  • jPOS-EE DB module
  • QBean XML configurations
  • Any component using Environment.get()

Loading External Keys and Key Rotation

The system strictly enforces loading encryption keys from the environment. Keys are never cached internally. This stateless design provides two major benefits:

  1. Seamless Integration: You can generate keys externally (via HSMs, enterprise key managers, or simple scripts) and inject them without relying on jPOS's internal generation.
  2. Real-time Key Rotation: If you update the environment variable in your container or server, jPOS will immediately use the new key on the next decryption/encryption call without requiring a JVM restart.

Method: Set Key via Environment Variable

  1. Generate Key Externally: Create a 256-bit (32-byte) AES key using your preferred method (or use jpos crypto generate)
  2. Base64 Encode: Convert the binary key to Base64 format
  3. Set Environment Variable: Set the appropriate environment variable with the Base64-encoded key

For example:

# For default key
export JPOS_ENCRYPTION_KEY="5G+HWiKUhIn3aSeyf6YRJUn9vhPUA4RV6Y4+S9gkc90="

# For named key "db"
export JPOS_ENCRYPTION_KEY_DB="5G+HWiKUhIn3aSeyf6YRJUn9vhPUA4RV6Y4+S9gkc90="

Key Requirements

  • Must be exactly 32 bytes (256 bits) for AES-256
  • Must be properly Base64-encoded
  • Must be set in the appropriate environment variable:
    • Default key: JPOS_ENCRYPTION_KEY
    • Named key: JPOS_ENCRYPTION_KEY_<KEYNAME> (uppercase)

Best Practices

  1. Always use named keys for production: Avoid default key, use specific names like db, api, cache
  2. Generate keys securely: Use crypto generate <keyname> or an enterprise secret manager.
  3. Inject via Environment: Inject keys dynamically via CI/CD pipelines, Docker secrets, or Kubernetes secrets.
  4. Never commit plaintext secrets or keys to version control
  5. Rotate keys periodically: Since keys are not cached, you can rotate the environment variable and update the encrypted configuration values simultaneously.
  6. Document key names: Keep track of which keys are used for which purposes

Comparison with Obfuscation

Feature obf:: (ObfEnvironmentProvider) enc:: (CryptoEnvironmentProvider)
Security XOR obfuscation AES-256-GCM encryption
Authentication No Yes (GCM tag)
Key SystemSeed (simple XOR) 32-byte AES key (Strict Environment Loading)
IV Fixed per value Random per encryption
Tamper Detection No Yes
PCI Compliance No Yes (with proper key management)

Key Generation

Keys must be explicitly generated and exported to the environment.

Using the CRYPTO CLI Command in q2

The CRYPTO CLI command can be used in two ways:

Prerequisites

  • Ensure you have a recent JDK installed (Java 8 or later recommended)
  • The q2 JAR requires only standard Java APIs (javax.crypto, java.util.Base64)

Option 1: Using jPOS Distribution (Recommended)

$ cd jpos
$ ./gradlew dist

# Extract and use the distribution:
$ tar -xzf jpos/build/distributions/jpos-*.tar.gz
$ cd jpos-*
$ ./bin/q2 -i

Option 2: Build and Run from Source

# Build q2 jar
$ cd jpos
$ ./gradlew q2jar

# Start q2 with interactive CLI
$ java -jar build/libs/jpos-*-q2.jar -i

Option 3: Single Command

The -c flag runs a single CLI command and prints the result. Q2 stays running after the command — press Ctrl+C to exit, or use the shutdown command from another terminal.

# Run a single command (Q2 will keep running after output)
$ java -jar build/libs/jpos-*-q2.jar -c "crypto generate db"

# To stop Q2 after the command, open another terminal and run:
$ cd jpos && ./bin/q2 -c "shutdown"

Environment Variable Configuration

After generating a key, you must make it available to the environment:

For default key:

JPOS_ENCRYPTION_KEY=$env{JPOS_ENCRYPTION_KEY}

For named keys (e.g., "db"):

JPOS_ENCRYPTION_KEY_DB=$env{JPOS_ENCRYPTION_KEY_DB}

In cfg/default.cfg:

# Load all keys from environment variables
JPOS_ENCRYPTION_KEY=$env{JPOS_ENCRYPTION_KEY}
JPOS_ENCRYPTION_KEY_DB=$env{JPOS_ENCRYPTION_KEY_DB}

In cfg/default.yml:

# Load all keys from environment variables
JPOS_ENCRYPTION_KEY: $env{JPOS_ENCRYPTION_KEY}
JPOS_ENCRYPTION_KEY_DB: $env{JPOS_ENCRYPTION_KEY_DB}

Troubleshooting

"No key found in environment" Error

This occurs when:

  • The environment variable is not set
  • The key name in the encrypted value doesn't match the environment variable

Solution: Verify the environment variable is set correctly in the current session:

echo $JPOS_ENCRYPTION_KEY
# or
echo $JPOS_ENCRYPTION_KEY_DB

If passing via JVM property, ensure it is set correctly: -DJPOS_ENCRYPTION_KEY=...

"Decryption failed" Error

This occurs when:

  • The encrypted data has been tampered with
  • The wrong key is being used (or the key was incorrectly copied)
  • The base64 key has invalid padding or formatting

Solution: Verify the key matches the encrypted value and the environment variable is set correctly.

Testing

The system includes comprehensive tests:

  • CryptoEnvironmentProviderTest.java - Tests encryption/decryption functionality
  • SystemKeyManagerTest.java - Tests key management functionality
  • DeployWithEncryptedPropsTest.java - Tests deployment with encrypted properties
  • CRYPTOTest.java - Tests CLI command functionality

Additional Notes

  • Encryption is transparent - applications don't need to know if a value is encrypted
  • All encryption/decryption operations are handled by the framework
  • The system is designed for production use with enterprise-grade security
  • Keys are never internally cached

For more information, see the source code in:

  • jpos/src/main/java/org/jpos/core/SystemKeyManager.java
  • jpos/src/main/java/org/jpos/core/CryptoEnvironmentProvider.java
  • jpos/src/main/java/org/jpos/q2/cli/CRYPTO.java
  • jpos/src/test/java/org/jpos/core/ - Test files

jPOS Cryptographic Key Management — Process Diagrams

This document contains Mermaid diagrams illustrating the complete lifecycle of encryption keys in jPOS.


1. jPOS-Generated Key Lifecycle

1a. Key Generation (via q2 CLI crypto generate)

sequenceDiagram
  actor User
  participant q2 as q2 CLI
  participant SKM as SystemKeyManager
  participant KG as KeyGenerator
  participant SR as SecureRandom
  participant Out as Stdout

  User->>q2: crypto generate db
  q2->>SKM: generateKey(db)
  SKM->>KG: KeyGenerator.init(256, SecureRandom())
  KG-->>SKM: SecretKey AES-256
  SKM->>SKM: Base64.encode(key)
  SKM-->>Out: Base64 key string
  Out-->>User: === Key Generated === Key Name: db Env Var: JPOS_ENCRYPTION_KEY_DB Key: [base64]
  Note over SKM: Key returned as Base64 only. Never cached or stored.
Loading

1b. Exporting Key to Environment

flowchart LR
  A[Generated Base64 Key] --> B[export JPOS_ENCRYPTION_KEY_DB key]
  B --> C[JPOS_ENCRYPTION_KEY_DB in environment]
  C --> D[System.getenv or System.getProperty]
  D --> E[new SecretKeySpec fresh instance every call]
  E --> F[SystemKeyManager getKey db no cache]
Loading

1c. Encrypting a Value

sequenceDiagram
  actor User
  participant q2 as q2 CLI
  participant SKM as SystemKeyManager
  participant SR as SecureRandom
  participant Cipher as AES GCM Cipher

  User->>q2: crypto my-password db
  q2->>SKM: getKey(db)
  SKM->>SKM: System.getenv JPOS_ENCRYPTION_KEY_DB
  SKM->>SKM: Base64.decode envValue
  SKM->>SKM: new SecretKeySpec bytes AES
  SKM-->>q2: SecretKey
  Note over SKM,Cipher: Fresh key no cache
  q2->>SR: nextBytes iv 12 bytes
  SR-->>q2: random IV
  q2->>Cipher: init ENCRYPT_MODE key GCMParameterSpec
  q2->>Cipher: doFinal my-password
  Cipher-->>q2: ciphertext plus GCM tag
  q2->>q2: Base64.encode iv plus ciphertext
  q2-->>User: enc db base64-output
Loading

1d. Complete Encryption to Config to Decryption Flow

flowchart TD
  subgraph G[Key Generation one-time]
    G1[q2 crypto generate db]
    G2[Base64 key printed to stdout]
    G3[export JPOS_ENCRYPTION_KEY_DB]
  end

  subgraph E[Encrypt at build or CI time]
    E1[q2 crypto db-password db]
    E2[enc db kCoa31d4 ciphertext]
    E3[db.properties file]
  end

  subgraph R[Runtime Decryption]
    R1[jPOS loads db.properties]
    R2[Environment.get detects enc prefix]
    R3[CryptoEnvironmentProvider.get]
    R4[SystemKeyManager.getKey db]
    R5[System.getenv JPOS_ENCRYPTION_KEY_DB]
    R6[AES-256-GCM Decrypt]
    R7[Plaintext to Hibernate]
  end

  G1 --> G2 --> G3
  G3 -.-> R5
  E1 --> E2 --> E3
  E3 --> R1
  R1 --> R2 --> R3 --> R4 --> R5 --> R6 --> R7
Loading

2. Externally-Generated Key Lifecycle

2a. Key Provisioning (External Sources)

flowchart TD
  subgraph Sources[External Key Sources choose one]
    HSM[HSM Hardware Security Module]
    Vault[HashiCorp Vault]
    Cloud[AWS Secrets Mgr or Azure Key Vault or GCP]
    Script[Bash openssl rand base64 32]
    KMS[Enterprise KMS]
  end

  subgraph Encode[Base64 Encoding]
    Raw[32-byte raw AES key]
    B64[Base64-encoded string]
  end

  subgraph Export[Environment Export]
    ENV[JPOS_ENCRYPTION_KEY_DB set]
  end

  HSM --> Raw
  Vault --> Raw
  Cloud --> Raw
  Script --> Raw
  KMS --> Raw
  Raw --> B64 --> ENV
Loading

2b. End-to-End External Key Flow

sequenceDiagram
  actor Ops as DevOps Security
  participant KMS as External Key Store HSM Vault Cloud
  participant ENV as Environment Variable
  participant App as jPOS Application
  participant SKM as SystemKeyManager
  participant CEP as CryptoEnvironmentProvider

  Note over Ops,KMS: Step 1 Key Provisioning

  Ops->>KMS: Generate 256-bit AES key
  KMS-->>Ops: 32-byte key
  Ops->>Ops: Base64.encode key
  Ops->>ENV: export JPOS_ENCRYPTION_KEY_DB base64
  Note over ENV: Key now available to JVM

  Note over App,CEP: Step 2 Application Runtime

  App->>CEP: Environment.get enc db ciphertext
  CEP->>SKM: getKey db
  SKM->>ENV: System.getenv JPOS_ENCRYPTION_KEY_DB
  ENV-->>SKM: base64
  SKM->>SKM: new SecretKeySpec bytes AES
  SKM-->>CEP: SecretKey fresh no cache
  CEP-->>App: plaintext password

  Note over Ops,CEP: Step 3 Key Rotation no restart

  Ops->>KMS: Rotate key to version 2
  KMS-->>Ops: new 32-byte key
  Ops->>ENV: export JPOS_ENCRYPTION_KEY_DB new-base64
  Note over ENV: New key picked up immediately

  App->>CEP: Environment.get enc db new-ciphertext
  CEP->>SKM: getKey db
  SKM->>ENV: System.getenv JPOS_ENCRYPTION_KEY_DB
  ENV-->>SKM: new-base64 new version
  SKM->>SKM: new SecretKeySpec newBytes AES
  SKM-->>CEP: new SecretKey
  CEP-->>App: new plaintext
Loading

2c. jPOS-Generated vs External Key Side by Side

flowchart LR
  subgraph JPOS[jPOS-Generated Key]
    G[q2 crypto generate db] --> O[Base64 key printed]
    O --> E[export JPOS_ENCRYPTION_KEY_DB key]
  end

  subgraph External[External Key]
    X[HSM or Vault or openssl] --> B[Base64-encode key]
    B --> E2[export JPOS_ENCRYPTION_KEY_DB key]
  end

  subgraph Use[Both Paths]
    U[Use in config enc db ciphertext]
  end

  E --> U
  E2 --> U
Loading

3. Runtime Decryption — Detailed Flow

sequenceDiagram
  participant CFG as Config File db.properties yml
  participant ENV as Environment.get
  participant CEP as CryptoEnvironmentProvider
  participant SKM as SystemKeyManager
  participant Cipher as AES GCM Cipher
  participant App as Application

  CFG->>ENV: enc db kCoa31d4 ciphertext
  ENV->>CEP: get db ciphertext
  Note over CEP: Environment strips "enc::" prefix before calling get()

  CEP->>CEP: Parse db:base64 (split on first ':')
  CEP->>CEP: Base64.decode base64
  CEP->>CEP: Split IV bytes 0-11 and ciphertext

  CEP->>SKM: getKey db
  SKM->>ENV: System.getenv JPOS_ENCRYPTION_KEY_DB
  ENV-->>SKM: base64
  SKM->>SKM: Base64.decode envValue
  SKM->>SKM: new SecretKeySpec bytes AES
  SKM-->>CEP: SecretKey

  Note over SKM: No caching. Fresh key every call.

  CEP->>Cipher: init DECRYPT_MODE key GCMParameterSpec
  CEP->>Cipher: doFinal ciphertext
  Note over Cipher: GCM tag verified. Tampering throws AEAD exception.
  Cipher-->>CEP: plaintext bytes
  CEP-->>App: plaintext string
Loading

4. Key Rotation

Because keys are loaded from the environment on every access, rotation is straightforward: update the env var, then re-encrypt all config values. No JVM restart required if you are updating env vars dynamically.

flowchart TD
  subgraph Old[Before Rotation]
    O1[JPOS_ENCRYPTION_KEY_DB old-base64]
    O2[enc db old-ciphertext]
    O3[App decrypts with old key]
  end

  subgraph Step[Rotation Steps]
    R1[Update env var to new-base64]
    R2[Generate new key jPOS or external]
    R3[Re-encrypt all config values]
    R4[Update config files with new ciphertexts]
  end

  subgraph New[After Rotation]
    N1[JPOS_ENCRYPTION_KEY_DB new-base64]
    N2[enc db new-ciphertext]
    N3[App decrypts with new key]
  end

  classDef warn fill:#fef3c7,stroke:#b45309,stroke-width:2px
  classDef ok fill:#d1fae5,stroke:#059669,stroke-width:2px

  Old --> R1
  R1 --> R2
  R2 --> R3
  R3 --> R4
  R4 --> New
  class R1 warn
  class R3 ok
Loading

Breaking change: Rotating a key invalidates all previously-encrypted values. You must simultaneously: (1) update the env var and (2) re-encrypt all config values. Old ciphertexts cannot be recovered with a new key.


5. Component Architecture

classDiagram
  direction TB

  class SystemKeyManager {
    <<singleton>>
    KEY_SIZE_BITS = 256
    +getInstance() SystemKeyManager
    +getKey(keyName) SecretKey
    +generateKey(keyName) String
    +encrypt(data, keyName) byte[]
    +decrypt(data, keyName) byte[]
    +getEnvVarName(keyName) String
  }

  class CryptoEnvironmentProvider {
    <<implements EnvironmentProvider>>
    +prefix() String
    +get(config) String
    +encrypt(value, keyName) String
  }

  class CRYPTO {
    <<CLI command>>
    +exec(cli, args) void
    +encrypt(value, keyName) String
  }

  class Environment {
    +get(key) String
  }

  CryptoEnvironmentProvider ..> SystemKeyManager : calls getKey
  CRYPTO ..> CryptoEnvironmentProvider : calls encrypt
  Environment ..> CryptoEnvironmentProvider : delegates (strips prefix first)
Loading

6. Test Injection Pattern

System.getenv() cannot be modified at runtime in Java. Tests inject keys via System.setProperty() which getKey() falls back to when the env var is absent.

sequenceDiagram
  participant Test as JUnit Test
  participant SKM as SystemKeyManager
  participant Prop as System.getProperty

  Note over Test,SKM: @BeforeEach setup

  Test->>SKM: generateKey default
  SKM-->>Test: Base64 key string
  Test->>Test: System.setProperty JPOS_ENCRYPTION_KEY base64Key
  Note over Test: Property is set in JVM

  Note over Test,SKM: Test execution

  Test->>SKM: encrypt data
  SKM->>SKM: System.getenv returns null
  SKM->>Prop: System.getProperty JPOS_ENCRYPTION_KEY
  Prop-->>SKM: base64Key
  SKM->>SKM: new SecretKeySpec bytes AES
  SKM-->>Test: encrypted bytes
  Test->>Test: assertEquals original decrypted

  Note over Test,SKM: @AfterEach cleanup

  Test->>Test: System.clearProperty JPOS_ENCRYPTION_KEY
  Test->>Test: System.clearProperty JPOS_ENCRYPTION_KEY_DB
  Test->>Test: System.clearProperty JPOS_ENCRYPTION_KEY_API
  Note over Test: All test properties cleared
Loading

@chhil chhil force-pushed the crypto-environment-provider branch from 89ebeeb to c6383be Compare February 22, 2026 01:40
- CryptoEnvironmentProvider: AES-256-GCM authenticated encryption with crypto:: prefix
- CRYPTO CLI: Command-line tool to generate encrypted values
- EnvironmentProvider: ServiceLoader registration for automatic decryption
- Tests: 20 unit tests for encryption/decryption and tamper detection

Security features:
- AES-256-GCM authenticated encryption (prevents tampering)
- Random 12-byte IV per encryption (different output each time)
- 16-byte GCM authentication tag
- Tamper detection via GCM tag (throws AEADBadTagException on modification)
- Key derived from SystemSeed (consistent across deployments)

This allows encrypting sensitive database credentials, API keys, and other
secrets in configuration files using the crypto:: prefix format.
@chhil chhil force-pushed the crypto-environment-provider branch 2 times, most recently from 1813a92 to 7ad0162 Compare March 19, 2026 10:51
@chhil
Copy link
Contributor Author

chhil commented Mar 19, 2026

This was completely implemented using an LLM (local : qwen coder next and a couple free online ones)

Use StandardCharsets.UTF_8 instead of platform-default charset in
CryptoEnvironmentProvider and CRYPTO to ensure consistent encryption
output across all JVM locales and platforms.
@chhil chhil force-pushed the crypto-environment-provider branch from 7ad0162 to f7b46c2 Compare March 19, 2026 12:06
chhil added 9 commits March 19, 2026 17:50
…on keys

Key changes:
- Add SystemKeyManager: singleton managing AES-256 keys loaded from env vars
  with no internal caching for real-time key rotation
- generateKey() uses new SecureRandom() (OS entropy) for true randomness
- getKey() reads from System.getenv(), falls back to System.getProperty()
- CryptoEnvironmentProvider and CRYPTO CLI delegate to SystemKeyManager
- Named key support: enc::keyname:: prefix for multiple encryption contexts
- Use explicit StandardCharsets.UTF_8 for string/bytes conversion

Full test coverage: 13 SystemKeyManagerTest, 9 CryptoEnvironmentProviderTest,
13 CRYPTOTest, 1 DeployWithEncryptedPropsTest
…ncrypt()

Remove duplicate encryption code from CRYPTO. The CLI now delegates to
CryptoEnvironmentProvider.encrypt() which is the authoritative implementation.
This eliminates code duplication and ensures a single source of truth for
the AES-256-GCM encryption logic.

Also update CRYPTOTest to use CryptoEnvironmentProvider.encrypt()
instead of the removed CRYPTO.encrypt() methods.
clearKeys() was a no-op since keys are not cached internally.
It was misleading and not used in production code. Removing it.
…tedPropsTest

Test was creating files at relative paths (cfg/, deploy/) in working directory.
Use JUnit 5 @tempdir to create files in a temporary directory that is
automatically cleaned up after the test. Also add proper @AfterEach cleanup
for the system property.
…::db:

Format enc::db:: looked like another environment provider prefix.
Use enc::db: to clearly indicate db is a key name, not a provider.
…ovider

Use indexOf(':') to find key name separator instead of split().
Handles both default key (enc::base64) and named key (enc::db:base64)
formats correctly. Defensively handles prefix stripping since Environment
already strips it before calling get(), but get() remains robust for direct calls.
- Define ENC_PREFIX constant to avoid duplication
- Remove unused KEY_SIZE_BITS field
- Reuse static SecureRandom instance instead of creating new ones
- Use unnamed pattern for unused exception variable
Generate and set encryption key in @beforeeach, clean up in @AfterEach
@ar-agt
Copy link
Collaborator

ar-agt commented Mar 20, 2026

Overall: solid, well-documented feature. AES-256-GCM is the right choice, the no-caching design is intentional and defensible, and the test coverage is good. A few issues worth addressing before merging.


1. get() re-parses the enc:: prefix it should not receive

Per the EnvironmentProvider contract, Environment strips the prefix before calling get(). This block is dead code and should be removed:

if (config.startsWith(ENC_PREFIX)) {
    remainder = config.substring(5);
}

2. get() uses a colon as key name delimiter — fragile

int colonIdx = remainder.indexOf(':');

Base64 does not contain colons so this is accidentally safe, but it is fragile. A safer approach: check whether the portion before the first : matches a valid key name pattern (e.g. [a-zA-Z0-9_]+) before treating it as a key name.

3. SystemKeyManager duplicates AES/GCM logic from CryptoEnvironmentProvider

Both classes implement encrypt/decrypt with AES/GCM/NoPadding. SystemKeyManager.encrypt/decrypt are only used in tests; the runtime path goes through CryptoEnvironmentProvider. This is a maintenance liability — IV size, tag length, or algorithm could drift between the two. SystemKeyManager should own key management only; crypto operations should live solely in CryptoEnvironmentProvider.

4. SecureRandom as a static field

private static final SecureRandom SECURE_RANDOM = new SecureRandom();

SecureRandom is not guaranteed thread-safe across all JVM implementations. Under concurrent load, prefer new SecureRandom() per encryption or a ThreadLocal<SecureRandom>.

5. Silent swallowing of invalid Base64 in getKey()

} catch (IllegalArgumentException _) {
    // Invalid Base64 or length
}

If the env var contains a malformed key, getKey() returns null and the caller throws a generic "Key not found" error. The actual cause (bad Base64, wrong length) is silently discarded. At minimum, log a warning.

6. Dead code in handleEncrypt()

String value = command.startsWith("crypto::") ? command.substring(8) : args[1];

command is args[1], so the crypto:: branch is unreachable in normal use. Remove it.

7. getEnvVarName() does not sanitize key names

"db"JPOS_ENCRYPTION_KEY_DB works. But "my-key"JPOS_ENCRYPTION_KEY_MY-KEY is not a valid env var name. Key names with hyphens, spaces, or other non-alphanumeric characters should be rejected early or normalized (uppercase, hyphens to underscores).

8. CRYPTO.exec() arity check too loose

if (args.length < 2 || args.length > 4) {

args[0] is the command name, so this permits an undocumented 4th argument. Should be > 3.

9. Documentation should be committed, not left in the PR description

The PR body is a comprehensive guide with Mermaid diagrams and usage examples. It will not survive the merge. This content should be committed to the jPOS docs repo or at minimum as Javadoc on the key classes — otherwise it is lost.


Minor:

  • Trailing blank line before the closing } in SystemKeyManager.java
  • getDefaultKey() and getKeyBase64() are public with no callers — consider package-private or removal

Priority: Fix 1, 3, 6, 7 before merging. Items 2, 4, 5 are lower priority but worth addressing.

chhil added 8 commits March 21, 2026 05:54
Per the EnvironmentProvider contract, the Environment class strips the
provider's prefix before invoking the get(String) method. The
CryptoEnvironmentProvider was erroneously checking for and manually
stripping
the `enc::` prefix, which was redundant dead code.
This commit:
- Removes the dead prefix-stripping code from
`CryptoEnvironmentProvider.get`
- Updates `CryptoEnvironmentProviderTest` to correctly simulate
`Environment`
  behavior by stripping the prefix before calling `provider.get`
- Refactors `DeployWithEncryptedPropsTest` to actually test
`Environment`
  resolving the property configuration, rather than bypassing it
- Rewrites `SimpleConfigurationWithCryptoTest` to correctly verify that
  `SimpleConfiguration` and `Environment` automatically decrypt
properties
- Adds `EnvironmentResolveTest` to ensure `Environment.get("${...}")`
  correctly handles encrypted system properties
`SystemKeyManager.getEnvVarName` previously appended user-provided
key names directly after uppercasing them. Keys containing hyphens or
spaces
(e.g., "my-key") produced invalid OS environment variable names
(e.g., "JPOS_ENCRYPTION_KEY_MY-KEY"). This has been updated to normalize
all
non-alphanumeric characters to underscores.
CRYPTO CLI command contained an unreachable branch that checked
if `args[1]` started with "crypto::" before stripping it. Since
`args[1]` holds
the raw user secret, this condition was dead code and has been removed.
The CRYPTO CLI command `exec()` method was previously checking if the
argument length was strictly greater than 4 (`args.length > 4`). Since
`args[0]` represents the command name itself ("crypto"), `args[1]`
represents the secret (or "generate"), and `args[2]` represents the
optional key name, the maximum valid argument count is 3.
Permitting `args.length > 4` silently allowed an undocumented fourth
argument to be passed without triggering the usage instructions.
This commit updates the arity check to restrict `args.length > 3` and
adds a unit test in `CRYPTOTest` to explicitly verify that passing a
fourth argument properly triggers the CLI `usage()` method.
The `SecureRandom` instance was previously initialized as a static
field (`SECURE_RANDOM = new SecureRandom()`) in both `SystemKeyManager`
and `CryptoEnvironmentProvider`.
While `SecureRandom.nextBytes()` is thread-safe on modern JVMs, heavy
concurrent usage of a single static instance can create severe thread
contention and locking bottlenecks, particularly when continuously
generating Initialization Vectors (IVs) for encryption under load.
This commit refactors these classes to instantiate a new `SecureRandom`
instance inline (`new SecureRandom().nextBytes(iv)`) per encryption
operation. This completely eliminates synchronization bottlenecks and
guarantees high-throughput thread safety without compromising
cryptographic entropy.
SystemKeyManager previously duplicated AES/GCM/NoPadding encryption and
decryption logic identical to the primary implementation found in
CryptoEnvironmentProvider.
These proxy methods were exclusively utilized by the unit test suite and
were never invoked in the runtime application pathway. Maintaining
duplicate cryptography logic across two separate domains violates the
Single Responsibility Principle and introduces significant maintenance
liabilities (e.g., initialization vector sizes, algorithm choices, or
tag
lengths could subtly drift between the two classes over time).
This commit entirely removes the `encrypt` and `decrypt` methods from
`SystemKeyManager` (alongside their proxy unit tests), restricting the
class strictly to OS environment key generation and loading. All
cryptography operations are now centralized exclusively within
`CryptoEnvironmentProvider`.
…errors

Previously, CryptoEnvironmentProvider.get() assumed that any string
preceding the first colon (`:`) was a key name. This delimiter parsing
was extremely fragile because malformed or standard Base64 payloads
could inadvertently contain colons and trigger invalid sub-stringing.
This commit hardens the extraction by enforcing a strict regular
expression pattern (`^[a-zA-Z0-9_\\-]+$`). If the string before the
colon does not strictly consist of alphanumerics, hyphens, or
underscores, it is correctly treated as part of the Base64 payload
rather than an explicit key name.
Additionally, SystemKeyManager.getKey() silently swallowed invalid
Base64
decoding strings and invalid key byte lengths via an empty catch
block (`catch (IllegalArgumentException _) { }`), forcing callers to
throw a generic "Key not found" error without context. This has been
updated to use explicit `java.util.logging.Logger` warnings, providing
valuable context regarding payload size mismatches or decoding failures.
SystemKeyManager contained two public methods (`getDefaultKey()` and
`getKeyBase64()`) that had no callers outside the test suite. Since
they are only invoked by tests within the same package
(`org.jpos.core`), their visibility has been reduced to package-private,
eliminating unnecessary public API surface area.
Additionally, removed a trailing blank line before the class closing
brace.
No functional changes — all tests continue to pass.
@ar-agt
Copy link
Collaborator

ar-agt commented Mar 23, 2026

Good progress — most of the previous feedback has been addressed. Two remaining items worth resolving before merge:

1. Use jPOS's own logger instead of java.util.logging

SystemKeyManager currently uses java.util.logging.Logger to report invalid Base64 and wrong key lengths. jPOS has its own logging infrastructure (org.jpos.util.Log, org.jpos.util.LogEvent, org.jpos.util.Logger) which is what the rest of the framework uses — and what MGL's Log Viewer (and any structured log consumer) indexes and searches.

Using java.util.logging here means these warnings are invisible to the jPOS operator tooling. They go to stderr or a JUL handler, not to the Q2 log stream.

Since SystemKeyManager is not a QBean and can't extend Log directly, the cleanest approach is to create a static logger:

private static final org.jpos.util.Logger log = 
    org.jpos.util.Logger.getLogger("crypto");

Or accept the logger as a constructor/singleton parameter. Either way, the warnings should flow through jPOS's event system so they're visible, searchable, and correlated with the surrounding context.

We wrote about this recently — the structured audit log and why consistent logging matters operationally: https://jpos.org/blog/2026/03/logview-demo

2. Hyphen-to-underscore normalization in getEnvVarName() is invisible to callers

The key name validation in get() accepts hyphens (^[a-zA-Z0-9_\\-]+$), but getEnvVarName() silently converts them to underscores when constructing the env var name. So getKey("my-key") looks up JPOS_ENCRYPTION_KEY_MY_KEY, not JPOS_ENCRYPTION_KEY_MY-KEY (which isn't a valid env var name anyway).

This is the correct behavior, but it's not obvious. A user who encrypts a value with key name "my-key" and sets JPOS_ENCRYPTION_KEY_MY-KEY in their shell will get a silent decryption failure. A comment in getEnvVarName() noting the normalization — or logging the resolved env var name on first use — would make this much easier to diagnose.

@ar-agt
Copy link
Collaborator

ar-agt commented Mar 23, 2026

To clarify my suggestion on item 1 — the right pattern for a non-QBean class in jPOS core is Log.getLog() against the Q2 logger:

private static final Log log = Log.getLog(Q2.LOGGER_NAME, "crypto");

Then use it as:

log.warn("Invalid key length in " + envVarName + 
    ": expected 32 bytes, got " + keyBytes.length);
log.warn("Invalid Base64 in " + envVarName + ": " + e.getMessage());

This wires SystemKeyManager into the same Q2 log stream that everything else uses — visible in the Log Viewer, correlated with surrounding events, and subject to whatever log listeners are configured. SelectDestination in the same codebase uses exactly this pattern for a utility class that isn't a QBean.

…ent classes

Replaced `java.util.logging.Logger` in `SystemKeyManager` with jPOS's
native logging
infrastructure (`org.jpos.util.Log`) so that invalid Base64 and bad key
length warnings
are correctly routed to the Q2 log stream and correctly tagged.
Also addressed potential silent decryption failures by:
- Updating `CryptoEnvironmentProvider` to actively log a Q2 warning
resolving the exact
  environment variable name it tried to use when a key isn't found,
preventing developers
  from flying blind during setup.
- Updating `SystemKeyManager.getEnvVarName()` Javadoc to explicitly
document that
  hyphens (and other non-alphanumeric characters) are normalized to
underscores in
  environment variable resolution.
- Assigned the log events to the specific `crypto-env-provider` realm to
ensure these
  events are easily searchable and structured within log viewer tooling.
- Fixed `SimpleConfigurationWithCryptoTest` to correctly simulate
`Environment` placeholder
  resolutions instead of pushing `enc::` values directly into Properties
un-templated.
- Added comprehensive unit tests in `SystemKeyManagerTest` to verify
that `LogEvent`
  payloads are emitted via Q2 logger when invalid configurations occur.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants