Add CryptoEnvironmentProvider with AES-256-GCM encryption#675
Add CryptoEnvironmentProvider with AES-256-GCM encryption#675
Conversation
89ebeeb to
c6383be
Compare
- CryptoEnvironmentProvider: AES-256-GCM authenticated encryption with crypto:: prefix - CRYPTO CLI: Command-line tool to generate encrypted values - EnvironmentProvider: ServiceLoader registration for automatic decryption - Tests: 20 unit tests for encryption/decryption and tamper detection Security features: - AES-256-GCM authenticated encryption (prevents tampering) - Random 12-byte IV per encryption (different output each time) - 16-byte GCM authentication tag - Tamper detection via GCM tag (throws AEADBadTagException on modification) - Key derived from SystemSeed (consistent across deployments) This allows encrypting sensitive database credentials, API keys, and other secrets in configuration files using the crypto:: prefix format.
1813a92 to
7ad0162
Compare
|
This was completely implemented using an LLM (local : qwen coder next and a couple free online ones) |
Use StandardCharsets.UTF_8 instead of platform-default charset in CryptoEnvironmentProvider and CRYPTO to ensure consistent encryption output across all JVM locales and platforms.
7ad0162 to
f7b46c2
Compare
…on keys Key changes: - Add SystemKeyManager: singleton managing AES-256 keys loaded from env vars with no internal caching for real-time key rotation - generateKey() uses new SecureRandom() (OS entropy) for true randomness - getKey() reads from System.getenv(), falls back to System.getProperty() - CryptoEnvironmentProvider and CRYPTO CLI delegate to SystemKeyManager - Named key support: enc::keyname:: prefix for multiple encryption contexts - Use explicit StandardCharsets.UTF_8 for string/bytes conversion Full test coverage: 13 SystemKeyManagerTest, 9 CryptoEnvironmentProviderTest, 13 CRYPTOTest, 1 DeployWithEncryptedPropsTest
…ncrypt() Remove duplicate encryption code from CRYPTO. The CLI now delegates to CryptoEnvironmentProvider.encrypt() which is the authoritative implementation. This eliminates code duplication and ensures a single source of truth for the AES-256-GCM encryption logic. Also update CRYPTOTest to use CryptoEnvironmentProvider.encrypt() instead of the removed CRYPTO.encrypt() methods.
clearKeys() was a no-op since keys are not cached internally. It was misleading and not used in production code. Removing it.
…tedPropsTest Test was creating files at relative paths (cfg/, deploy/) in working directory. Use JUnit 5 @tempdir to create files in a temporary directory that is automatically cleaned up after the test. Also add proper @AfterEach cleanup for the system property.
…::db: Format enc::db:: looked like another environment provider prefix. Use enc::db: to clearly indicate db is a key name, not a provider.
…ovider
Use indexOf(':') to find key name separator instead of split().
Handles both default key (enc::base64) and named key (enc::db:base64)
formats correctly. Defensively handles prefix stripping since Environment
already strips it before calling get(), but get() remains robust for direct calls.
- Define ENC_PREFIX constant to avoid duplication - Remove unused KEY_SIZE_BITS field - Reuse static SecureRandom instance instead of creating new ones - Use unnamed pattern for unused exception variable
Generate and set encryption key in @beforeeach, clean up in @AfterEach
|
Overall: solid, well-documented feature. AES-256-GCM is the right choice, the no-caching design is intentional and defensible, and the test coverage is good. A few issues worth addressing before merging. 1. Per the if (config.startsWith(ENC_PREFIX)) {
remainder = config.substring(5);
}2. int colonIdx = remainder.indexOf(':');Base64 does not contain colons so this is accidentally safe, but it is fragile. A safer approach: check whether the portion before the first 3. Both classes implement 4. private static final SecureRandom SECURE_RANDOM = new SecureRandom();
5. Silent swallowing of invalid Base64 in } catch (IllegalArgumentException _) {
// Invalid Base64 or length
}If the env var contains a malformed key, 6. Dead code in String value = command.startsWith("crypto::") ? command.substring(8) : args[1];
7.
8. if (args.length < 2 || args.length > 4) {
9. Documentation should be committed, not left in the PR description The PR body is a comprehensive guide with Mermaid diagrams and usage examples. It will not survive the merge. This content should be committed to the jPOS docs repo or at minimum as Javadoc on the key classes — otherwise it is lost. Minor:
Priority: Fix 1, 3, 6, 7 before merging. Items 2, 4, 5 are lower priority but worth addressing. |
Per the EnvironmentProvider contract, the Environment class strips the
provider's prefix before invoking the get(String) method. The
CryptoEnvironmentProvider was erroneously checking for and manually
stripping
the `enc::` prefix, which was redundant dead code.
This commit:
- Removes the dead prefix-stripping code from
`CryptoEnvironmentProvider.get`
- Updates `CryptoEnvironmentProviderTest` to correctly simulate
`Environment`
behavior by stripping the prefix before calling `provider.get`
- Refactors `DeployWithEncryptedPropsTest` to actually test
`Environment`
resolving the property configuration, rather than bypassing it
- Rewrites `SimpleConfigurationWithCryptoTest` to correctly verify that
`SimpleConfiguration` and `Environment` automatically decrypt
properties
- Adds `EnvironmentResolveTest` to ensure `Environment.get("${...}")`
correctly handles encrypted system properties
`SystemKeyManager.getEnvVarName` previously appended user-provided key names directly after uppercasing them. Keys containing hyphens or spaces (e.g., "my-key") produced invalid OS environment variable names (e.g., "JPOS_ENCRYPTION_KEY_MY-KEY"). This has been updated to normalize all non-alphanumeric characters to underscores.
CRYPTO CLI command contained an unreachable branch that checked if `args[1]` started with "crypto::" before stripping it. Since `args[1]` holds the raw user secret, this condition was dead code and has been removed.
The CRYPTO CLI command `exec()` method was previously checking if the
argument length was strictly greater than 4 (`args.length > 4`). Since
`args[0]` represents the command name itself ("crypto"), `args[1]`
represents the secret (or "generate"), and `args[2]` represents the
optional key name, the maximum valid argument count is 3.
Permitting `args.length > 4` silently allowed an undocumented fourth
argument to be passed without triggering the usage instructions.
This commit updates the arity check to restrict `args.length > 3` and
adds a unit test in `CRYPTOTest` to explicitly verify that passing a
fourth argument properly triggers the CLI `usage()` method.
The `SecureRandom` instance was previously initialized as a static field (`SECURE_RANDOM = new SecureRandom()`) in both `SystemKeyManager` and `CryptoEnvironmentProvider`. While `SecureRandom.nextBytes()` is thread-safe on modern JVMs, heavy concurrent usage of a single static instance can create severe thread contention and locking bottlenecks, particularly when continuously generating Initialization Vectors (IVs) for encryption under load. This commit refactors these classes to instantiate a new `SecureRandom` instance inline (`new SecureRandom().nextBytes(iv)`) per encryption operation. This completely eliminates synchronization bottlenecks and guarantees high-throughput thread safety without compromising cryptographic entropy.
SystemKeyManager previously duplicated AES/GCM/NoPadding encryption and decryption logic identical to the primary implementation found in CryptoEnvironmentProvider. These proxy methods were exclusively utilized by the unit test suite and were never invoked in the runtime application pathway. Maintaining duplicate cryptography logic across two separate domains violates the Single Responsibility Principle and introduces significant maintenance liabilities (e.g., initialization vector sizes, algorithm choices, or tag lengths could subtly drift between the two classes over time). This commit entirely removes the `encrypt` and `decrypt` methods from `SystemKeyManager` (alongside their proxy unit tests), restricting the class strictly to OS environment key generation and loading. All cryptography operations are now centralized exclusively within `CryptoEnvironmentProvider`.
…errors
Previously, CryptoEnvironmentProvider.get() assumed that any string
preceding the first colon (`:`) was a key name. This delimiter parsing
was extremely fragile because malformed or standard Base64 payloads
could inadvertently contain colons and trigger invalid sub-stringing.
This commit hardens the extraction by enforcing a strict regular
expression pattern (`^[a-zA-Z0-9_\\-]+$`). If the string before the
colon does not strictly consist of alphanumerics, hyphens, or
underscores, it is correctly treated as part of the Base64 payload
rather than an explicit key name.
Additionally, SystemKeyManager.getKey() silently swallowed invalid
Base64
decoding strings and invalid key byte lengths via an empty catch
block (`catch (IllegalArgumentException _) { }`), forcing callers to
throw a generic "Key not found" error without context. This has been
updated to use explicit `java.util.logging.Logger` warnings, providing
valuable context regarding payload size mismatches or decoding failures.
SystemKeyManager contained two public methods (`getDefaultKey()` and `getKeyBase64()`) that had no callers outside the test suite. Since they are only invoked by tests within the same package (`org.jpos.core`), their visibility has been reduced to package-private, eliminating unnecessary public API surface area. Additionally, removed a trailing blank line before the class closing brace. No functional changes — all tests continue to pass.
|
Good progress — most of the previous feedback has been addressed. Two remaining items worth resolving before merge: 1. Use jPOS's own logger instead of
Using Since private static final org.jpos.util.Logger log =
org.jpos.util.Logger.getLogger("crypto");Or accept the logger as a constructor/singleton parameter. Either way, the warnings should flow through jPOS's event system so they're visible, searchable, and correlated with the surrounding context. We wrote about this recently — the structured audit log and why consistent logging matters operationally: https://jpos.org/blog/2026/03/logview-demo 2. Hyphen-to-underscore normalization in The key name validation in This is the correct behavior, but it's not obvious. A user who encrypts a value with key name |
|
To clarify my suggestion on item 1 — the right pattern for a non-QBean class in jPOS core is private static final Log log = Log.getLog(Q2.LOGGER_NAME, "crypto");Then use it as: log.warn("Invalid key length in " + envVarName +
": expected 32 bytes, got " + keyBytes.length);
log.warn("Invalid Base64 in " + envVarName + ": " + e.getMessage());This wires |
…ent classes Replaced `java.util.logging.Logger` in `SystemKeyManager` with jPOS's native logging infrastructure (`org.jpos.util.Log`) so that invalid Base64 and bad key length warnings are correctly routed to the Q2 log stream and correctly tagged. Also addressed potential silent decryption failures by: - Updating `CryptoEnvironmentProvider` to actively log a Q2 warning resolving the exact environment variable name it tried to use when a key isn't found, preventing developers from flying blind during setup. - Updating `SystemKeyManager.getEnvVarName()` Javadoc to explicitly document that hyphens (and other non-alphanumeric characters) are normalized to underscores in environment variable resolution. - Assigned the log events to the specific `crypto-env-provider` realm to ensure these events are easily searchable and structured within log viewer tooling. - Fixed `SimpleConfigurationWithCryptoTest` to correctly simulate `Environment` placeholder resolutions instead of pushing `enc::` values directly into Properties un-templated. - Added comprehensive unit tests in `SystemKeyManagerTest` to verify that `LogEvent` payloads are emitted via Q2 logger when invalid configurations occur.
jPOS Cryptographic Implementation Guide
Overview
jPOS implements AES-256-GCM authenticated encryption for securing sensitive configuration data. This system allows encrypted values to be stored in configuration files while automatically decrypting them at runtime.
Key Components
1. SystemKeyManager
The
SystemKeyManageris a singleton class that manages encryption keys. It supports multiple named keys for different encryption contexts.new SecureRandom()(OS-provided entropy) to generate cryptographically random 256-bit keys2. CryptoEnvironmentProvider
This class provides encryption/decryption services for configuration values. It uses the SystemKeyManager to retrieve keys and implements AES-256-GCM encryption.
enc::[keyname]:[base64-encoded-iv+ciphertext]enc::[base64-encoded-iv+ciphertext]enc::keyname:[base64-encoded-iv+ciphertext]Configuration Setup
Step 1: Generate a Key
Use the CRYPTO CLI command to generate a key. Keys are generated but never cached in memory - you must save the output.
This will output:
For named keys, use:
This will output:
Step 2: Set Environment Variable
Set the environment variable with the Base64 output from the previous step. This is mandatory, as jPOS will read the key from the environment on every encryption/decryption operation.
export JPOS_ENCRYPTION_KEY=[base64-encoded-key]For named keys, use:
export JPOS_ENCRYPTION_KEY_DB=[base64-encoded-key](Note: For development or testing, you can also set these as Java system properties:
-DJPOS_ENCRYPTION_KEY_DB=[base64-encoded-key])Step 3: Encrypt Configuration Values
Ensure the environment variable is set in your current session, then encrypt sensitive values using the CRYPTO command:
jpos crypto "my-database-password" dbThis will output an encrypted value like:
Step 4: Use in Configuration Files
Add the encrypted value to your configuration files:
db.properties
default.properties
default.yml
Encryption Format Details
The encrypted format follows this pattern:
enc::- Prefix indicating encrypted content[keyname]:- Optional key name (for named keys)[base64-encoded-iv+ciphertext]- Base64-encoded initialization vector + ciphertextThe IV (12 bytes) is prepended to the ciphertext for decryption.
Security Features
Usage Examples
Using q2 CLI
Using Java Code
Configuration Files
cfg/db.properties(for jPOS-EE DB module):cfg/default.yml(for standard jPOS Environment):How It Works
When jPOS loads configuration files, the
Environmentclass automatically processesenc::values:cfg/db.propertiesorcfg/default.ymlenc::via ServiceLoaderEnvironmentremoves theenc::prefix before callingget()(matching theObfEnvironmentProviderpattern)System.getenv()(orSystem.getProperty())CryptoEnvironmentProvider.get()with the stripped value to decrypt using SystemKeyManagerThis happens automatically for:
Loading External Keys and Key Rotation
The system strictly enforces loading encryption keys from the environment. Keys are never cached internally. This stateless design provides two major benefits:
Method: Set Key via Environment Variable
jpos crypto generate)For example:
Key Requirements
JPOS_ENCRYPTION_KEYJPOS_ENCRYPTION_KEY_<KEYNAME>(uppercase)Best Practices
db,api,cachecrypto generate <keyname>or an enterprise secret manager.Comparison with Obfuscation
obf::(ObfEnvironmentProvider)enc::(CryptoEnvironmentProvider)Key Generation
Keys must be explicitly generated and exported to the environment.
Using the CRYPTO CLI Command in q2
The CRYPTO CLI command can be used in two ways:
Prerequisites
javax.crypto,java.util.Base64)Option 1: Using jPOS Distribution (Recommended)
Option 2: Build and Run from Source
Option 3: Single Command
The
-cflag runs a single CLI command and prints the result. Q2 stays running after the command — press Ctrl+C to exit, or use theshutdowncommand from another terminal.Environment Variable Configuration
After generating a key, you must make it available to the environment:
For default key:
For named keys (e.g., "db"):
In
cfg/default.cfg:In
cfg/default.yml:Troubleshooting
"No key found in environment" Error
This occurs when:
Solution: Verify the environment variable is set correctly in the current session:
If passing via JVM property, ensure it is set correctly:
-DJPOS_ENCRYPTION_KEY=..."Decryption failed" Error
This occurs when:
Solution: Verify the key matches the encrypted value and the environment variable is set correctly.
Testing
The system includes comprehensive tests:
CryptoEnvironmentProviderTest.java- Tests encryption/decryption functionalitySystemKeyManagerTest.java- Tests key management functionalityDeployWithEncryptedPropsTest.java- Tests deployment with encrypted propertiesCRYPTOTest.java- Tests CLI command functionalityAdditional Notes
For more information, see the source code in:
jpos/src/main/java/org/jpos/core/SystemKeyManager.javajpos/src/main/java/org/jpos/core/CryptoEnvironmentProvider.javajpos/src/main/java/org/jpos/q2/cli/CRYPTO.javajpos/src/test/java/org/jpos/core/- Test filesjPOS Cryptographic Key Management — Process Diagrams
This document contains Mermaid diagrams illustrating the complete lifecycle of encryption keys in jPOS.
1. jPOS-Generated Key Lifecycle
1a. Key Generation (via q2 CLI
crypto generate)1b. Exporting Key to Environment
1c. Encrypting a Value
1d. Complete Encryption to Config to Decryption Flow
flowchart TD subgraph G[Key Generation one-time] G1[q2 crypto generate db] G2[Base64 key printed to stdout] G3[export JPOS_ENCRYPTION_KEY_DB] end subgraph E[Encrypt at build or CI time] E1[q2 crypto db-password db] E2[enc db kCoa31d4 ciphertext] E3[db.properties file] end subgraph R[Runtime Decryption] R1[jPOS loads db.properties] R2[Environment.get detects enc prefix] R3[CryptoEnvironmentProvider.get] R4[SystemKeyManager.getKey db] R5[System.getenv JPOS_ENCRYPTION_KEY_DB] R6[AES-256-GCM Decrypt] R7[Plaintext to Hibernate] end G1 --> G2 --> G3 G3 -.-> R5 E1 --> E2 --> E3 E3 --> R1 R1 --> R2 --> R3 --> R4 --> R5 --> R6 --> R72. Externally-Generated Key Lifecycle
2a. Key Provisioning (External Sources)
flowchart TD subgraph Sources[External Key Sources choose one] HSM[HSM Hardware Security Module] Vault[HashiCorp Vault] Cloud[AWS Secrets Mgr or Azure Key Vault or GCP] Script[Bash openssl rand base64 32] KMS[Enterprise KMS] end subgraph Encode[Base64 Encoding] Raw[32-byte raw AES key] B64[Base64-encoded string] end subgraph Export[Environment Export] ENV[JPOS_ENCRYPTION_KEY_DB set] end HSM --> Raw Vault --> Raw Cloud --> Raw Script --> Raw KMS --> Raw Raw --> B64 --> ENV2b. End-to-End External Key Flow
2c. jPOS-Generated vs External Key Side by Side
flowchart LR subgraph JPOS[jPOS-Generated Key] G[q2 crypto generate db] --> O[Base64 key printed] O --> E[export JPOS_ENCRYPTION_KEY_DB key] end subgraph External[External Key] X[HSM or Vault or openssl] --> B[Base64-encode key] B --> E2[export JPOS_ENCRYPTION_KEY_DB key] end subgraph Use[Both Paths] U[Use in config enc db ciphertext] end E --> U E2 --> U3. Runtime Decryption — Detailed Flow
4. Key Rotation
Because keys are loaded from the environment on every access, rotation is straightforward: update the env var, then re-encrypt all config values. No JVM restart required if you are updating env vars dynamically.
flowchart TD subgraph Old[Before Rotation] O1[JPOS_ENCRYPTION_KEY_DB old-base64] O2[enc db old-ciphertext] O3[App decrypts with old key] end subgraph Step[Rotation Steps] R1[Update env var to new-base64] R2[Generate new key jPOS or external] R3[Re-encrypt all config values] R4[Update config files with new ciphertexts] end subgraph New[After Rotation] N1[JPOS_ENCRYPTION_KEY_DB new-base64] N2[enc db new-ciphertext] N3[App decrypts with new key] end classDef warn fill:#fef3c7,stroke:#b45309,stroke-width:2px classDef ok fill:#d1fae5,stroke:#059669,stroke-width:2px Old --> R1 R1 --> R2 R2 --> R3 R3 --> R4 R4 --> New class R1 warn class R3 okBreaking change: Rotating a key invalidates all previously-encrypted values. You must simultaneously: (1) update the env var and (2) re-encrypt all config values. Old ciphertexts cannot be recovered with a new key.
5. Component Architecture
classDiagram direction TB class SystemKeyManager { <<singleton>> KEY_SIZE_BITS = 256 +getInstance() SystemKeyManager +getKey(keyName) SecretKey +generateKey(keyName) String +encrypt(data, keyName) byte[] +decrypt(data, keyName) byte[] +getEnvVarName(keyName) String } class CryptoEnvironmentProvider { <<implements EnvironmentProvider>> +prefix() String +get(config) String +encrypt(value, keyName) String } class CRYPTO { <<CLI command>> +exec(cli, args) void +encrypt(value, keyName) String } class Environment { +get(key) String } CryptoEnvironmentProvider ..> SystemKeyManager : calls getKey CRYPTO ..> CryptoEnvironmentProvider : calls encrypt Environment ..> CryptoEnvironmentProvider : delegates (strips prefix first)6. Test Injection Pattern
System.getenv()cannot be modified at runtime in Java. Tests inject keys viaSystem.setProperty()whichgetKey()falls back to when the env var is absent.