Skip to content

Commit d4c86c4

Browse files
josecelanojosecelano
committed
feat: [#232] implement MySQL environment variable injection pattern
Phase 2: Extend environment configuration for MySQL support Core Changes: - Add MySQL variant to domain DatabaseConfig enum with host, port, database_name, username, password fields - Add MySQL variant to application layer DatabaseSection enum - Update conversion methods to handle MySQL configuration - Regenerate JSON schema to include MySQL database driver Environment Variable Injection: - Extend EnvContext to include optional MySQL credentials - Add new_with_mysql() constructor for MySQL mode - Update .env.tera template to conditionally include MySQL variables - Change docker-compose.yml.tera to use ${MYSQL_*} references instead of hardcoded values - Update RenderDockerComposeTemplatesStep to create proper contexts based on database driver Documentation: - Add comprehensive comment block in docker-compose.yml.tera explaining environment variable injection pattern - Create ADR docs/decisions/environment-variable-injection-in-docker-compose.md - Update environment templates with MySQL example comments Testing: - Add 6 new unit tests for MySQL configuration (serialization, deserialization, conversion) - Update docker-compose renderer tests to verify environment variable references - All 1466 tests passing Benefits: - System administrators can modify .env values and restart services without regenerating templates - Follows Docker Compose best practices for configuration management - Separates template generation (deploy-time) from configuration (runtime) - Enables secure credential rotation without redeployment
1 parent 9a77348 commit d4c86c4

File tree

11 files changed

+640
-59
lines changed

11 files changed

+640
-59
lines changed

‎docs/decisions/README.md‎

Lines changed: 25 additions & 24 deletions
Original file line numberDiff line numberDiff line change
@@ -4,30 +4,31 @@ This directory contains architectural decision records for the Torrust Tracker D
44

55
## Decision Index
66

7-
| Status | Date | Decision | Summary |
8-
| ------------- | ---------- | ----------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------- |
9-
| ✅ Accepted | 2025-12-11 | [Cloud-Init SSH Port Configuration with Reboot](./cloud-init-ssh-port-reboot.md) | Use cloud-init with reboot pattern to configure custom SSH ports during VM provisioning |
10-
| ✅ Accepted | 2025-12-10 | [Single Docker Image for Sequential E2E Command Testing](./single-docker-image-sequential-testing.md) | Use single Docker image with sequential command execution instead of multi-image phases |
11-
| ✅ Accepted | 2025-12-09 | [Register Command SSH Port Override](./register-ssh-port-override.md) | Add optional --ssh-port argument to register command for non-standard SSH ports |
12-
| ✅ Accepted | 2025-11-19 | [Disable MD060 Table Formatting Rule](./md060-table-formatting-disabled.md) | Disable MD060 to allow flexible table formatting and emoji usage |
13-
| ✅ Accepted | 2025-11-19 | [Test Command as Smoke Test](./test-command-as-smoke-test.md) | Test command validates running services, not infrastructure components |
14-
| ✅ Accepted | 2025-11-13 | [Migration to AGENTS.md Standard](./agents-md-migration.md) | Adopt open AGENTS.md standard for multi-agent compatibility while keeping GitHub redirect |
15-
| ✅ Accepted | 2025-11-11 | [Use ReentrantMutex Pattern for UserOutput Reentrancy](./reentrant-mutex-useroutput-pattern.md) | Use Arc<ReentrantMutex<RefCell<UserOutput>>> to fix same-thread deadlock in issue #164 |
16-
| ❌ Superseded | 2025-11-11 | [Remove UserOutput Mutex](./user-output-mutex-removal.md) | Remove Arc<Mutex<UserOutput>> pattern for simplified, deadlock-free architecture |
17-
| ✅ Accepted | 2025-11-07 | [ExecutionContext Wrapper Pattern](./execution-context-wrapper.md) | Use ExecutionContext wrapper around Container for future-proof command signatures |
18-
| ✅ Accepted | 2025-11-03 | [Environment Variable Prefix](./environment-variable-prefix.md) | Use `TORRUST_TD_` prefix for all environment variables |
19-
| ✅ Accepted | 2025-10-15 | [External Tool Adapters Organization](./external-tool-adapters-organization.md) | Consolidate external tool wrappers in `src/adapters/` for better discoverability |
20-
| ✅ Accepted | 2025-10-10 | [Repository Rename to Deployer](./repository-rename-to-deployer.md) | Rename from "Torrust Tracker Deploy" to "Torrust Tracker Deployer" for production use |
21-
| ✅ Accepted | 2025-10-03 | [Error Context Strategy](./error-context-strategy.md) | Use structured error context with trace files for complete error information |
22-
| ✅ Accepted | 2025-10-03 | [Command State Return Pattern](./command-state-return-pattern.md) | Commands return typed states (Environment<S> → Environment<T>) for compile-time safety |
23-
| ✅ Accepted | 2025-10-03 | [Actionable Error Messages](./actionable-error-messages.md) | Use tiered help system with brief tips + .help() method for detailed troubleshooting |
24-
| ✅ Accepted | 2025-10-01 | [Type Erasure for Environment States](./type-erasure-for-environment-states.md) | Use enum-based type erasure to enable runtime handling and serialization of typed states |
25-
| ✅ Accepted | 2025-09-29 | [Test Context vs Deployment Environment Naming](./test-context-vs-deployment-environment-naming.md) | Rename TestEnvironment to TestContext to avoid conflicts with multi-environment feature |
26-
| ✅ Accepted | 2025-09-10 | [LXD VMs over Containers](./lxd-vm-over-containers.md) | Use LXD virtual machines instead of containers for production alignment |
27-
| ✅ Accepted | 2025-09-09 | [Tera Minimal Templating Strategy](./tera-minimal-templating-strategy.md) | Use Tera with minimal variables and templates to avoid complexity and delimiter conflicts |
28-
| ✅ Accepted | - | [LXD over Multipass](./lxd-over-multipass.md) | Choose LXD containers over Multipass VMs for deployment testing |
29-
| ✅ Resolved | - | [Docker Testing Evolution](./docker-testing-evolution.md) | Evolution from Docker rejection to hybrid approach for split E2E testing |
30-
| ✅ Accepted | - | [Meson Removal](./meson-removal.md) | Remove Meson build system from the project |
7+
| Status | Date | Decision | Summary |
8+
| ------------- | ---------- | --------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------- |
9+
| ✅ Accepted | 2025-12-13 | [Environment Variable Injection in Docker Compose](./environment-variable-injection-in-docker-compose.md) | Use .env file injection instead of hardcoded values for runtime configuration changes |
10+
| ✅ Accepted | 2025-12-11 | [Cloud-Init SSH Port Configuration with Reboot](./cloud-init-ssh-port-reboot.md) | Use cloud-init with reboot pattern to configure custom SSH ports during VM provisioning |
11+
| ✅ Accepted | 2025-12-10 | [Single Docker Image for Sequential E2E Command Testing](./single-docker-image-sequential-testing.md) | Use single Docker image with sequential command execution instead of multi-image phases |
12+
| ✅ Accepted | 2025-12-09 | [Register Command SSH Port Override](./register-ssh-port-override.md) | Add optional --ssh-port argument to register command for non-standard SSH ports |
13+
| ✅ Accepted | 2025-11-19 | [Disable MD060 Table Formatting Rule](./md060-table-formatting-disabled.md) | Disable MD060 to allow flexible table formatting and emoji usage |
14+
| ✅ Accepted | 2025-11-19 | [Test Command as Smoke Test](./test-command-as-smoke-test.md) | Test command validates running services, not infrastructure components |
15+
| ✅ Accepted | 2025-11-13 | [Migration to AGENTS.md Standard](./agents-md-migration.md) | Adopt open AGENTS.md standard for multi-agent compatibility while keeping GitHub redirect |
16+
| ✅ Accepted | 2025-11-11 | [Use ReentrantMutex Pattern for UserOutput Reentrancy](./reentrant-mutex-useroutput-pattern.md) | Use Arc<ReentrantMutex<RefCell<UserOutput>>> to fix same-thread deadlock in issue #164 |
17+
| ❌ Superseded | 2025-11-11 | [Remove UserOutput Mutex](./user-output-mutex-removal.md) | Remove Arc<Mutex<UserOutput>> pattern for simplified, deadlock-free architecture |
18+
| ✅ Accepted | 2025-11-07 | [ExecutionContext Wrapper Pattern](./execution-context-wrapper.md) | Use ExecutionContext wrapper around Container for future-proof command signatures |
19+
| ✅ Accepted | 2025-11-03 | [Environment Variable Prefix](./environment-variable-prefix.md) | Use `TORRUST_TD_` prefix for all environment variables |
20+
| ✅ Accepted | 2025-10-15 | [External Tool Adapters Organization](./external-tool-adapters-organization.md) | Consolidate external tool wrappers in `src/adapters/` for better discoverability |
21+
| ✅ Accepted | 2025-10-10 | [Repository Rename to Deployer](./repository-rename-to-deployer.md) | Rename from "Torrust Tracker Deploy" to "Torrust Tracker Deployer" for production use |
22+
| ✅ Accepted | 2025-10-03 | [Error Context Strategy](./error-context-strategy.md) | Use structured error context with trace files for complete error information |
23+
| ✅ Accepted | 2025-10-03 | [Command State Return Pattern](./command-state-return-pattern.md) | Commands return typed states (Environment<S> → Environment<T>) for compile-time safety |
24+
| ✅ Accepted | 2025-10-03 | [Actionable Error Messages](./actionable-error-messages.md) | Use tiered help system with brief tips + .help() method for detailed troubleshooting |
25+
| ✅ Accepted | 2025-10-01 | [Type Erasure for Environment States](./type-erasure-for-environment-states.md) | Use enum-based type erasure to enable runtime handling and serialization of typed states |
26+
| ✅ Accepted | 2025-09-29 | [Test Context vs Deployment Environment Naming](./test-context-vs-deployment-environment-naming.md) | Rename TestEnvironment to TestContext to avoid conflicts with multi-environment feature |
27+
| ✅ Accepted | 2025-09-10 | [LXD VMs over Containers](./lxd-vm-over-containers.md) | Use LXD virtual machines instead of containers for production alignment |
28+
| ✅ Accepted | 2025-09-09 | [Tera Minimal Templating Strategy](./tera-minimal-templating-strategy.md) | Use Tera with minimal variables and templates to avoid complexity and delimiter conflicts |
29+
| ✅ Accepted | - | [LXD over Multipass](./lxd-over-multipass.md) | Choose LXD containers over Multipass VMs for deployment testing |
30+
| ✅ Resolved | - | [Docker Testing Evolution](./docker-testing-evolution.md) | Evolution from Docker rejection to hybrid approach for split E2E testing |
31+
| ✅ Accepted | - | [Meson Removal](./meson-removal.md) | Remove Meson build system from the project |
3132

3233
## ADR Template
3334

‎docs/decisions/environment-variable-injection-in-docker-compose.md‎

Lines changed: 183 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,183 @@
1+
# Environment Variable Injection in Docker Compose Templates
2+
3+
- **Status**: ✅ Accepted
4+
- **Date**: 2025-12-13
5+
- **Related Issues**: #232 (MySQL Slice - Release and Run Commands)
6+
- **Related Decisions**:
7+
- [Tera Minimal Templating Strategy](./tera-minimal-templating-strategy.md)
8+
- [Environment Variable Prefix](./environment-variable-prefix.md)
9+
10+
## Context
11+
12+
When implementing MySQL support for the tracker deployment (#232), we initially hardcoded MySQL credentials directly in the `docker-compose.yml.tera` template using Tera template variables:
13+
14+
```yaml
15+
# ❌ INCORRECT: Hardcoded values at template generation time
16+
environment:
17+
- MYSQL_ROOT_PASSWORD={{ database.mysql.root_password }}
18+
- MYSQL_DATABASE={{ database.mysql.database }}
19+
- MYSQL_USER={{ database.mysql.user }}
20+
- MYSQL_PASSWORD={{ database.mysql.password }}
21+
```
22+
23+
This approach had a critical flaw: values are embedded into the `docker-compose.yml` file at template generation time (during the `release` command). System administrators managing a deployed system cannot modify these values without regenerating the entire template infrastructure.
24+
25+
The existing tracker service already uses the correct pattern - referencing environment variables from the `.env` file:
26+
27+
```yaml
28+
# ✅ CORRECT: Reference to .env file variable
29+
environment:
30+
- TORRUST_TRACKER_CONFIG_OVERRIDE_HTTP_API__ACCESS_TOKENS__ADMIN=${TORRUST_TRACKER_CONFIG_OVERRIDE_HTTP_API__ACCESS_TOKENS__ADMIN}
31+
```
32+
33+
This pattern allows system administrators to:
34+
35+
1. Edit the `.env` file with new values
36+
2. Restart the Docker Compose stack: `docker-compose down && docker-compose up -d`
37+
3. Have the new configuration take effect immediately
38+
39+
Without regenerating any templates or redeploying the entire application stack.
40+
41+
## Decision
42+
43+
**All configuration values that may need to be changed during system maintenance must be injected via environment variables from the `.env` file, not hardcoded in the `docker-compose.yml` template.**
44+
45+
### Implementation Pattern
46+
47+
**Template Generation (deploy-time)**:
48+
49+
- `EnvContext` contains actual credential values
50+
- `.env.tera` template renders these values: `MYSQL_ROOT_PASSWORD='{{ mysql_root_password }}'`
51+
- `docker-compose.yml.tera` references environment variables: `MYSQL_ROOT_PASSWORD=${MYSQL_ROOT_PASSWORD}`
52+
53+
**Runtime (maintenance)**:
54+
55+
- System administrator edits `.env` file with new credentials
56+
- Restarts Docker Compose services
57+
- New credentials take effect without template regeneration
58+
59+
### Template Documentation
60+
61+
Added comment block to `docker-compose.yml.tera` to prevent future violations:
62+
63+
```yaml
64+
# IMPORTANT: Environment Variable Injection Pattern
65+
# ================================================
66+
# All configuration values that may need to be changed during maintenance
67+
# should be injected via environment variables from the .env file, not
68+
# hardcoded in this docker-compose template.
69+
#
70+
# Pattern to follow:
71+
# CORRECT: - MYSQL_ROOT_PASSWORD=${MYSQL_ROOT_PASSWORD}
72+
# INCORRECT: - MYSQL_ROOT_PASSWORD=hardcoded_value
73+
#
74+
# Rationale:
75+
# - System administrators can modify .env values and restart services without
76+
# regenerating templates
77+
# - Supports runtime configuration changes without redeployment
78+
# - Follows Docker Compose best practices for configuration management
79+
# - Separates template generation (deploy-time) from configuration (runtime)
80+
```
81+
82+
## Consequences
83+
84+
### Positive
85+
86+
1. **Runtime Configuration Changes**: System administrators can modify credentials and configuration without regenerating templates or redeploying
87+
2. **Standard Docker Compose Practice**: Follows official Docker Compose recommendations for environment variable management
88+
3. **Separation of Concerns**: Clear separation between:
89+
- Template structure (generated once during release)
90+
- Configuration values (modifiable during maintenance)
91+
4. **Consistent Pattern**: All services follow the same environment variable injection pattern
92+
5. **Security Benefits**: Credentials can be rotated without code changes or redeployment
93+
6. **Reduced Coupling**: Template generation is independent of specific credential values
94+
95+
### Negative
96+
97+
1. **Template Complexity**: Requires maintaining both `.env.tera` and `docker-compose.yml.tera` templates
98+
2. **Two-File Coordination**: Developers must ensure `.env.tera` defines all variables referenced in `docker-compose.yml.tera`
99+
3. **Testing Considerations**: Tests must verify both template generation and environment variable injection work correctly
100+
101+
### Neutral
102+
103+
1. **Documentation Requirement**: Pattern must be clearly documented to prevent future hardcoding
104+
2. **Code Review Focus**: PRs adding new services must verify environment variable injection pattern
105+
106+
## Implementation Notes
107+
108+
### Files Modified
109+
110+
1. **`src/infrastructure/templating/docker_compose/template/wrappers/env/context.rs`**:
111+
112+
- Extended `EnvContext` to include optional MySQL credentials
113+
- Added `new_with_mysql()` constructor for MySQL mode
114+
- Added getters for MySQL fields
115+
116+
2. **`templates/docker-compose/.env.tera`**:
117+
118+
- Added conditional MySQL environment variables section
119+
- Variables only rendered when MySQL is configured
120+
121+
3. **`templates/docker-compose/docker-compose.yml.tera`**:
122+
123+
- Changed MySQL service to use `${MYSQL_ROOT_PASSWORD}` syntax
124+
- Added documentation comment explaining the pattern
125+
- Port hardcoded to 3306 (not configuration-dependent)
126+
127+
4. **`src/application/steps/rendering/docker_compose_templates.rs`**:
128+
- Updated to create `EnvContext` with MySQL credentials when configured
129+
- Passes MySQL values to both `EnvContext` and `DockerComposeContext`
130+
131+
### Testing Strategy
132+
133+
- Unit tests verify environment variable references appear in rendered `docker-compose.yml`
134+
- Tests check for `${MYSQL_ROOT_PASSWORD}` instead of hardcoded values
135+
- `.env` file generation tests verify MySQL variables are present when configured
136+
137+
## Alternatives Considered
138+
139+
### 1. Hardcode Values in docker-compose.yml
140+
141+
**Approach**: Keep using `{{ database.mysql.root_password }}` in `docker-compose.yml.tera`
142+
143+
**Rejected because**:
144+
145+
- Requires regenerating entire template stack for credential changes
146+
- Violates Docker Compose best practices
147+
- Inconsistent with existing tracker service pattern
148+
- Forces redeployment for routine maintenance tasks
149+
150+
### 2. Use docker-compose.yml Variables with Defaults
151+
152+
**Approach**: Use `${MYSQL_ROOT_PASSWORD:-default_value}` syntax
153+
154+
**Rejected because**:
155+
156+
- Defaults in production deployments are security anti-pattern
157+
- Still requires `.env` file for actual production values
158+
- Adds unnecessary complexity
159+
- No benefit over explicit `.env` requirement
160+
161+
### 3. External Configuration Management (Vault, AWS Secrets Manager)
162+
163+
**Approach**: Fetch credentials from external secrets management system
164+
165+
**Deferred because**:
166+
167+
- Out of scope for current milestone
168+
- Adds infrastructure dependencies
169+
- Current pattern is sufficient for target use case
170+
- Can be added later without breaking changes
171+
172+
## References
173+
174+
- [Docker Compose Environment Variables Documentation](https://docs.docker.com/compose/environment-variables/)
175+
- [Issue #232: MySQL Slice - Release and Run Commands](https://github.com/torrust/torrust-tracker-deployer/issues/232)
176+
- [12-Factor App: Config](https://12factor.net/config)
177+
178+
## Notes
179+
180+
- This pattern applies to all Docker Compose services, not just MySQL
181+
- Future services must follow the same environment variable injection pattern
182+
- Template variable syntax (`{{ var }}`) should only be used for structural elements that never change at runtime
183+
- Port mapping for MySQL hardcoded to 3306:3306 as it's not expected to vary per deployment

0 commit comments

Comments
 (0)