You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
feat: [#232] Implement MySQL support for tracker deployment with critical service dependency fix
This commit completes the MySQL slice implementation for the release and run commands,
adding full MySQL database support to the Torrust Tracker deployment workflow.
CRITICAL FIX: Docker Compose Service Dependencies
==================================================
Added 'depends_on' with 'condition: service_healthy' to the tracker service in the
docker-compose template. This ensures the tracker container waits for the MySQL
healthcheck to pass before starting, preventing startup race conditions and
'Connection refused' errors.
Without this fix, the tracker would attempt to connect to MySQL immediately upon
container start, often before MySQL was ready to accept connections, resulting in:
ERROR r2d2: DriverError { Could not connect to address `mysql:3306': Connection refused }
The healthcheck configuration includes:
- Interval: 10s
- Timeout: 5s
- Retries: 5
- Start period: 30s
This increases total startup time by ~6 seconds (from ~19s to ~25s) but ensures
reliable service initialization.
Template Changes
================
1. Docker Compose Template (docker-compose.yml.tera):
- Added depends_on section to tracker service
- Conditional dependency on mysql service when database.driver == "mysql"
- Uses service_healthy condition for proper startup ordering
- Added standardized TORRUST_TRACKER_CONFIG_OVERRIDE_CORE__DATABASE__DRIVER env var
- Removed hardcoded 'sqlite3' value, now uses environment variable injection
2. Environment Variables Template (.env.tera):
- Standardized variable: TORRUST_TRACKER_CONFIG_OVERRIDE_CORE__DATABASE__DRIVER
- Replaced legacy DATABASE_DRIVER with standardized name
- Added explanatory comment about entrypoint script requirement
- Added section headers: "Tracker Service Configuration" and "MySQL Service Configuration"
- Changed conditional from {% if mysql_root_password %} to {% if database_driver == "mysql" %}
- Explicit database_driver check improves readability and correctness
Code Changes
============
3. EnvContext Structure (src/infrastructure/templating/docker_compose/template/wrappers/env/context.rs):
- Added database_driver field to EnvContext struct
- new() method sets database_driver to "sqlite3" (default)
- new_with_mysql() method sets database_driver to "mysql"
- Field controls template conditionals and variable rendering
Documentation
=============
4. Tracker Issues Documentation (docs/tracker-issues/):
- Created README.md explaining folder purpose and scope
- Created database-driver-double-specification.md documenting tracker entrypoint behavior
- Documents that tracker requires driver in both config file AND environment variable
- Explains root cause, impact, workaround, and recommended solution
- References tracker entrypoint script source code
5. Manual Testing Guide (docs/e2e-testing/manual-testing-mysql.md):
- Complete end-to-end manual testing workflow documented
- Updated with actual command outputs from successful test run
- Corrected VM file locations (changed from /srv/torrust to /opt/torrust)
- Added debugging section explaining data/logs/log.txt usage
- Documented depends_on fix and its importance
- Added CRITICAL warning about envs/ vs data/ directories distinction
- Deep verification sections for MySQL, tracker, and HTTP API
6. AGENTS.md Updates:
- Added Rule 1: CRITICAL explanation of envs/ vs data/ directories
- Most frequently violated rule - clearly distinguishes user input from application state
- Renumbered subsequent rules (2-17)
- Maintains complete AI assistant instructions
Environment Variable Standardization
=====================================
The tracker container's develop branch requires the standardized variable name:
TORRUST_TRACKER_CONFIG_OVERRIDE_CORE__DATABASE__DRIVER
This follows the TORRUST_TRACKER_CONFIG_OVERRIDE_* pattern used by the tracker's
entrypoint script to select the appropriate default configuration template.
The entrypoint script REQUIRES this variable to be set, otherwise the container
Testing
=======
All tests passing:
- Unit tests: 1469 passed
- Integration tests: 40 passed
- E2E infrastructure lifecycle: PASSED
- E2E deployment workflow: PASSED
- Manual MySQL E2E test: Complete and successful
- Pre-commit checks: ALL PASSED
Manual test verified:
- No startup race conditions
- No "Connection refused" errors
- MySQL tables created correctly
- Tracker HTTP API responding
- Both containers healthy
- Environment variables correctly injected
Implementation Details
======================
This implementation follows ADRs:
- Environment Variable Injection in Docker Compose
- Database Configuration Structure in Templates
- Template System Architecture
Backwards compatibility:
- SQLite remains the default driver (database_driver = "sqlite3")
- MySQL slice is opt-in via environment configuration
Related Issue: #232
Copy file name to clipboardExpand all lines: AGENTS.md
+45-16Lines changed: 45 additions & 16 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -69,16 +69,45 @@ These principles should guide all development decisions, code reviews, and featu
69
69
70
70
## 🔧 Essential Rules
71
71
72
-
1.**Before placing code in DDD layers**: Read [`docs/contributing/ddd-layer-placement.md`](docs/contributing/ddd-layer-placement.md) for comprehensive guidance on which code belongs in which layer (Domain, Application, Infrastructure, Presentation). This guide includes rules, red flags, examples, and a decision flowchart to help you make the right architectural decisions.
- Usage: Passed to `create environment --env-file envs/your-config.json`
82
+
- Example: `envs/manual-test-mysql.json`
83
+
- Version Control: Gitignored (user-specific)
84
+
-**Rule**: You MAY create/edit files here as part of documentation or testing
85
+
86
+
-**`data/` directory** - Internal Application State (APPLICATION MANAGED):
87
+
- Purpose: Serialized Rust structs representing deployment state machine
88
+
- Format: Rust `Environment<State>` domain model serialization
89
+
- Contains: State transitions, runtime outputs, trace IDs, timestamps
90
+
- Usage: Internal state management, read-only inspection
91
+
- Example: `data/manual-test-mysql/environment.json` (NOT the same as `envs/manual-test-mysql.json`)
92
+
- Version Control: Gitignored (runtime-generated)
93
+
-**Rule**: You MUST NEVER create/edit files here - READ ONLY for debugging/verification
94
+
95
+
**NEVER CONFUSE THESE TWO!** When documenting or testing:
96
+
97
+
- User creates config in `envs/your-env.json`
98
+
- Application manages state in `data/your-env/environment.json`
99
+
- These are completely different JSON structures with different purposes
100
+
101
+
2.**Before placing code in DDD layers**: Read [`docs/contributing/ddd-layer-placement.md`](docs/contributing/ddd-layer-placement.md) for comprehensive guidance on which code belongs in which layer (Domain, Application, Infrastructure, Presentation). This guide includes rules, red flags, examples, and a decision flowchart to help you make the right architectural decisions.
102
+
103
+
3.**Before creating branches**: Read [`docs/contributing/branching.md`](docs/contributing/branching.md) for naming conventions (`{issue-number}-{short-description}`)
104
+
105
+
4.**Before committing**: Read [`docs/contributing/commit-process.md`](docs/contributing/commit-process.md) for conventional commits
77
106
78
107
-**With issue branch**: `{type}: [#{issue}] {description}` (when branch name starts with `{issue-number}-`)
79
108
-**Without issue branch**: `{type}: {description}` (when working on main or branch without issue number prefix)
80
109
81
-
4.**Before committing**: Always run the pre-commit verification script - all checks must pass before staging files or creating commits, regardless of the tool or method used:
110
+
5.**Before committing**: Always run the pre-commit verification script - all checks must pass before staging files or creating commits, regardless of the tool or method used:
82
111
83
112
```bash
84
113
./scripts/pre-commit.sh
@@ -92,32 +121,32 @@ These principles should guide all development decisions, code reviews, and featu
92
121
- Git clients: GitHub Desktop, GitKraken, etc.
93
122
- CI/CD: Any automated commits or merges
94
123
95
-
5.**Before working with Tera templates**: Read [`docs/contributing/templates.md`](docs/contributing/templates.md) for correct variable syntax - use `{{ variable }}` not `{ { variable } }`. Tera template files have the `.tera` extension.
124
+
6.**Before working with Tera templates**: Read [`docs/contributing/templates.md`](docs/contributing/templates.md) for correct variable syntax - use `{{ variable }}` not `{ { variable } }`. Tera template files have the `.tera` extension.
96
125
97
-
6.**When adding new Ansible playbooks**: Read [`docs/contributing/templates.md`](docs/contributing/templates.md) for the complete guide. **CRITICAL**: Static playbooks (without `.tera` extension) must be registered in `src/infrastructure/external_tools/ansible/template/renderer/project_generator.rs` in the `copy_static_templates` method, otherwise they won't be copied to the build directory and Ansible will fail with "playbook not found" error.
126
+
7.**When adding new Ansible playbooks**: Read [`docs/contributing/templates.md`](docs/contributing/templates.md) for the complete guide. **CRITICAL**: Static playbooks (without `.tera` extension) must be registered in `src/infrastructure/external_tools/ansible/template/renderer/project_generator.rs` in the `copy_static_templates` method, otherwise they won't be copied to the build directory and Ansible will fail with "playbook not found" error.
98
127
99
-
7.**When handling errors in code**: Read [`docs/contributing/error-handling.md`](docs/contributing/error-handling.md) for error handling principles. Prefer explicit enum errors over anyhow for better pattern matching and user experience. Make errors clear, include sufficient context for traceability, and ensure they are actionable with specific fix instructions.
128
+
8.**When handling errors in code**: Read [`docs/contributing/error-handling.md`](docs/contributing/error-handling.md) for error handling principles. Prefer explicit enum errors over anyhow for better pattern matching and user experience. Make errors clear, include sufficient context for traceability, and ensure they are actionable with specific fix instructions.
100
129
101
-
8.**When producing any output to users** (CRITICAL for architecture): Read [`docs/contributing/output-handling.md`](docs/contributing/output-handling.md) for output handling conventions. **NEVER use `println!`, `eprintln!`, `print!`, `eprint!`, or direct access to `std::io::stdout()`/`std::io::stderr()`**. Always use `UserOutput` methods through the execution context. This ensures testability, consistent formatting, proper channel routing (stdout vs stderr), verbosity control, and theme support. Example: `ctx.user_output().lock().borrow_mut().progress("Processing...")` instead of `println!("Processing...")`.
130
+
9.**When producing any output to users** (CRITICAL for architecture): Read [`docs/contributing/output-handling.md`](docs/contributing/output-handling.md) for output handling conventions. **NEVER use `println!`, `eprintln!`, `print!`, `eprint!`, or direct access to `std::io::stdout()`/`std::io::stderr()`**. Always use `UserOutput` methods through the execution context. This ensures testability, consistent formatting, proper channel routing (stdout vs stderr), verbosity control, and theme support. Example: `ctx.user_output().lock().borrow_mut().progress("Processing...")` instead of `println!("Processing...")`.
102
131
103
-
9.**Understanding expected errors**: Read [`docs/contributing/known-issues.md`](docs/contributing/known-issues.md) for known issues and expected behaviors. Some errors that appear red in E2E test output (like SSH host key warnings) are normal and expected - not actual failures.
132
+
10.**Understanding expected errors**: Read [`docs/contributing/known-issues.md`](docs/contributing/known-issues.md) for known issues and expected behaviors. Some errors that appear red in E2E test output (like SSH host key warnings) are normal and expected - not actual failures.
104
133
105
-
10.**Before making engineering decisions**: Document significant architectural or design decisions as Architectural Decision Records (ADRs) in `docs/decisions/`. Read [`docs/decisions/README.md`](docs/decisions/README.md) for the ADR template and guidelines. This ensures decisions are properly documented with context, rationale, and consequences for future reference.
134
+
11.**Before making engineering decisions**: Document significant architectural or design decisions as Architectural Decision Records (ADRs) in `docs/decisions/`. Read [`docs/decisions/README.md`](docs/decisions/README.md) for the ADR template and guidelines. This ensures decisions are properly documented with context, rationale, and consequences for future reference.
106
135
107
-
11.**When organizing code within modules**: Follow the module organization conventions in [`docs/contributing/module-organization.md`](docs/contributing/module-organization.md). Use top-down organization with public items first, high-level abstractions before low-level details, and important responsibilities before secondary concerns like error types.
136
+
12.**When organizing code within modules**: Follow the module organization conventions in [`docs/contributing/module-organization.md`](docs/contributing/module-organization.md). Use top-down organization with public items first, high-level abstractions before low-level details, and important responsibilities before secondary concerns like error types.
108
137
109
-
12.**When writing Rust imports** (CRITICAL for code style): Follow the import conventions in [`docs/contributing/module-organization.md`](docs/contributing/module-organization.md). Two essential rules:
138
+
13.**When writing Rust imports** (CRITICAL for code style): Follow the import conventions in [`docs/contributing/module-organization.md`](docs/contributing/module-organization.md). Two essential rules:
110
139
111
140
-**Imports Always First**: Keep all imports at the top of the file, organized in groups (std → external crates → internal crate).
112
141
-**Prefer Imports Over Full Paths**: Always import types and use short names (e.g., `Arc<UserOutput>`) rather than fully-qualified paths. Never use long paths like `std::sync::Arc<crate::presentation::views::UserOutput>` in regular code - only use full paths when disambiguating naming conflicts.
113
142
114
-
13.**When writing Markdown documentation**: Be aware of GitHub Flavored Markdown's automatic linking behavior. Read [`docs/contributing/github-markdown-pitfalls.md`](docs/contributing/github-markdown-pitfalls.md) for critical patterns to avoid. **NEVER use hash-number patterns for enumeration or step numbering** - this creates unintended links to GitHub issues/PRs. Use ordered lists or alternative formats instead.
143
+
14.**When writing Markdown documentation**: Be aware of GitHub Flavored Markdown's automatic linking behavior. Read [`docs/contributing/github-markdown-pitfalls.md`](docs/contributing/github-markdown-pitfalls.md) for critical patterns to avoid. **NEVER use hash-number patterns for enumeration or step numbering** - this creates unintended links to GitHub issues/PRs. Use ordered lists or alternative formats instead.
115
144
116
-
14.**When creating new environment variables**: Read [`docs/contributing/environment-variables-naming.md`](docs/contributing/environment-variables-naming.md) for comprehensive guidance on naming conventions (condition-based vs action-based), decision frameworks, and best practices. Also review [`docs/decisions/environment-variable-prefix.md`](docs/decisions/environment-variable-prefix.md) to ensure all project environment variables use the `TORRUST_TD_` prefix for proper namespacing and avoiding conflicts with system or user variables.
145
+
15.**When creating new environment variables**: Read [`docs/contributing/environment-variables-naming.md`](docs/contributing/environment-variables-naming.md) for comprehensive guidance on naming conventions (condition-based vs action-based), decision frameworks, and best practices. Also review [`docs/decisions/environment-variable-prefix.md`](docs/decisions/environment-variable-prefix.md) to ensure all project environment variables use the `TORRUST_TD_` prefix for proper namespacing and avoiding conflicts with system or user variables.
117
146
118
-
15.**When adding new templates**: Read [`docs/technical/template-system-architecture.md`](docs/technical/template-system-architecture.md) to understand the Project Generator pattern. The `templates/` directory contains source templates. Dynamic templates (`.tera`) are automatically processed, but static files must be explicitly registered in their respective `ProjectGenerator` to be copied to the build directory.
147
+
16.**When adding new templates**: Read [`docs/technical/template-system-architecture.md`](docs/technical/template-system-architecture.md) to understand the Project Generator pattern. The `templates/` directory contains source templates. Dynamic templates (`.tera`) are automatically processed, but static files must be explicitly registered in their respective `ProjectGenerator` to be copied to the build directory.
119
148
120
-
16.**When writing unit tests** (CRITICAL for test quality): Read [`docs/contributing/testing/unit-testing.md`](docs/contributing/testing/unit-testing.md) and follow the behavior-driven naming convention. **NEVER use the `test_` prefix** for test function names. Always use the `it_should_{expected_behavior}_when_{condition}` or `it_should_{expected_behavior}_given_{state}` pattern. This ensures tests clearly document the behavior being validated and the conditions under which it occurs. Example: `it_should_return_error_when_username_is_invalid()` instead of `test_invalid_username()`. Test names should follow the three-part structure (What-When-Then) and be descriptive enough that the test's purpose is clear without reading the code.
149
+
17.**When writing unit tests** (CRITICAL for test quality): Read [`docs/contributing/testing/unit-testing.md`](docs/contributing/testing/unit-testing.md) and follow the behavior-driven naming convention. **NEVER use the `test_` prefix** for test function names. Always use the `it_should_{expected_behavior}_when_{condition}` or `it_should_{expected_behavior}_given_{state}` pattern. This ensures tests clearly document the behavior being validated and the conditions under which it occurs. Example: `it_should_return_error_when_username_is_invalid()` instead of `test_invalid_username()`. Test names should follow the three-part structure (What-When-Then) and be descriptive enough that the test's purpose is clear without reading the code.
0 commit comments