Skip to content

Commit 363a70a

Browse files
committed
feat: [#232] Implement MySQL support for tracker deployment with critical service dependency fix
This commit completes the MySQL slice implementation for the release and run commands, adding full MySQL database support to the Torrust Tracker deployment workflow. CRITICAL FIX: Docker Compose Service Dependencies ================================================== Added 'depends_on' with 'condition: service_healthy' to the tracker service in the docker-compose template. This ensures the tracker container waits for the MySQL healthcheck to pass before starting, preventing startup race conditions and 'Connection refused' errors. Without this fix, the tracker would attempt to connect to MySQL immediately upon container start, often before MySQL was ready to accept connections, resulting in: ERROR r2d2: DriverError { Could not connect to address `mysql:3306': Connection refused } The healthcheck configuration includes: - Interval: 10s - Timeout: 5s - Retries: 5 - Start period: 30s This increases total startup time by ~6 seconds (from ~19s to ~25s) but ensures reliable service initialization. Template Changes ================ 1. Docker Compose Template (docker-compose.yml.tera): - Added depends_on section to tracker service - Conditional dependency on mysql service when database.driver == "mysql" - Uses service_healthy condition for proper startup ordering - Added standardized TORRUST_TRACKER_CONFIG_OVERRIDE_CORE__DATABASE__DRIVER env var - Removed hardcoded 'sqlite3' value, now uses environment variable injection 2. Environment Variables Template (.env.tera): - Standardized variable: TORRUST_TRACKER_CONFIG_OVERRIDE_CORE__DATABASE__DRIVER - Replaced legacy DATABASE_DRIVER with standardized name - Added explanatory comment about entrypoint script requirement - Added section headers: "Tracker Service Configuration" and "MySQL Service Configuration" - Changed conditional from {% if mysql_root_password %} to {% if database_driver == "mysql" %} - Explicit database_driver check improves readability and correctness Code Changes ============ 3. EnvContext Structure (src/infrastructure/templating/docker_compose/template/wrappers/env/context.rs): - Added database_driver field to EnvContext struct - new() method sets database_driver to "sqlite3" (default) - new_with_mysql() method sets database_driver to "mysql" - Field controls template conditionals and variable rendering Documentation ============= 4. Tracker Issues Documentation (docs/tracker-issues/): - Created README.md explaining folder purpose and scope - Created database-driver-double-specification.md documenting tracker entrypoint behavior - Documents that tracker requires driver in both config file AND environment variable - Explains root cause, impact, workaround, and recommended solution - References tracker entrypoint script source code 5. Manual Testing Guide (docs/e2e-testing/manual-testing-mysql.md): - Complete end-to-end manual testing workflow documented - Updated with actual command outputs from successful test run - Corrected VM file locations (changed from /srv/torrust to /opt/torrust) - Added debugging section explaining data/logs/log.txt usage - Documented depends_on fix and its importance - Added CRITICAL warning about envs/ vs data/ directories distinction - Deep verification sections for MySQL, tracker, and HTTP API 6. AGENTS.md Updates: - Added Rule 1: CRITICAL explanation of envs/ vs data/ directories - Most frequently violated rule - clearly distinguishes user input from application state - Renumbered subsequent rules (2-17) - Maintains complete AI assistant instructions Environment Variable Standardization ===================================== The tracker container's develop branch requires the standardized variable name: TORRUST_TRACKER_CONFIG_OVERRIDE_CORE__DATABASE__DRIVER This follows the TORRUST_TRACKER_CONFIG_OVERRIDE_* pattern used by the tracker's entrypoint script to select the appropriate default configuration template. The entrypoint script REQUIRES this variable to be set, otherwise the container Testing ======= All tests passing: - Unit tests: 1469 passed - Integration tests: 40 passed - E2E infrastructure lifecycle: PASSED - E2E deployment workflow: PASSED - Manual MySQL E2E test: Complete and successful - Pre-commit checks: ALL PASSED Manual test verified: - No startup race conditions - No "Connection refused" errors - MySQL tables created correctly - Tracker HTTP API responding - Both containers healthy - Environment variables correctly injected Implementation Details ====================== This implementation follows ADRs: - Environment Variable Injection in Docker Compose - Database Configuration Structure in Templates - Template System Architecture Backwards compatibility: - SQLite remains the default driver (database_driver = "sqlite3") - MySQL slice is opt-in via environment configuration Related Issue: #232
1 parent 6f38ffd commit 363a70a

File tree

7 files changed

+950
-22
lines changed

7 files changed

+950
-22
lines changed

AGENTS.md

Lines changed: 45 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -69,16 +69,45 @@ These principles should guide all development decisions, code reviews, and featu
6969

7070
## 🔧 Essential Rules
7171

72-
1. **Before placing code in DDD layers**: Read [`docs/contributing/ddd-layer-placement.md`](docs/contributing/ddd-layer-placement.md) for comprehensive guidance on which code belongs in which layer (Domain, Application, Infrastructure, Presentation). This guide includes rules, red flags, examples, and a decision flowchart to help you make the right architectural decisions.
72+
1. **CRITICAL - Understanding `envs/` vs `data/` directories** (⚠️ **MOST FREQUENTLY VIOLATED RULE**):
7373

74-
2. **Before creating branches**: Read [`docs/contributing/branching.md`](docs/contributing/branching.md) for naming conventions (`{issue-number}-{short-description}`)
74+
**TWO COMPLETELY DIFFERENT FILE PURPOSES:**
7575

76-
3. **Before committing**: Read [`docs/contributing/commit-process.md`](docs/contributing/commit-process.md) for conventional commits
76+
- **`envs/` directory** - User Environment Configurations (USER INPUT):
77+
78+
- Purpose: User-created configuration files for environment creation
79+
- Format: Environment creation schema (see `envs/environment-schema.json`)
80+
- Contains: Provider config, SSH credentials, tracker settings, database config
81+
- Usage: Passed to `create environment --env-file envs/your-config.json`
82+
- Example: `envs/manual-test-mysql.json`
83+
- Version Control: Gitignored (user-specific)
84+
- **Rule**: You MAY create/edit files here as part of documentation or testing
85+
86+
- **`data/` directory** - Internal Application State (APPLICATION MANAGED):
87+
- Purpose: Serialized Rust structs representing deployment state machine
88+
- Format: Rust `Environment<State>` domain model serialization
89+
- Contains: State transitions, runtime outputs, trace IDs, timestamps
90+
- Usage: Internal state management, read-only inspection
91+
- Example: `data/manual-test-mysql/environment.json` (NOT the same as `envs/manual-test-mysql.json`)
92+
- Version Control: Gitignored (runtime-generated)
93+
- **Rule**: You MUST NEVER create/edit files here - READ ONLY for debugging/verification
94+
95+
**NEVER CONFUSE THESE TWO!** When documenting or testing:
96+
97+
- User creates config in `envs/your-env.json`
98+
- Application manages state in `data/your-env/environment.json`
99+
- These are completely different JSON structures with different purposes
100+
101+
2. **Before placing code in DDD layers**: Read [`docs/contributing/ddd-layer-placement.md`](docs/contributing/ddd-layer-placement.md) for comprehensive guidance on which code belongs in which layer (Domain, Application, Infrastructure, Presentation). This guide includes rules, red flags, examples, and a decision flowchart to help you make the right architectural decisions.
102+
103+
3. **Before creating branches**: Read [`docs/contributing/branching.md`](docs/contributing/branching.md) for naming conventions (`{issue-number}-{short-description}`)
104+
105+
4. **Before committing**: Read [`docs/contributing/commit-process.md`](docs/contributing/commit-process.md) for conventional commits
77106

78107
- **With issue branch**: `{type}: [#{issue}] {description}` (when branch name starts with `{issue-number}-`)
79108
- **Without issue branch**: `{type}: {description}` (when working on main or branch without issue number prefix)
80109

81-
4. **Before committing**: Always run the pre-commit verification script - all checks must pass before staging files or creating commits, regardless of the tool or method used:
110+
5. **Before committing**: Always run the pre-commit verification script - all checks must pass before staging files or creating commits, regardless of the tool or method used:
82111

83112
```bash
84113
./scripts/pre-commit.sh
@@ -92,32 +121,32 @@ These principles should guide all development decisions, code reviews, and featu
92121
- Git clients: GitHub Desktop, GitKraken, etc.
93122
- CI/CD: Any automated commits or merges
94123

95-
5. **Before working with Tera templates**: Read [`docs/contributing/templates.md`](docs/contributing/templates.md) for correct variable syntax - use `{{ variable }}` not `{ { variable } }`. Tera template files have the `.tera` extension.
124+
6. **Before working with Tera templates**: Read [`docs/contributing/templates.md`](docs/contributing/templates.md) for correct variable syntax - use `{{ variable }}` not `{ { variable } }`. Tera template files have the `.tera` extension.
96125

97-
6. **When adding new Ansible playbooks**: Read [`docs/contributing/templates.md`](docs/contributing/templates.md) for the complete guide. **CRITICAL**: Static playbooks (without `.tera` extension) must be registered in `src/infrastructure/external_tools/ansible/template/renderer/project_generator.rs` in the `copy_static_templates` method, otherwise they won't be copied to the build directory and Ansible will fail with "playbook not found" error.
126+
7. **When adding new Ansible playbooks**: Read [`docs/contributing/templates.md`](docs/contributing/templates.md) for the complete guide. **CRITICAL**: Static playbooks (without `.tera` extension) must be registered in `src/infrastructure/external_tools/ansible/template/renderer/project_generator.rs` in the `copy_static_templates` method, otherwise they won't be copied to the build directory and Ansible will fail with "playbook not found" error.
98127

99-
7. **When handling errors in code**: Read [`docs/contributing/error-handling.md`](docs/contributing/error-handling.md) for error handling principles. Prefer explicit enum errors over anyhow for better pattern matching and user experience. Make errors clear, include sufficient context for traceability, and ensure they are actionable with specific fix instructions.
128+
8. **When handling errors in code**: Read [`docs/contributing/error-handling.md`](docs/contributing/error-handling.md) for error handling principles. Prefer explicit enum errors over anyhow for better pattern matching and user experience. Make errors clear, include sufficient context for traceability, and ensure they are actionable with specific fix instructions.
100129

101-
8. **When producing any output to users** (CRITICAL for architecture): Read [`docs/contributing/output-handling.md`](docs/contributing/output-handling.md) for output handling conventions. **NEVER use `println!`, `eprintln!`, `print!`, `eprint!`, or direct access to `std::io::stdout()`/`std::io::stderr()`**. Always use `UserOutput` methods through the execution context. This ensures testability, consistent formatting, proper channel routing (stdout vs stderr), verbosity control, and theme support. Example: `ctx.user_output().lock().borrow_mut().progress("Processing...")` instead of `println!("Processing...")`.
130+
9. **When producing any output to users** (CRITICAL for architecture): Read [`docs/contributing/output-handling.md`](docs/contributing/output-handling.md) for output handling conventions. **NEVER use `println!`, `eprintln!`, `print!`, `eprint!`, or direct access to `std::io::stdout()`/`std::io::stderr()`**. Always use `UserOutput` methods through the execution context. This ensures testability, consistent formatting, proper channel routing (stdout vs stderr), verbosity control, and theme support. Example: `ctx.user_output().lock().borrow_mut().progress("Processing...")` instead of `println!("Processing...")`.
102131

103-
9. **Understanding expected errors**: Read [`docs/contributing/known-issues.md`](docs/contributing/known-issues.md) for known issues and expected behaviors. Some errors that appear red in E2E test output (like SSH host key warnings) are normal and expected - not actual failures.
132+
10. **Understanding expected errors**: Read [`docs/contributing/known-issues.md`](docs/contributing/known-issues.md) for known issues and expected behaviors. Some errors that appear red in E2E test output (like SSH host key warnings) are normal and expected - not actual failures.
104133

105-
10. **Before making engineering decisions**: Document significant architectural or design decisions as Architectural Decision Records (ADRs) in `docs/decisions/`. Read [`docs/decisions/README.md`](docs/decisions/README.md) for the ADR template and guidelines. This ensures decisions are properly documented with context, rationale, and consequences for future reference.
134+
11. **Before making engineering decisions**: Document significant architectural or design decisions as Architectural Decision Records (ADRs) in `docs/decisions/`. Read [`docs/decisions/README.md`](docs/decisions/README.md) for the ADR template and guidelines. This ensures decisions are properly documented with context, rationale, and consequences for future reference.
106135

107-
11. **When organizing code within modules**: Follow the module organization conventions in [`docs/contributing/module-organization.md`](docs/contributing/module-organization.md). Use top-down organization with public items first, high-level abstractions before low-level details, and important responsibilities before secondary concerns like error types.
136+
12. **When organizing code within modules**: Follow the module organization conventions in [`docs/contributing/module-organization.md`](docs/contributing/module-organization.md). Use top-down organization with public items first, high-level abstractions before low-level details, and important responsibilities before secondary concerns like error types.
108137

109-
12. **When writing Rust imports** (CRITICAL for code style): Follow the import conventions in [`docs/contributing/module-organization.md`](docs/contributing/module-organization.md). Two essential rules:
138+
13. **When writing Rust imports** (CRITICAL for code style): Follow the import conventions in [`docs/contributing/module-organization.md`](docs/contributing/module-organization.md). Two essential rules:
110139

111140
- **Imports Always First**: Keep all imports at the top of the file, organized in groups (std → external crates → internal crate).
112141
- **Prefer Imports Over Full Paths**: Always import types and use short names (e.g., `Arc<UserOutput>`) rather than fully-qualified paths. Never use long paths like `std::sync::Arc<crate::presentation::views::UserOutput>` in regular code - only use full paths when disambiguating naming conflicts.
113142

114-
13. **When writing Markdown documentation**: Be aware of GitHub Flavored Markdown's automatic linking behavior. Read [`docs/contributing/github-markdown-pitfalls.md`](docs/contributing/github-markdown-pitfalls.md) for critical patterns to avoid. **NEVER use hash-number patterns for enumeration or step numbering** - this creates unintended links to GitHub issues/PRs. Use ordered lists or alternative formats instead.
143+
14. **When writing Markdown documentation**: Be aware of GitHub Flavored Markdown's automatic linking behavior. Read [`docs/contributing/github-markdown-pitfalls.md`](docs/contributing/github-markdown-pitfalls.md) for critical patterns to avoid. **NEVER use hash-number patterns for enumeration or step numbering** - this creates unintended links to GitHub issues/PRs. Use ordered lists or alternative formats instead.
115144

116-
14. **When creating new environment variables**: Read [`docs/contributing/environment-variables-naming.md`](docs/contributing/environment-variables-naming.md) for comprehensive guidance on naming conventions (condition-based vs action-based), decision frameworks, and best practices. Also review [`docs/decisions/environment-variable-prefix.md`](docs/decisions/environment-variable-prefix.md) to ensure all project environment variables use the `TORRUST_TD_` prefix for proper namespacing and avoiding conflicts with system or user variables.
145+
15. **When creating new environment variables**: Read [`docs/contributing/environment-variables-naming.md`](docs/contributing/environment-variables-naming.md) for comprehensive guidance on naming conventions (condition-based vs action-based), decision frameworks, and best practices. Also review [`docs/decisions/environment-variable-prefix.md`](docs/decisions/environment-variable-prefix.md) to ensure all project environment variables use the `TORRUST_TD_` prefix for proper namespacing and avoiding conflicts with system or user variables.
117146

118-
15. **When adding new templates**: Read [`docs/technical/template-system-architecture.md`](docs/technical/template-system-architecture.md) to understand the Project Generator pattern. The `templates/` directory contains source templates. Dynamic templates (`.tera`) are automatically processed, but static files must be explicitly registered in their respective `ProjectGenerator` to be copied to the build directory.
147+
16. **When adding new templates**: Read [`docs/technical/template-system-architecture.md`](docs/technical/template-system-architecture.md) to understand the Project Generator pattern. The `templates/` directory contains source templates. Dynamic templates (`.tera`) are automatically processed, but static files must be explicitly registered in their respective `ProjectGenerator` to be copied to the build directory.
119148

120-
16. **When writing unit tests** (CRITICAL for test quality): Read [`docs/contributing/testing/unit-testing.md`](docs/contributing/testing/unit-testing.md) and follow the behavior-driven naming convention. **NEVER use the `test_` prefix** for test function names. Always use the `it_should_{expected_behavior}_when_{condition}` or `it_should_{expected_behavior}_given_{state}` pattern. This ensures tests clearly document the behavior being validated and the conditions under which it occurs. Example: `it_should_return_error_when_username_is_invalid()` instead of `test_invalid_username()`. Test names should follow the three-part structure (What-When-Then) and be descriptive enough that the test's purpose is clear without reading the code.
149+
17. **When writing unit tests** (CRITICAL for test quality): Read [`docs/contributing/testing/unit-testing.md`](docs/contributing/testing/unit-testing.md) and follow the behavior-driven naming convention. **NEVER use the `test_` prefix** for test function names. Always use the `it_should_{expected_behavior}_when_{condition}` or `it_should_{expected_behavior}_given_{state}` pattern. This ensures tests clearly document the behavior being validated and the conditions under which it occurs. Example: `it_should_return_error_when_username_is_invalid()` instead of `test_invalid_username()`. Test names should follow the three-part structure (What-When-Then) and be descriptive enough that the test's purpose is clear without reading the code.
121150

122151
## 🧪 Build & Test
123152

0 commit comments

Comments
 (0)