Merge #235: Implement MySQL support for tracker deployment (#232)

cb591c5 docs: [#232] update manual testing guide with realistic HTTP tracker testing (Jose Celano)
4d53c20 refactor: [#232] Split EnvContext into service-specific types (Jose Celano)
dc98108 refactor: [#232] Reorganize external tool issues under new external-issues folder (Jose Celano)
363a70a feat: [#232] Implement MySQL support for tracker deployment with critical service dependency fix (Jose Celano)
6f38ffd feat: [#232] implement MySQL database configuration in tracker template (Jose Celano)
d4c86c4 feat: [#232] implement MySQL environment variable injection pattern (Jose Celano)
9a77348 feat: [#232] convert docker-compose template to dynamic Tera with conditional MySQL support (Jose Celano)

Pull request description:

  ## Overview

  This PR implements full MySQL database support for Torrust Tracker deployment, completing issue #232. The implementation extends the docker-compose stack with MySQL service and environment-driven configuration while maintaining backward compatibility with SQLite (default).

  ## Critical Fix: Service Dependencies

  **Most Important Change**: Added `depends_on` with `condition: service_healthy` to the tracker service in docker-compose template. This prevents startup race conditions and "Connection refused" errors by ensuring MySQL is fully healthy before the tracker container starts.

  **Impact**:
  - ❌ **Before**: Tracker attempts MySQL connection immediately → Connection refused errors → ~5 retries → eventual success or crash
  - ✅ **After**: Tracker waits for MySQL healthcheck → Clean startup → No connection errors

  **Trade-off**: Increases startup time by ~6 seconds (from ~19s to ~25s) but ensures reliable initialization.

  ## Changes Summary

  ### Templates

  1. **docker-compose.yml.tera**:
     - Added `depends_on` with `condition: service_healthy` for tracker service (CRITICAL)
     - Added standardized `TORRUST_TRACKER_CONFIG_OVERRIDE_CORE__DATABASE__DRIVER` environment variable
     - Removed hardcoded `sqlite3` value, now uses environment variable injection

  2. **.env.tera**:
     - Standardized variable: `TORRUST_TRACKER_CONFIG_OVERRIDE_CORE__DATABASE__DRIVER` (replaces legacy `DATABASE_DRIVER`)
     - Added section headers: "Tracker Service Configuration" and "MySQL Service Configuration"
     - Changed conditional from `{% if mysql_root_password %}` to `{% if database_driver == "mysql" %}` (explicit and correct)
     - Added explanatory comments about entrypoint script requirements

  ### Code

  3. **EnvContext** (`src/infrastructure/templating/docker_compose/template/wrappers/env/context.rs`):
     - Added `database_driver` field (controls template conditionals)
     - `new()` method sets `"sqlite3"` (default)
     - `new_with_mysql()` method sets `"mysql"`

  ### Documentation

  4. **Tracker Issues** (`docs/tracker-issues/`):
     - Created README explaining folder purpose and scope
     - Created `database-driver-double-specification.md` documenting tracker entrypoint behavior
     - Documents that tracker requires driver in both config file AND environment variable
     - References tracker entrypoint script source code

  5. **Manual Testing Guide** (`docs/e2e-testing/manual-testing-mysql.md`):
     - Complete end-to-end MySQL deployment workflow
     - Actual command outputs from successful test run
     - Corrected VM file locations (`/opt/torrust`)
     - Added debugging section (`data/logs/log.txt` usage)
     - Documented `depends_on` fix importance
     - Deep verification sections for MySQL, tracker, and HTTP API

  6. **AGENTS.md**:
     - Added Rule 1: CRITICAL explanation of `envs/` vs `data/` directories distinction
     - Most frequently violated rule - clearly separates user input from application state
     - Renumbered subsequent rules (2-17)

  ## Environment Variable Standardization

  The tracker container's `develop` branch requires:
  ```
  TORRUST_TRACKER_CONFIG_OVERRIDE_CORE__DATABASE__DRIVER
  ```

  This follows the `TORRUST_TRACKER_CONFIG_OVERRIDE_*` pattern used by the tracker's entrypoint script. The entrypoint **requires** this variable to be set, otherwise the container fails to start.

  **Note**: Legacy container versions may have used `DATABASE_DRIVER`, but this is no longer supported in current images.

  ## Testing

  **All tests passing**:
  - ✅ Unit tests: 1469 passed
  - ✅ Integration tests: 40 passed
  - ✅ E2E infrastructure lifecycle: PASSED
  - ✅ E2E deployment workflow: PASSED
  - ✅ Manual MySQL E2E test: Complete and successful
  - ✅ Pre-commit checks: ALL PASSED

  **Manual test verified**:
  - ✅ No startup race conditions
  - ✅ No "Connection refused" errors
  - ✅ MySQL tables created correctly
  - ✅ Tracker HTTP API responding
  - ✅ Both containers healthy
  - ✅ Environment variables correctly injected
  - ✅ Depends_on with healthcheck working as expected

  ## Backward Compatibility

  - ✅ SQLite remains the default driver (`database_driver = "sqlite3"`)
  - ✅ MySQL slice is opt-in via environment configuration
  - ✅ Existing SQLite deployments unaffected

  ## Implementation Details

  This implementation follows ADRs:
  - Environment Variable Injection in Docker Compose
  - Database Configuration Structure in Templates
  - Template System Architecture

  Closes #232

ACKs for top commit:
  josecelano:
    ACK cb591c5

Tree-SHA512: faf5b61995acfb782fe4ea0c860f11d37c124e76ed3f4fa2fb2cd87debb2630c4d239ddd694058d449d0d3a8b0084254dc35d0a4305e0f23d7df4b6d2b707a67

Author: josecelano

AGENTS.md

@@ -69,16 +69,45 @@ These principles should guide all development decisions, code reviews, and featu
 
 ## 🔧 Essential Rules
 
-1. **Before placing code in DDD layers**: Read [`docs/contributing/ddd-layer-placement.md`](docs/contributing/ddd-layer-placement.md) for comprehensive guidance on which code belongs in which layer (Domain, Application, Infrastructure, Presentation). This guide includes rules, red flags, examples, and a decision flowchart to help you make the right architectural decisions.
+1. **CRITICAL - Understanding `envs/` vs `data/` directories** (⚠️ **MOST FREQUENTLY VIOLATED RULE**):
 
-2. **Before creating branches**: Read [`docs/contributing/branching.md`](docs/contributing/branching.md) for naming conventions (`{issue-number}-{short-description}`)
+   **TWO COMPLETELY DIFFERENT FILE PURPOSES:**
 
-3. **Before committing**: Read [`docs/contributing/commit-process.md`](docs/contributing/commit-process.md) for conventional commits
+   - **`envs/` directory** - User Environment Configurations (USER INPUT):
+
+     - Purpose: User-created configuration files for environment creation
+     - Format: Environment creation schema (see `envs/environment-schema.json`)
+     - Contains: Provider config, SSH credentials, tracker settings, database config
+     - Usage: Passed to `create environment --env-file envs/your-config.json`
+     - Example: `envs/manual-test-mysql.json`
+     - Version Control: Gitignored (user-specific)
+     - **Rule**: You MAY create/edit files here as part of documentation or testing
+
+   - **`data/` directory** - Internal Application State (APPLICATION MANAGED):
+     - Purpose: Serialized Rust structs representing deployment state machine
+     - Format: Rust `Environment<State>` domain model serialization
+     - Contains: State transitions, runtime outputs, trace IDs, timestamps
+     - Usage: Internal state management, read-only inspection
+     - Example: `data/manual-test-mysql/environment.json` (NOT the same as `envs/manual-test-mysql.json`)
+     - Version Control: Gitignored (runtime-generated)
+     - **Rule**: You MUST NEVER create/edit files here - READ ONLY for debugging/verification
+
+   **NEVER CONFUSE THESE TWO!** When documenting or testing:
+
+   - User creates config in `envs/your-env.json`
+   - Application manages state in `data/your-env/environment.json`
+   - These are completely different JSON structures with different purposes
+
+2. **Before placing code in DDD layers**: Read [`docs/contributing/ddd-layer-placement.md`](docs/contributing/ddd-layer-placement.md) for comprehensive guidance on which code belongs in which layer (Domain, Application, Infrastructure, Presentation). This guide includes rules, red flags, examples, and a decision flowchart to help you make the right architectural decisions.
+
+3. **Before creating branches**: Read [`docs/contributing/branching.md`](docs/contributing/branching.md) for naming conventions (`{issue-number}-{short-description}`)
+
+4. **Before committing**: Read [`docs/contributing/commit-process.md`](docs/contributing/commit-process.md) for conventional commits
 
    - **With issue branch**: `{type}: [#{issue}] {description}` (when branch name starts with `{issue-number}-`)
    - **Without issue branch**: `{type}: {description}` (when working on main or branch without issue number prefix)
 
-4. **Before committing**: Always run the pre-commit verification script - all checks must pass before staging files or creating commits, regardless of the tool or method used:
+5. **Before committing**: Always run the pre-commit verification script - all checks must pass before staging files or creating commits, regardless of the tool or method used:
 
    ```bash
    ./scripts/pre-commit.sh
@@ -92,32 +121,32 @@ These principles should guide all development decisions, code reviews, and featu
    - Git clients: GitHub Desktop, GitKraken, etc.
    - CI/CD: Any automated commits or merges
 
-5. **Before working with Tera templates**: Read [`docs/contributing/templates.md`](docs/contributing/templates.md) for correct variable syntax - use `{{ variable }}` not `{ { variable } }`. Tera template files have the `.tera` extension.
+6. **Before working with Tera templates**: Read [`docs/contributing/templates.md`](docs/contributing/templates.md) for correct variable syntax - use `{{ variable }}` not `{ { variable } }`. Tera template files have the `.tera` extension.
 
-6. **When adding new Ansible playbooks**: Read [`docs/contributing/templates.md`](docs/contributing/templates.md) for the complete guide. **CRITICAL**: Static playbooks (without `.tera` extension) must be registered in `src/infrastructure/external_tools/ansible/template/renderer/project_generator.rs` in the `copy_static_templates` method, otherwise they won't be copied to the build directory and Ansible will fail with "playbook not found" error.
+7. **When adding new Ansible playbooks**: Read [`docs/contributing/templates.md`](docs/contributing/templates.md) for the complete guide. **CRITICAL**: Static playbooks (without `.tera` extension) must be registered in `src/infrastructure/external_tools/ansible/template/renderer/project_generator.rs` in the `copy_static_templates` method, otherwise they won't be copied to the build directory and Ansible will fail with "playbook not found" error.
 
-7. **When handling errors in code**: Read [`docs/contributing/error-handling.md`](docs/contributing/error-handling.md) for error handling principles. Prefer explicit enum errors over anyhow for better pattern matching and user experience. Make errors clear, include sufficient context for traceability, and ensure they are actionable with specific fix instructions.
+8. **When handling errors in code**: Read [`docs/contributing/error-handling.md`](docs/contributing/error-handling.md) for error handling principles. Prefer explicit enum errors over anyhow for better pattern matching and user experience. Make errors clear, include sufficient context for traceability, and ensure they are actionable with specific fix instructions.
 
-8. **When producing any output to users** (CRITICAL for architecture): Read [`docs/contributing/output-handling.md`](docs/contributing/output-handling.md) for output handling conventions. **NEVER use `println!`, `eprintln!`, `print!`, `eprint!`, or direct access to `std::io::stdout()`/`std::io::stderr()`**. Always use `UserOutput` methods through the execution context. This ensures testability, consistent formatting, proper channel routing (stdout vs stderr), verbosity control, and theme support. Example: `ctx.user_output().lock().borrow_mut().progress("Processing...")` instead of `println!("Processing...")`.
+9. **When producing any output to users** (CRITICAL for architecture): Read [`docs/contributing/output-handling.md`](docs/contributing/output-handling.md) for output handling conventions. **NEVER use `println!`, `eprintln!`, `print!`, `eprint!`, or direct access to `std::io::stdout()`/`std::io::stderr()`**. Always use `UserOutput` methods through the execution context. This ensures testability, consistent formatting, proper channel routing (stdout vs stderr), verbosity control, and theme support. Example: `ctx.user_output().lock().borrow_mut().progress("Processing...")` instead of `println!("Processing...")`.
 
-9. **Understanding expected errors**: Read [`docs/contributing/known-issues.md`](docs/contributing/known-issues.md) for known issues and expected behaviors. Some errors that appear red in E2E test output (like SSH host key warnings) are normal and expected - not actual failures.
+10. **Understanding expected errors**: Read [`docs/contributing/known-issues.md`](docs/contributing/known-issues.md) for known issues and expected behaviors. Some errors that appear red in E2E test output (like SSH host key warnings) are normal and expected - not actual failures.
 
-10. **Before making engineering decisions**: Document significant architectural or design decisions as Architectural Decision Records (ADRs) in `docs/decisions/`. Read [`docs/decisions/README.md`](docs/decisions/README.md) for the ADR template and guidelines. This ensures decisions are properly documented with context, rationale, and consequences for future reference.
+11. **Before making engineering decisions**: Document significant architectural or design decisions as Architectural Decision Records (ADRs) in `docs/decisions/`. Read [`docs/decisions/README.md`](docs/decisions/README.md) for the ADR template and guidelines. This ensures decisions are properly documented with context, rationale, and consequences for future reference.
 
-11. **When organizing code within modules**: Follow the module organization conventions in [`docs/contributing/module-organization.md`](docs/contributing/module-organization.md). Use top-down organization with public items first, high-level abstractions before low-level details, and important responsibilities before secondary concerns like error types.
+12. **When organizing code within modules**: Follow the module organization conventions in [`docs/contributing/module-organization.md`](docs/contributing/module-organization.md). Use top-down organization with public items first, high-level abstractions before low-level details, and important responsibilities before secondary concerns like error types.
 
-12. **When writing Rust imports** (CRITICAL for code style): Follow the import conventions in [`docs/contributing/module-organization.md`](docs/contributing/module-organization.md). Two essential rules:
+13. **When writing Rust imports** (CRITICAL for code style): Follow the import conventions in [`docs/contributing/module-organization.md`](docs/contributing/module-organization.md). Two essential rules:
 
     - **Imports Always First**: Keep all imports at the top of the file, organized in groups (std → external crates → internal crate).
     - **Prefer Imports Over Full Paths**: Always import types and use short names (e.g., `Arc<UserOutput>`) rather than fully-qualified paths. Never use long paths like `std::sync::Arc<crate::presentation::views::UserOutput>` in regular code - only use full paths when disambiguating naming conflicts.
 
-13. **When writing Markdown documentation**: Be aware of GitHub Flavored Markdown's automatic linking behavior. Read [`docs/contributing/github-markdown-pitfalls.md`](docs/contributing/github-markdown-pitfalls.md) for critical patterns to avoid. **NEVER use hash-number patterns for enumeration or step numbering** - this creates unintended links to GitHub issues/PRs. Use ordered lists or alternative formats instead.
+14. **When writing Markdown documentation**: Be aware of GitHub Flavored Markdown's automatic linking behavior. Read [`docs/contributing/github-markdown-pitfalls.md`](docs/contributing/github-markdown-pitfalls.md) for critical patterns to avoid. **NEVER use hash-number patterns for enumeration or step numbering** - this creates unintended links to GitHub issues/PRs. Use ordered lists or alternative formats instead.
 
-14. **When creating new environment variables**: Read [`docs/contributing/environment-variables-naming.md`](docs/contributing/environment-variables-naming.md) for comprehensive guidance on naming conventions (condition-based vs action-based), decision frameworks, and best practices. Also review [`docs/decisions/environment-variable-prefix.md`](docs/decisions/environment-variable-prefix.md) to ensure all project environment variables use the `TORRUST_TD_` prefix for proper namespacing and avoiding conflicts with system or user variables.
+15. **When creating new environment variables**: Read [`docs/contributing/environment-variables-naming.md`](docs/contributing/environment-variables-naming.md) for comprehensive guidance on naming conventions (condition-based vs action-based), decision frameworks, and best practices. Also review [`docs/decisions/environment-variable-prefix.md`](docs/decisions/environment-variable-prefix.md) to ensure all project environment variables use the `TORRUST_TD_` prefix for proper namespacing and avoiding conflicts with system or user variables.
 
-15. **When adding new templates**: Read [`docs/technical/template-system-architecture.md`](docs/technical/template-system-architecture.md) to understand the Project Generator pattern. The `templates/` directory contains source templates. Dynamic templates (`.tera`) are automatically processed, but static files must be explicitly registered in their respective `ProjectGenerator` to be copied to the build directory.
+16. **When adding new templates**: Read [`docs/technical/template-system-architecture.md`](docs/technical/template-system-architecture.md) to understand the Project Generator pattern. The `templates/` directory contains source templates. Dynamic templates (`.tera`) are automatically processed, but static files must be explicitly registered in their respective `ProjectGenerator` to be copied to the build directory.
 
-16. **When writing unit tests** (CRITICAL for test quality): Read [`docs/contributing/testing/unit-testing.md`](docs/contributing/testing/unit-testing.md) and follow the behavior-driven naming convention. **NEVER use the `test_` prefix** for test function names. Always use the `it_should_{expected_behavior}_when_{condition}` or `it_should_{expected_behavior}_given_{state}` pattern. This ensures tests clearly document the behavior being validated and the conditions under which it occurs. Example: `it_should_return_error_when_username_is_invalid()` instead of `test_invalid_username()`. Test names should follow the three-part structure (What-When-Then) and be descriptive enough that the test's purpose is clear without reading the code.
+17. **When writing unit tests** (CRITICAL for test quality): Read [`docs/contributing/testing/unit-testing.md`](docs/contributing/testing/unit-testing.md) and follow the behavior-driven naming convention. **NEVER use the `test_` prefix** for test function names. Always use the `it_should_{expected_behavior}_when_{condition}` or `it_should_{expected_behavior}_given_{state}` pattern. This ensures tests clearly document the behavior being validated and the conditions under which it occurs. Example: `it_should_return_error_when_username_is_invalid()` instead of `test_invalid_username()`. Test names should follow the three-part structure (What-When-Then) and be descriptive enough that the test's purpose is clear without reading the code.
 
 ## 🧪 Build & Test
 
@@ -191,7 +220,7 @@ The project has comprehensive documentation organized in the [`docs/`](docs/) di
 | Write unit tests          | [`docs/contributing/testing/unit-testing.md`](docs/contributing/testing/unit-testing.md) |
 | Understand a decision     | [`docs/decisions/README.md`](docs/decisions/README.md)                                   |
 | Plan a new feature        | [`docs/features/README.md`](docs/features/README.md)                                     |
-| Fix a CI issue            | [`docs/github-actions-issues/README.md`](docs/github-actions-issues/README.md)           |
+| Fix external tool issues  | [`docs/external-issues/README.md`](docs/external-issues/README.md)                       |
 | Work with templates       | [`docs/contributing/templates.md`](docs/contributing/templates.md)                       |
 | Handle errors properly    | [`docs/contributing/error-handling.md`](docs/contributing/error-handling.md)             |
 | Handle output properly    | [`docs/contributing/output-handling.md`](docs/contributing/output-handling.md)           |

docs/README.md

@@ -36,7 +36,7 @@ docs/
 ├── refactors/            🔄 Refactoring plans and tracking
 ├── implementation-plans/ 📝 Step-by-step plans for complex changes
 ├── issues/               📋 Issue templates and specifications
-├── github-actions-issues/ ⚠️ CI/CD troubleshooting
+├── external-issues/      ⚠️ External tool issues (GitHub Actions, Tracker)
 └── analysis/             📊 Code analysis (presentation layer structure)
 ```
 
@@ -82,7 +82,7 @@ docs/
 | Write unit tests          | [`contributing/testing/unit-testing.md`](contributing/testing/unit-testing.md) |
 | Understand a decision     | [`decisions/README.md`](decisions/README.md)                                   |
 | Plan a new feature        | [`features/README.md`](features/README.md)                                     |
-| Fix a CI issue            | [`github-actions-issues/README.md`](github-actions-issues/README.md)           |
+| Fix external tool issues  | [`external-issues/README.md`](external-issues/README.md)                       |
 | Work with templates       | [`contributing/templates.md`](contributing/templates.md)                       |
 | Handle errors properly    | [`contributing/error-handling.md`](contributing/error-handling.md)             |
 | Handle output properly    | [`contributing/output-handling.md`](contributing/output-handling.md)           |
@@ -246,9 +246,11 @@ Ansible testing strategy, Docker vs LXD, E2E testing, UX patterns, MVVM analysis
 
 - [`issues/`](issues/) - Templates for epics, issues, and specifications
 
-**CI/CD Troubleshooting:**
+**External Tool Issues:**
 
-- [`github-actions-issues/README.md`](github-actions-issues/README.md) - GitHub Actions runner issues
+- [`external-issues/README.md`](external-issues/README.md) - Issues with external tools (GitHub Actions, Tracker)
+- [`external-issues/github-actions/`](external-issues/github-actions/) - GitHub Actions CI/CD issues
+- [`external-issues/tracker/`](external-issues/tracker/) - Torrust Tracker issues
 
 **Code Analysis:**