Merge #221: feat: [#220] Tracker Slice - Release and Run Commands

cff2e0d docs: [#220] update documentation with real command outputs and state management warnings (Jose Celano)
aacdcd6 chore: [#220] remove completed implementation plan (Jose Celano)
fbfeb65 docs: [#220] mark Phase 3 complete - infrastructure reorganization (Jose Celano)
32d31d7 docs: [#220] add ADR for infrastructure module organization (Jose Celano)
901a817 refactor: [#220] update imports to use external_validators path (Jose Celano)
e86fe48 refactor: [#220] update infrastructure exports for external validators (Jose Celano)
dbe2876 refactor: [#220] move running_services validator to external_validators (Jose Celano)
d6a5c3b refactor: [#220] create external_validators module structure (Jose Celano)
223b83d docs: [#220] mark Step 4.4 complete - all validation passed (Jose Celano)
0c14fd6 docs: [#220] update architecture docs for running services validator (Jose Celano)
20afd81 docs: [#220] update run command documentation (Jose Celano)
fbb300b docs: [#220] mark Phase 2 as complete in implementation plan (Jose Celano)
3be2c3a step: [#220] add unit tests for multiple HTTP tracker support (Jose Celano)
2ca1e04 step: [#220] update RunningServicesValidator for multiple HTTP trackers (Jose Celano)
7312107 docs: [#220] mark Phase 1 complete - port 0 validation (Jose Celano)
ff04fe7 step: [#220] add port 0 validation tests (Jose Celano)
877a550 docs: [#220] mark Step 1.3 complete (Jose Celano)
2d8e8f4 step: [#220] add port 0 validation in conversion methods (Jose Celano)
b9a739b docs: [#220] mark Step 1.2 complete (Jose Celano)
1cbb942 step: [#220] add DynamicPortNotSupported error variant (Jose Celano)
27557f0 docs: [#220] add ADR for port zero not supported in bind addresses (Jose Celano)
5cc10f5 docs: [#220] mark Phase 0 complete - architecture fix done (Jose Celano)
01be3af step: [#220] update EnvironmentCreationConfig to use TrackerSection (Jose Celano)
de1fdb4 step: [#220] update domain types to use SocketAddr (Jose Celano)
b985a09 step: [#220] implement TrackerSection DTO (Jose Celano)
3234f16 step: [#220] implement TrackerCoreSection DTO (Jose Celano)
a6741cf step: [#220] implement HttpApiSection DTO (Jose Celano)
701cc28 step: [#220] implement HttpTrackerSection DTO (Jose Celano)
6c86c44 step: [#220] implement UdpTrackerSection DTO (Jose Celano)
d70ab59 step: [#220] create tracker DTO module structure (Jose Celano)
b908a24 docs: [#220] add implementation plan for test command architecture improvements (Jose Celano)
a39ba13 feat: [#220] enhance test command with external health checks (Jose Celano)
8e0e096 fix: [#220] correct netstat flags and add tulnp to project dictionary (Jose Celano)
5592ff7 docs: [#220] add comprehensive documentation for release and run commands (Jose Celano)
de5d88b docs: update workflow badges to match renamed workflow files (Jose Celano)
2827ab8 docs: add cross-references between template documentation files (Jose Celano)
9d64e9b feat: [#220] make HTTP API bind address configurable in tracker template (Jose Celano)
893ac72 docs: [#220] reorganize E2E testing documentation into structured directory (Jose Celano)
c7b27ce docs: [#220] document decision to use single Docker image for sequential E2E testing (Jose Celano)
61a712d refactor: [#220] update cargo aliases for renamed E2E test binaries (Jose Celano)
1e957ba refactor: [#220] rename E2E test binaries to better reflect their purpose (Jose Celano)
13cca4c refactor: [#220] make E2eConfigEnvironment single source of truth for E2E test configuration (Jose Celano)
638561e refactor: [#220] remove unused update_environment_ssh_port function (Jose Celano)
862b95d refactor: [#220] remove unused with_working_dir functions (Jose Celano)
e22e601 refactor: [#220] consolidate create_test_environment_config into shared function (Jose Celano)
e704f27 refactor: [#220] organize E2E tests into module structure (Jose Celano)
b7e3527 refactor: [#220] return JSON string directly from config helper (Jose Celano)
67efd7c feat: [#220] add default tracker configuration to environment template (Jose Celano)
430c52e docs: [#221] add ADR for register SSH port override and update E2E testing guide (Jose Celano)
f16d6cd feat: [#221] add optional --ssh-port argument to register command (Jose Celano)
83254bb refactor: [#221] separate E2E configuration from runtime state with bridge networking (Jose Celano)
35a256f refactor: [#220] simplify RunningServicesValidator to external-only validation with parameterized ports (Jose Celano)
9453e80 feat: [#220] add .env file check to run-compose-services playbook (Jose Celano)
b947d1b refactor: [#220] remove default UDP tracker on port 6868 (Jose Celano)
3f6721b feat: [#220] wire tracker config through provision workflow (Phase 7 complete) (Jose Celano)
3d397c8 chore: delete mmanual test file (Jose Celano)
9c7daac docs: [#220] exclude generated folders from taplo and update linting docs (Jose Celano)
8294d9e fix: [#220] clippy doc_markdown and rustdoc warnings for Phase 7 (Jose Celano)
6939553 feat: [#220] Phase 7 infrastructure for tracker firewall configuration (partial) (Jose Celano)
52d7c2a feat: [#220] make tracker configuration mandatory in environment config (Jose Celano)
748951d docs: [#220] mark Phase 5 complete in implementation progress tracking (Jose Celano)
59e3762 feat: [#220] replace nginx demo with Torrust Tracker in docker-compose (Phase 5) (Jose Celano)
4957b01 docs: [#220] mark Phase 4 complete in implementation progress tracking (Jose Celano)
659e407 refactor: [#220] reorganize tracker template structure to match ansible pattern (Jose Celano)
aa1c494 feat: [#220] add docker compose .env template with project generator pattern (Jose Celano)
794ef06 feat: [#220] implement Phase 2 - initialize tracker SQLite database (Jose Celano)
1b83dfb docs: [#220] add Phase 1 manual E2E test results (Jose Celano)
311d987 feat: [#220] implement Phase 1 - create tracker storage directories (Jose Celano)
dff5a09 docs: [#220] mark Phase 0 as completed (Jose Celano)
2d5625c refactor: [#220] rename external_tools module to templating (Jose Celano)

Pull request description:

  ## Summary

  This PR implements the complete **Tracker Slice** for Release and Run commands, replacing the demo nginx service with a fully functional Torrust Tracker deployment. The implementation follows an incremental, testable approach with 8 phases, all of which have been successfully completed.

  ## What's Been Implemented

  ### ✅ All 8 Phases Complete

  **Phase 0: Module Refactoring (Completed)**
  - Renamed `src/infrastructure/external_tools/` → `src/infrastructure/templating/`
  - Better reflects the module's purpose (template rendering logic)

  **Phase 1: Storage Directory Creation (Completed)**
  - Ansible playbook creates tracker storage structure: `/opt/torrust/storage/tracker/{etc,lib/database,log}`
  - Integrated into ReleaseCommand workflow
  - Manual E2E test verified directory creation with correct ownership and permissions

  **Phase 2: SQLite Database Initialization (Completed)**
  - Ansible playbook initializes SQLite database file (`tracker.db`)
  - Empty database file created with proper permissions (0644, torrust:torrust)
  - Integrated into ReleaseCommand workflow after storage creation

  **Phase 3: Docker Compose Environment Variables (Completed)**
  - Implemented `.env` file generation using Project Generator pattern
  - Created `EnvContext`, `EnvTemplate`, and `EnvRenderer` components
  - Refactored to three-layer architecture (Wrapper → Renderer → Generator)
  - `.env` file deployed to `/opt/torrust/.env` with tracker configuration

  **Phase 4: Tracker Configuration Template (Completed)**
  - Created `templates/tracker/tracker.toml.tera` with Torrust Tracker configuration
  - Implemented `TrackerProjectGenerator` following Project Generator pattern
  - Created `RenderTrackerTemplatesStep` and `DeployTrackerConfigStep`
  - Configuration deployed to `/opt/torrust/storage/tracker/etc/tracker.toml`

  **Phase 5: Docker Compose Service Replacement (Completed)**
  - Replaced nginx demo service with Torrust Tracker in docker-compose
  - Configured UDP tracker ports (6969), HTTP tracker (7070), and HTTP API (1212)
  - Volume mounts for storage directories (lib, log, etc)
  - Environment variable integration for configuration

  **Phase 6: Environment Configuration Support (Completed)**
  - Added `TrackerConfig` schema to environment configuration
  - Users can configure: UDP trackers, HTTP trackers, private/public mode, API admin token
  - All templates dynamically populated from environment config
  - Template generation uses tracker configuration for variable substitution

  **Phase 7: Firewall Configuration (Completed)**
  - Ansible playbook configures UFW firewall for tracker ports
  - Opens UDP tracker ports, HTTP tracker ports, and HTTP API port
  - Port extraction from environment configuration
  - Integrated into ConfigureCommand workflow
  - Manual E2E test verified all firewall rules correctly applied

  **Phase 8: E2E Test Updates (Completed)**
  - Updated `RunningServicesValidator` for external-only validation
  - Checks tracker HTTP API health endpoint (`http://<vm-ip>:1212/api/health_check`)
  - Checks HTTP tracker health endpoint (`http://<vm-ip>:7070/api/health_check`)
  - Simplified to external validation (proves service AND firewall working)
  - Refactored validation code for better maintainability

  ### Architecture Improvements

  **Domain-Driven Design (DDD)**
  - Proper layer separation: Domain DTOs, Infrastructure templates, Application workflows
  - Created tracker DTO module with validation (port 0 not supported, documented in ADR)
  - Multiple HTTP tracker support with external validation for each instance

  **Infrastructure Module Organization**
  - Created `external_validators/` module for direct HTTP validation (no SSH required)
  - Better separation: `remote_actions/` (via SSH) vs `external_validators/` (direct HTTP)
  - Documented in ADR for future maintainability

  **Template System**
  - Consistent Project Generator pattern across all template modules
  - Three-layer architecture: Wrapper → Renderer → Generator
  - Static templates explicitly registered, dynamic templates with `.tera` extension

  ## Testing

  ### Manual E2E Testing
  All phases manually tested with complete verification:
  - ✅ Storage directories created with correct permissions
  - ✅ SQLite database initialized
  - ✅ `.env` file deployed with tracker configuration
  - ✅ `tracker.toml` configuration deployed
  - ✅ Tracker container running with all ports exposed
  - ✅ Firewall rules applied (UDP 6969, HTTP 7070, API 1212)
  - ✅ External HTTP health checks passing

  ### Automated E2E Testing
  - ✅ Full E2E test suite passing (complete workflow validation)
  - ✅ Infrastructure lifecycle tests passing (provisioning and destruction)
  - ✅ Deployment workflow tests passing (configuration, release, run)
  - ✅ All 1424 unit tests passing
  - ✅ All 40 integration tests passing
  - ✅ All pre-commit checks passing

  ### Real-World Validation
  - ✅ Tracker HTTP API accessible: `curl http://<vm-ip>:1212/api/health_check` returns HTTP 200
  - ✅ HTTP Tracker accessible: `curl http://<vm-ip>:7070/api/health_check` returns HTTP 200
  - ✅ All tracker ports open in firewall (verified with `sudo ufw status`)
  - ✅ Docker services running: `docker compose ps` shows tracker in "running" state

  ## Documentation Updates

  - ✅ Complete implementation plan with all 8 phases documented
  - ✅ Manual testing workflow guide with real command outputs
  - ✅ Architecture Decision Records (ADRs) for key design decisions
  - ✅ Updated quick-start guide with tracker deployment workflow
  - ✅ Updated manual E2E testing documentation

  ## Acceptance Criteria

  All acceptance criteria from issue #220 have been met:

  **Quality Checks**
  - ✅ Pre-commit checks pass
  - ✅ All linters passing (markdown, yaml, toml, cspell, clippy, rustfmt, shellcheck)
  - ✅ All unit, integration, and E2E tests passing

  **Functional Requirements**
  - ✅ Storage directories created on VM
  - ✅ SQLite database initialized
  - ✅ Docker compose `.env` file deployed
  - ✅ Tracker configuration file deployed
  - ✅ Docker compose service updated with tracker
  - ✅ Firewall configured for all tracker ports
  - ✅ Tracker container starts successfully
  - ✅ Tracker HTTP API responds to health checks
  - ✅ Tracker ports accessible externally

  **Configuration Requirements**
  - ✅ Environment config supports tracker configuration
  - ✅ Users can specify UDP/HTTP tracker instances
  - ✅ Private/public mode configurable
  - ✅ Database name configurable
  - ✅ API admin token configurable
  - ✅ Template variables correctly substituted

  **Architecture Requirements**
  - ✅ Tracker templates isolated in separate module
  - ✅ Project Generator pattern implemented
  - ✅ Static templates registered in renderer
  - ✅ No mixing of tracker and docker-compose templates

  **Testing Requirements**
  - ✅ All phases manually testable
  - ✅ E2E tests updated for tracker validation
  - ✅ Example environment files demonstrate configuration

  ## Related Documentation

  - Issue: #220
  - Parent Epic: #216
  - Implementation Plan: [docs/issues/220-tracker-slice-release-run-commands.md](https://github.com/torrust/torrust-tracker-deployer/blob/220-tracker-slice-release-run-commands/docs/issues/220-tracker-slice-release-run-commands.md)
  - Manual Testing Guide: [docs/e2e-testing/manual-testing.md](https://github.com/torrust/torrust-tracker-deployer/blob/220-tracker-slice-release-run-commands/docs/e2e-testing/manual-testing.md)
  - Quick Start Guide: [docs/user-guide/quick-start.md](https://github.com/torrust/torrust-tracker-deployer/blob/220-tracker-slice-release-run-commands/docs/user-guide/quick-start.md)

  ## Deployment Workflow

  The complete workflow for deploying a Torrust Tracker:

  ```bash
  # 1. Create environment from template
  cargo run -- create template --provider lxd > envs/my-tracker.json

  # 2. Edit environment configuration with your values

  # 3. Create environment
  cargo run -- create environment --env-file envs/my-tracker.json

  # 4. Provision VM
  cargo run -- provision my-tracker

  # 5. Configure VM (installs Docker, configures firewall, creates storage)
  cargo run -- configure my-tracker

  # 6. Release (deploys configurations and docker-compose)
  cargo run -- release my-tracker

  # 7. Run services
  cargo run -- run my-tracker

  # 8. Verify deployment
  VM_IP=$(cargo run -- show my-tracker | grep 'IP Address' | awk '{print $3}')
  curl http://$VM_IP:1212/api/health_check  # Tracker API
  curl http://$VM_IP:7070/api/health_check  # HTTP Tracker
  ```

  ## Next Steps

  This PR is ready to merge. All phases completed, all tests passing, all documentation updated.

  Future enhancements (not in this PR):
  - Additional tracker configuration options (database driver, intervals, logging)
  - Support for multiple tracker instances
  - Tracker management commands (status, logs, restart)
  - Integration with Index and Grafana services

  Closes #220

ACKs for top commit:
  josecelano:
    ACK cff2e0d

Tree-SHA512: 991c96c81321f38bc53ece9eff2be6406c23b42a7d591b5f0fd115f207048167decb96ad28149dd205fae45044c23a05e39eedc6048bb8b247fe502f557ce734

Author: josecelano

.cargo/config.toml

@@ -1,8 +1,8 @@
 [alias]
 lint = "run --bin linter all"
-e2e-full = "run --bin e2e-tests-full"
-e2e-provision = "run --bin e2e-provision-tests"
-e2e-config = "run --bin e2e-config-and-release-tests"
+e2e-complete = "run --bin e2e-complete-workflow-tests"
+e2e-infrastructure = "run --bin e2e-infrastructure-lifecycle-tests"
+e2e-deployment = "run --bin e2e-deployment-workflow-tests"
 cov = "llvm-cov"
 cov-check = "llvm-cov --all-features --workspace --fail-under-lines 70"
 cov-lcov = "llvm-cov --lcov --output-path=./.coverage/lcov.info"

.github/workflows/test-e2e-config.yml .github/workflows/test-e2e-deployment.yml.github/workflows/test-e2e-config.yml renamed to .github/workflows/test-e2e-deployment.yml

@@ -1,5 +1,5 @@
 ---
-name: E2E Configuration Tests
+name: E2E Deployment Workflow Tests
 
 # This workflow tests ONLY software configuration, release, and run phases
 # using Docker containers. It does NOT test infrastructure provisioning
@@ -22,7 +22,7 @@ on:
   workflow_dispatch: # Allow manual triggering
 
 jobs:
-  e2e-config-and-release-tests:
+  e2e-deployment-workflow-tests:
     runs-on: ubuntu-latest
     timeout-minutes: 45 # Timeout for complete configuration testing with software installation
 
@@ -62,16 +62,16 @@ jobs:
           chmod 600 fixtures/testing_rsa
           ls -la fixtures/testing_rsa
 
-      - name: Build E2E configuration and release tests binary
+      - name: Build E2E deployment workflow tests binary
         run: |
-          cargo build --bin e2e-config-and-release-tests --release
+          cargo build --bin e2e-deployment-workflow-tests --release
 
-      - name: Run E2E configuration and release test
+      - name: Run E2E deployment workflow test
         run: |
-          # Run the E2E configuration and release test with debug logging for better debugging
-          echo "🚀 Starting E2E configuration and release test at $(date)"
-          cargo run --bin e2e-config-and-release-tests
-          echo "✅ E2E configuration and release test completed at $(date)"
+          # Run the E2E deployment workflow test with debug logging for better debugging
+          echo "🚀 Starting E2E deployment workflow test at $(date)"
+          cargo run --bin e2e-deployment-workflow-tests
+          echo "✅ E2E deployment workflow test completed at $(date)"
         env:
           # Preserve environment variables for the E2E test
           RUST_LOG: debug
@@ -113,8 +113,8 @@ jobs:
           # properly (e.g., if the test was abruptly halted). Under normal circumstances,
           # the testcontainers crate should automatically clean up containers when tests finish.
 
-          # Clean up the specific container created for e2e-config tests
-          docker rm -f torrust-tracker-vm-e2e-config 2>/dev/null || echo "Container torrust-tracker-vm-e2e-config not found or already removed"
+          # Clean up the specific container created for e2e-deployment tests
+          docker rm -f torrust-tracker-vm-e2e-deployment 2>/dev/null || echo "Container torrust-tracker-vm-e2e-deployment not found or already removed"
 
           # Clean up any test images if needed
           docker images --filter "reference=torrust-provisioned-instance*" -q | xargs -r docker rmi -f || echo "No test images to remove"
@@ -126,9 +126,9 @@ jobs:
           docker ps -a
 
           echo "=== Test Summary ==="
-          echo "E2E configuration test workflow completed"
+          echo "E2E deployment workflow test completed"
           if [ "${{ job.status }}" = "success" ]; then
-            echo "✅ All configuration tests passed successfully"
+            echo "✅ All deployment workflow tests passed successfully"
           else
-            echo "❌ Some configuration tests failed - check logs above"
+            echo "❌ Some deployment workflow tests failed - check logs above"
           fi

.github/workflows/test-e2e-provision.yml …ub/workflows/test-e2e-infrastructure.yml.github/workflows/test-e2e-provision.yml renamed to .github/workflows/test-e2e-infrastructure.yml .github/workflows/test-e2e-provision.yml renamed to .github/workflows/test-e2e-infrastructure.yml

@@ -1,4 +1,4 @@
-name: E2E Provision and Destroy Tests
+name: E2E Infrastructure Lifecycle Tests
 
 # This workflow tests infrastructure provisioning and destruction (creating and destroying VMs/containers)
 # It does NOT test software configuration/installation to avoid GitHub Actions
@@ -18,7 +18,7 @@ on:
   workflow_dispatch: # Allow manual triggering
 
 jobs:
-  e2e-provision-tests:
+  e2e-infrastructure-lifecycle-tests:
     runs-on: ubuntu-latest
     timeout-minutes: 30 # Reduced timeout since we're not installing software
 
@@ -48,17 +48,17 @@ jobs:
           tofu version
           cargo --version
 
-      - name: Build E2E provision and destroy tests binary
+      - name: Build E2E infrastructure lifecycle tests binary
         run: |
-          cargo build --bin e2e-provision-and-destroy-tests --release
+          cargo build --bin e2e-infrastructure-lifecycle-tests --release
 
-      - name: Run E2E provision and destroy test
+      - name: Run E2E infrastructure lifecycle test
         run: |
-          # Run the E2E provision and destroy test with debug logging for better debugging
+          # Run the E2E infrastructure lifecycle test with debug logging for better debugging
           # Use sudo -E and preserve PATH to ensure cargo is accessible
-          echo "🚀 Starting E2E provision and destroy test at $(date)"
-          sudo -E env "PATH=$PATH" cargo run --bin e2e-provision-and-destroy-tests
-          echo "✅ E2E provision and destroy test completed at $(date)"
+          echo "🚀 Starting E2E infrastructure lifecycle test at $(date)"
+          sudo -E env "PATH=$PATH" cargo run --bin e2e-infrastructure-lifecycle-tests
+          echo "✅ E2E infrastructure lifecycle test completed at $(date)"
         env:
           # Preserve environment variables for the E2E test
           RUST_LOG: debug
@@ -68,18 +68,18 @@ jobs:
         run: |
           echo "=== Infrastructure Outputs ==="
           # Only check outputs if build directory still exists (it may be cleaned up by DestroyCommand)
-          if [ -d "build/e2e-provision/tofu/lxd" ]; then
-            cd build/e2e-provision/tofu/lxd
+          if [ -d "build/e2e-infrastructure/tofu/lxd" ]; then
+            cd build/e2e-infrastructure/tofu/lxd
             sudo -E tofu output || echo "No outputs available"
           else
             echo "Build directory not found (likely cleaned up by DestroyCommand)"
           fi
 
           echo "=== Container Status ==="
-          sudo lxc list torrust-tracker-vm-e2e-provision || echo "Container not found"
+          sudo lxc list torrust-tracker-vm-e2e-infrastructure || echo "Container not found"
 
           # Check if the container has an IP address before proceeding
-          sudo lxc info torrust-tracker-vm-e2e-provision || echo "Container info not available"
+          sudo lxc info torrust-tracker-vm-e2e-infrastructure || echo "Container info not available"
 
       - name: Debug information (on failure)
         if: failure()
@@ -88,8 +88,8 @@ jobs:
           sudo lxc list || echo "LXC list failed"
 
           echo "=== OpenTofu State ==="
-          if [ -d "build/e2e-provision/tofu/lxd" ]; then
-            cd build/e2e-provision/tofu/lxd
+          if [ -d "build/e2e-infrastructure/tofu/lxd" ]; then
+            cd build/e2e-infrastructure/tofu/lxd
             sudo -E tofu show || echo "No state to show"
           else
             echo "No OpenTofu state directory found"
@@ -108,18 +108,18 @@ jobs:
           echo "Test failed - attempting emergency cleanup..."
 
           # Try OpenTofu cleanup only if build directory still exists
-          if [ -d "build/e2e-provision/tofu/lxd" ]; then
+          if [ -d "build/e2e-infrastructure/tofu/lxd" ]; then
             echo "Found OpenTofu state directory, attempting tofu destroy..."
-            cd build/e2e-provision/tofu/lxd
+            cd build/e2e-infrastructure/tofu/lxd
             sudo -E tofu destroy -auto-approve || echo "Tofu destroy failed or nothing to destroy"
           else
             echo "No OpenTofu state directory found (likely cleaned up by DestroyCommand)"
           fi
 
           # Always attempt LXD cleanup (no working directory dependency)
           echo "Attempting LXD resource cleanup..."
-          sudo lxc delete torrust-tracker-vm-e2e-provision --force || echo "Container deletion failed or container doesn't exist"
-          sudo lxc profile delete torrust-profile-e2e-provision || echo "Profile deletion failed or profile doesn't exist"
+          sudo lxc delete torrust-tracker-vm-e2e-infrastructure --force || echo "Container deletion failed or container doesn't exist"
+          sudo lxc profile delete torrust-profile-e2e-infrastructure || echo "Profile deletion failed or profile doesn't exist"
 
       - name: Final verification
         if: always()
@@ -128,9 +128,9 @@ jobs:
           sudo lxc list
 
           echo "=== Test Summary ==="
-          echo "E2E provision and destroy test workflow completed"
+          echo "E2E infrastructure lifecycle test workflow completed"
           if [ "${{ job.status }}" = "success" ]; then
-            echo "✅ All provision and destroy tests passed successfully"
+            echo "✅ All infrastructure lifecycle tests passed successfully"
           else
-            echo "❌ Some provision and destroy tests failed - check logs above"
+            echo "❌ Some infrastructure lifecycle tests failed - check logs above"
           fi

.taplo.toml

@@ -1,5 +1,13 @@
 # Taplo configuration file for TOML formatting
 # Used by the "Even Better TOML" VS Code extension
+
+# Exclude generated and runtime folders from linting
+exclude = [
+  "build/**",
+  "data/**",
+  "envs/**",
+]
+
 [formatting]
 # Preserve blank lines that exist
 allowed_blank_lines = 1

AGENTS.md

@@ -130,10 +130,15 @@ These principles should guide all development decisions, code reviews, and featu
 - **Test**: `cargo test`
 - **Unit Tests**: When writing unit tests, follow conventions described in [`docs/contributing/testing/`](docs/contributing/testing/)
 - **E2E Tests**:
-  - `cargo run --bin e2e-tests-full` - Comprehensive tests (⚠️ **LOCAL ONLY** - cannot run on GitHub Actions due to network connectivity issues)
-  - `cargo run --bin e2e-provision-and-destroy-tests` - Infrastructure provisioning and destruction tests (GitHub runner-compatible)
-  - `cargo run --bin e2e-config-and-release-tests` - Software installation, configuration, release, and run workflow tests (GitHub runner-compatible)
-  - Pre-commit hook runs the split tests (`e2e-provision-and-destroy-tests` + `e2e-config-and-release-tests`) for GitHub Copilot compatibility
-  - See [`docs/e2e-testing.md`](docs/e2e-testing.md) for detailed information about CI limitations
+  - `cargo run --bin e2e-complete-workflow-tests` - Comprehensive tests (⚠️ **LOCAL ONLY** - cannot run on GitHub Actions due to network connectivity issues)
+  - `cargo run --bin e2e-infrastructure-lifecycle-tests` - Infrastructure provisioning and destruction tests (GitHub runner-compatible)
+  - `cargo run --bin e2e-deployment-workflow-tests` - Software installation, configuration, release, and run workflow tests (GitHub runner-compatible)
+  - Pre-commit hook runs the split tests (`e2e-infrastructure-lifecycle-tests` + `e2e-deployment-workflow-tests`) for GitHub Copilot compatibility
+  - See [`docs/e2e-testing/`](docs/e2e-testing/) for detailed information about CI limitations
+- **Manual E2E Testing**: For step-by-step manual testing with CLI commands, see [`docs/e2e-testing/manual-testing.md`](docs/e2e-testing/manual-testing.md). This guide covers:
+  - Complete manual test workflow from template creation to deployment
+  - Handling interrupted commands and state recovery
+  - Troubleshooting common issues
+  - Cleanup procedures for both application and LXD resources
 
 Follow the project conventions and ensure all checks pass.

Cargo.toml

@@ -21,16 +21,16 @@ name = "torrust-tracker-deployer"
 path = "src/main.rs"
 
 [[bin]]
-name = "e2e-tests-full"
-path = "src/bin/e2e_tests_full.rs"
+name = "e2e-complete-workflow-tests"
+path = "src/bin/e2e_complete_workflow_tests.rs"
 
 [[bin]]
-name = "e2e-config-and-release-tests"
-path = "src/bin/e2e_config_and_release_tests.rs"
+name = "e2e-deployment-workflow-tests"
+path = "src/bin/e2e_deployment_workflow_tests.rs"
 
 [[bin]]
-name = "e2e-provision-and-destroy-tests"
-path = "src/bin/e2e_provision_and_destroy_tests.rs"
+name = "e2e-infrastructure-lifecycle-tests"
+path = "src/bin/e2e_infrastructure_lifecycle_tests.rs"
 
 [[bin]]
 name = "linter"
@@ -48,6 +48,7 @@ clap = { version = "4.0", features = [ "derive" ] }
 derive_more = "0.99"
 figment = { version = "0.10", features = [ "json" ] }
 parking_lot = "0.12"
+reqwest = "0.12"
 rust-embed = "8.0"
 serde = { version = "1.0", features = [ "derive" ] }
 serde_json = "1.0"

README.md

@@ -1,4 +1,4 @@
-[![Linting](https://github.com/torrust/torrust-tracker-deployer/actions/workflows/linting.yml/badge.svg)](https://github.com/torrust/torrust-tracker-deployer/actions/workflows/linting.yml) [![Testing](https://github.com/torrust/torrust-tracker-deployer/actions/workflows/testing.yml/badge.svg)](https://github.com/torrust/torrust-tracker-deployer/actions/workflows/testing.yml) [![E2E Provision Tests](https://github.com/torrust/torrust-tracker-deployer/actions/workflows/test-e2e-provision.yml/badge.svg)](https://github.com/torrust/torrust-tracker-deployer/actions/workflows/test-e2e-provision.yml) [![E2E Config Tests](https://github.com/torrust/torrust-tracker-deployer/actions/workflows/test-e2e-config.yml/badge.svg)](https://github.com/torrust/torrust-tracker-deployer/actions/workflows/test-e2e-config.yml) [![Test LXD Container Provisioning](https://github.com/torrust/torrust-tracker-deployer/actions/workflows/test-lxd-provision.yml/badge.svg)](https://github.com/torrust/torrust-tracker-deployer/actions/workflows/test-lxd-provision.yml) [![Coverage](https://github.com/torrust/torrust-tracker-deployer/actions/workflows/coverage.yml/badge.svg)](https://github.com/torrust/torrust-tracker-deployer/actions/workflows/coverage.yml)
+[![Linting](https://github.com/torrust/torrust-tracker-deployer/actions/workflows/linting.yml/badge.svg)](https://github.com/torrust/torrust-tracker-deployer/actions/workflows/linting.yml) [![Testing](https://github.com/torrust/torrust-tracker-deployer/actions/workflows/testing.yml/badge.svg)](https://github.com/torrust/torrust-tracker-deployer/actions/workflows/testing.yml) [![E2E Infrastructure Tests](https://github.com/torrust/torrust-tracker-deployer/actions/workflows/test-e2e-infrastructure.yml/badge.svg)](https://github.com/torrust/torrust-tracker-deployer/actions/workflows/test-e2e-infrastructure.yml) [![E2E Deployment Tests](https://github.com/torrust/torrust-tracker-deployer/actions/workflows/test-e2e-deployment.yml/badge.svg)](https://github.com/torrust/torrust-tracker-deployer/actions/workflows/test-e2e-deployment.yml) [![Test LXD Container Provisioning](https://github.com/torrust/torrust-tracker-deployer/actions/workflows/test-lxd-provision.yml/badge.svg)](https://github.com/torrust/torrust-tracker-deployer/actions/workflows/test-lxd-provision.yml) [![Coverage](https://github.com/torrust/torrust-tracker-deployer/actions/workflows/coverage.yml/badge.svg)](https://github.com/torrust/torrust-tracker-deployer/actions/workflows/coverage.yml)
 
 # Torrust Tracker Deployer
 
@@ -12,9 +12,9 @@
 > - ✅ **Hetzner Cloud support** for production deployments
 > - ✅ Development and testing workflows
 > - ✅ Multi-provider architecture (provider selection via configuration)
-> - ❌ Application deployment (Torrust Tracker stack) - coming soon
+> - ✅ **Application deployment** (Torrust Tracker stack with Docker Compose)
 >
-> 📋 **MVP Goal:** After completing the [roadmap](docs/roadmap.md), we will have a fully automated deployment solution for Torrust Tracker with complete application stack management.
+> 📋 **MVP Goal:** After completing the [roadmap](docs/roadmap.md), we will have a fully automated deployment solution for Torrust Tracker with complete application stack management and multi-cloud provider support.
 
 This Rust application provides automated deployment infrastructure for Torrust tracker projects. It supports **local development** with LXD and **production deployments** with Hetzner Cloud. The multi-provider architecture allows easy extension to additional cloud providers.
 
@@ -28,12 +28,13 @@ This Rust application provides automated deployment infrastructure for Torrust t
 - ✅ **Fast, easy to install and use** local development solution
 - ✅ **No nested virtualization dependency** (CI compatibility)
 - ✅ **Multi-provider support** (LXD for local, Hetzner Cloud for production)
+- ✅ **Application stack deployment** (Torrust Tracker with Docker Compose)
 
 **Future MVP Goals:** (See [roadmap](docs/roadmap.md))
 
 - 🔄 **Additional cloud providers** (AWS, GCP, Azure)
-- 🔄 **Application stack deployment** (Torrust Tracker with Docker Compose)
 - 🔄 **Multi-environment management**
+- 🔄 **Enhanced observability** (monitoring, alerting, metrics)
 
 ## 🔧 Local Development Approach
 
@@ -164,18 +165,18 @@ Use the E2E test binaries to run automated infrastructure tests with hardcoded e
 
 ```bash
 # Run comprehensive E2E tests (LOCAL ONLY - connectivity issues in GitHub runners)
-cargo run --bin e2e-tests-full
+cargo run --bin e2e-complete-workflow-tests
 
 # Run individual E2E test suites
-cargo run --bin e2e-config-and-release-tests      # Configuration, release, and run workflow tests
-cargo run --bin e2e-provision-and-destroy-tests   # Infrastructure provisioning tests
+cargo run --bin e2e-deployment-workflow-tests         # Configuration, release, and run workflow tests
+cargo run --bin e2e-infrastructure-lifecycle-tests   # Infrastructure provisioning tests
 
 # Keep the test environment after completion for inspection
-cargo run --bin e2e-tests-full -- --keep
-cargo run --bin e2e-provision-and-destroy-tests -- --keep
+cargo run --bin e2e-complete-workflow-tests -- --keep
+cargo run --bin e2e-infrastructure-lifecycle-tests -- --keep
 
 # Use custom templates directory
-cargo run --bin e2e-tests-full -- --templates-dir ./custom/templates
+cargo run --bin e2e-complete-workflow-tests -- --templates-dir ./custom/templates
 
 # See all available options
 cargo run --bin e2e-tests-full -- --help
@@ -190,15 +191,16 @@ cargo run --bin e2e-tests-full -- --help
 
 ### 📖 Manual Deployment Steps
 
-> **✅ Infrastructure commands are now available!** You can create, provision, configure, test, and destroy deployment environments using the CLI.
+> **✅ Complete deployment workflow is now available!** You can create, provision, configure, test, deploy, run, and destroy Torrust Tracker environments using the CLI.
 >
 > **Current Status:**
 >
 > - ✅ **Environment Management**: Create and manage deployment environments
-> - ✅ **Infrastructure Provisioning**: Provision VM infrastructure with LXD
+> - ✅ **Infrastructure Provisioning**: Provision VM infrastructure with LXD or Hetzner Cloud
 > - ✅ **Configuration**: Configure provisioned infrastructure (Docker, Docker Compose)
 > - ✅ **Verification**: Test deployment infrastructure
-> - ⚠️ **Application Deployment**: Not yet available - tracker application deployment coming soon
+> - ✅ **Application Deployment**: Deploy Torrust Tracker configuration and database
+> - ✅ **Service Management**: Start and manage tracker services
 >
 > **Available Commands:**
 >
@@ -220,7 +222,13 @@ cargo run --bin e2e-tests-full -- --help
 > # 6. Verify deployment infrastructure
 > torrust-tracker-deployer test my-environment
 >
-> # 7. Destroy environment when done
+> # 7. Deploy tracker application configuration
+> torrust-tracker-deployer release my-environment
+>
+> # 8. Start tracker services
+> torrust-tracker-deployer run my-environment
+>
+> # 9. Destroy environment when done
 > torrust-tracker-deployer destroy my-environment
 > ```
 >

docker/provisioned-instance/README.md

@@ -176,4 +176,4 @@ This container configuration supports the E2E test split architecture:
 ## Related Documentation
 
 - [Docker Configuration Testing Research](../../docs/research/e2e-docker-config-testing.md)
-- [E2E Testing Guide](../../docs/e2e-testing.md)
+- [E2E Testing Guide](../../docs/e2e-testing/)