Skip to content

Commit 7b7066c

Browse files
committed
Merge #136: Implement dependency installation logic with async Rust converters for bash scripts
6e0b214 test: [#117] suppress stderr in error exit code test for clean output (Jose Celano) a0d9276 fix: [#117] correct test file paths in dependency-installer Cargo.toml (Jose Celano) 1e1ee7d docs: [#117] integrate dependency-installer package into global documentation (Jose Celano) d9db859 refactor: [#117] rename integration test files for clarity (Jose Celano) 4f25a43 test: [#117] use exit codes instead of parsing output in dependency-installer tests (Jose Celano) 04c9381 refactor: [#117] simplify error handling with automatic conversions (Jose Celano) c6f7898 feat: [#117] add snapd to Dockerfile and document LXD test limitations (Jose Celano) 9b5767a feat: [#117] optimize OpenTofu installer tests by moving dependencies to Dockerfile (Jose Celano) 92dc7a1 feat: [#117] add ImageBuilder to auto-build Docker images for package tests (Jose Celano) a453356 perf: [#117] optimize Ansible installer by removing apt-get update (Jose Celano) c2df939 perf: [#117] optimize Docker tests by pre-building container image with OS dependencies (Jose Celano) 65cc194 fix: conditionally import unused modules on Windows (copilot-swe-agent[bot]) 1ebd884 fix: resolve Windows compilation errors in OpenTofu installer (copilot-swe-agent[bot]) 2cd4101 fix: address code review feedback - async sleep and cross-platform concerns (copilot-swe-agent[bot]) edbfde5 docs: update README with install command and fix clippy warnings (copilot-swe-agent[bot]) 1706359 fix: format code and fix detector test imports (copilot-swe-agent[bot]) 8ce6ca9 test: add Docker tests for install command (copilot-swe-agent[bot]) 39f4f98 feat: add installer trait and implementations with install command (copilot-swe-agent[bot]) f3f8feb Initial plan (copilot-swe-agent[bot]) Pull request description: Implement Installation Logic (Issue #117) This PR implements the installation logic for the dependency-installer package, converting bash scripts to Rust implementations with structured logging. **Implementation Status**: ✅ Complete All phases completed: - [x] Phase 1: Installer trait and error types - [x] Phase 2: Installer implementations (4 dependencies) - [x] Phase 3: DependencyManager extensions - [x] Phase 4: Install command handler - [x] Phase 5: CLI and app updates - [x] Phase 6: Docker tests - [x] Phase 7: Testing and validation - [x] Phase 8: Documentation **Recent Fixes**: - Fixed Windows compilation errors by restructuring OpenTofu installer to return early on non-Unix systems - Fixed unused import warnings by making imports conditional with #[cfg(unix)] **Test Results**: - All 17 tests pass (6 non-ignored, 5 ignored for expensive operations, 6 unit tests) - Zero clippy warnings - Code properly formatted - Cross-platform compilation verified (Unix and Windows) <!-- START COPILOT CODING AGENT SUFFIX --> <details> <summary>Original prompt</summary> ---- *This section details on the original issue you should resolve* <issue_title>Implement Installation Logic</issue_title> <issue_description># Implement Installation Logic (Issue 1-1-4) ## Overview Implement the installation logic for all 4 dependencies (cargo-machete, OpenTofu, Ansible, LXD) and add the `install` command to the CLI. This completes the dependency installer package by converting bash installation scripts to Rust with structured logging for automation and CI/CD integration. **Design Philosophy**: Uses **structured logging only** (tracing crate) - no user-facing `println!()` output. Designed for automation and CI/CD pipelines. ## Parent Issue [#113](#113) - Create Dependency Installation Package for E2E Tests ## Dependencies **Depends On**: - [#116](#116) - Create Docker Test Infrastructure (Issue 1-1-3) **Blocks**: - Issue 1-2: Integrate dependency-installer with E2E tests ## Objectives - [ ] Define `DependencyInstaller` trait for installation abstraction - [ ] Convert bash scripts (`scripts/setup/`) to Rust installer implementations - [ ] Add `install` command to CLI binary with handler-based architecture - [ ] Extend `DependencyManager` to coordinate installation - [ ] Extend Docker tests to verify actual installations - [ ] Use structured logging (tracing) throughout for observability - [ ] Document exit codes and usage patterns ## Key Components ### DependencyInstaller Trait ```rust #[async_trait] pub trait DependencyInstaller: Send + Sync { fn name(&self) -> &str; fn dependency(&self) -> Dependency; async fn install(&self) -> Result<(), InstallationError>; fn requires_sudo(&self) -> bool { false } } ``` ### Installer Implementations Convert bash scripts to Rust: 1. **CargoMacheteInstaller** (`scripts/setup/install-cargo-machete.sh`) - Uses `cargo install cargo-machete` - No sudo required 2. **OpenTofuInstaller** (`scripts/setup/install-opentofu.sh`) - Downloads installer script with curl - Runs with sudo - Multi-step: download → chmod → execute → cleanup 3. **AnsibleInstaller** (`scripts/setup/install-ansible.sh`) - Uses apt-get package manager - Requires sudo 4. **LxdInstaller** (`scripts/setup/install-lxd.sh`) - Uses snap for installation - Configures user groups - Requires sudo ### Extended DependencyManager Add installation methods to existing manager: ```rust impl DependencyManager { pub fn get_installer(&self, dep: Dependency) -> Box<dyn DependencyInstaller>; pub async fn install(&self, dep: Dependency) -> Result<(), InstallationError>; pub async fn install_all(&self) -> Result<Vec<InstallResult>, InstallationError>; } ``` ### Install Command Handler New handler following existing pattern: ```rust // src/handlers/install.rs pub async fn handle_install( manager: &DependencyManager, dependency: Option<Dependency>, ) -> Result<(), InstallError> { // Handler implementation with structured logging } ``` ### CLI Command Add to existing CLI: ```bash # Install all dependencies dependency-installer install # Install specific dependency dependency-installer install --dependency opentofu # With verbose logging dependency-installer install --verbose ``` ### Docker Tests Extend testing to verify actual installations: ```rust // tests/docker_install_command.rs #[tokio::test] async fn test_install_cargo_machete() { // Verify installation in clean container } #[tokio::test] async fn test_install_idempotent() { // Install twice, both should succeed } #[tokio::test] async fn test_install_all() { // Install all dependencies } ``` ## Architecture ### Directory Structure ```text packages/dependency-installer/ ├── src/ │ ├── manager.rs # Add installation methods │ ├── detector/ # Existing detection logic │ ├── installer/ # NEW: Installation logic │ │ ├── mod.rs # Trait + error types │ │ ├── cargo_machete.rs │ │ ├── opentofu.rs │ │ ├── ansible.rs │ │ └── lxd.rs │ ├── handlers/ # Extend with install │ │ ├── check.rs # Existing │ │ ├── list.rs # Existing │ │ └── install.rs # NEW │ └── cli.rs # Add Install command └── tests/ └── docker_install_command.rs # NEW tests ``` ### Handler-Based Architecture Following existing pattern: ```rust // src/app.rs match cli.command { Commands::Check { dependency } => { check::handle_check(&manager, dependency)?; } Commands::List => { list::handle_list(&manager)?; } Commands::Install { dependency } => { // NEW install::handle_install(&manager, dependency).await?; } } ``` ## Structured Logging Examples All output uses `tracing` crate - no `println!()` statements: ### Installing All Dependencies ```bash $ dependency-installer install 2... </details> - Fixes #117 <!-- START COPILOT CODING AGENT TIPS --> --- 💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more [Copilot coding agent tips](https://gh.io/copilot-coding-agent-tips) in the docs. ACKs for top commit: josecelano: ACK 6e0b214 Tree-SHA512: f2117d188b0adfde2e9a7813a0da1db1301b0b071ed2c5f0ef8f0bf5d607e92ea8bbcb49cbe84a9fa548c5ba29c17f217b519c229bbd243379a4bc1d812f6766
2 parents 4fafe86 + 6e0b214 commit 7b7066c

34 files changed

+1729
-65
lines changed

.github/copilot-instructions.md

Lines changed: 12 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -20,7 +20,7 @@ This is a deployment infrastructure proof-of-concept for the Torrust ecosystem.
2020
## 📁 Key Directories
2121

2222
- `src/` - Rust source code organized by DDD layers (`domain/`, `application/`, `infrastructure/`, `shared/`)
23-
- `src/bin/` - Binary executables (linter, E2E tests)
23+
- `src/bin/` - Binary executables (linter, E2E tests, dependency installer)
2424
- `data/` - Environment-specific data and source templates
2525
- `templates/` - Generated template examples and test fixtures
2626
- `build/` - Generated runtime configurations (git-ignored)
@@ -29,7 +29,9 @@ This is a deployment infrastructure proof-of-concept for the Torrust ecosystem.
2929
- `docs/decisions/` - Architectural Decision Records (ADRs)
3030
- `scripts/` - Shell scripts for development tasks
3131
- `fixtures/` - Test data and keys for development
32-
- `packages/` - Rust workspace packages (linting tools)
32+
- `packages/` - Rust workspace packages (see `packages/README.md` for details)
33+
- `dependency-installer/` - Dependency detection and installation for development setup
34+
- `linting/` - Unified linting framework
3335

3436
## 📄 Key Configuration Files
3537

@@ -103,15 +105,22 @@ These principles should guide all development decisions, code reviews, and featu
103105

104106
## 🧪 Build & Test
105107

108+
- **Setup Dependencies**: `cargo run --bin dependency-installer install` (sets up required development tools)
109+
- **Check dependencies**: `cargo run --bin dependency-installer check` (verifies installation)
110+
- **List dependencies**: `cargo run --bin dependency-installer list` (shows all dependencies with status)
111+
- Required tools: OpenTofu, Ansible, LXD, cargo-machete
112+
- See [`packages/dependency-installer/README.md`](../packages/dependency-installer/README.md) for details
106113
- **Lint**: `cargo run --bin linter all` (comprehensive - tests stable & nightly toolchains)
107114
- Individual linters: `cargo run --bin linter {markdown|yaml|toml|cspell|clippy|rustfmt|shellcheck}`
108115
- Alternative: `./scripts/lint.sh` (wrapper that calls the Rust binary)
109116
- **Dependencies**: `cargo machete` (mandatory before commits - no unused dependencies)
110117
- **Build**: `cargo build`
111118
- **Test**: `cargo test`
112119
- **Unit Tests**: When writing unit tests, follow conventions described in [`docs/contributing/testing/`](../docs/contributing/testing/)
113-
- **E2E Tests**: `cargo run --bin e2e-tests-full` (comprehensive - all tests) or individual tests:
120+
- **E2E Tests**:
121+
- `cargo run --bin e2e-tests-full` - Comprehensive tests (⚠️ **LOCAL ONLY** - cannot run on GitHub Actions due to network connectivity issues)
114122
- `cargo run --bin e2e-provision-tests` - Infrastructure provisioning tests
115123
- `cargo run --bin e2e-config-tests` - Configuration validation tests
124+
- See [`docs/e2e-testing.md`](../docs/e2e-testing.md) for detailed information about CI limitations
116125

117126
Follow the project conventions and ensure all checks pass.

README.md

Lines changed: 25 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -64,10 +64,32 @@ This repository uses LXD virtual machines for local virtualization and developme
6464

6565
This is a Rust application that automates deployment infrastructure using OpenTofu and Ansible.
6666

67-
Install the required tools:
67+
#### Automated Setup (Recommended)
68+
69+
The project provides a dependency installer tool that automatically detects and installs required dependencies:
70+
71+
```bash
72+
# Install all required dependencies
73+
cargo run --bin dependency-installer install
74+
75+
# Check which dependencies are installed
76+
cargo run --bin dependency-installer check
77+
78+
# List all dependencies with status
79+
cargo run --bin dependency-installer list
80+
```
81+
82+
The installer supports: **OpenTofu**, **Ansible**, **LXD**, and **cargo-machete**.
83+
84+
For detailed information, see **[📖 Dependency Installer →](packages/dependency-installer/README.md)**.
85+
86+
#### Manual Setup
87+
88+
If you prefer manual installation or need to troubleshoot:
89+
90+
**Check installations:**
6891

6992
```bash
70-
# Check installations
7193
lxd version && tofu version && ansible --version && cargo --version
7294
```
7395

@@ -76,7 +98,7 @@ lxd version && tofu version && ansible --version && cargo --version
7698
- **[📖 OpenTofu Setup Guide →](docs/tech-stack/opentofu.md)**
7799
- **[📖 Ansible Setup Guide →](docs/tech-stack/ansible.md)**
78100

79-
**Quick install:**
101+
**Quick manual install:**
80102

81103
```bash
82104
# Install Rust (if not already installed)

docs/contributing/README.md

Lines changed: 8 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -25,7 +25,14 @@ This guide will help you understand our development practices and contribution w
2525
## 🚀 Getting Started
2626

2727
1. **Fork and clone** the repository
28-
2. **Set up your development environment** following the main [README](../../README.md)
28+
2. **Install dependencies** using the automated installer:
29+
30+
```bash
31+
cargo run --bin dependency-installer install
32+
```
33+
34+
See [Dependency Installer](../../packages/dependency-installer/README.md) for details.
35+
2936
3. **Read the branching** guidelines in [branching.md](./branching.md)
3037
4. **Install and run linters** as described in [linting.md](./linting.md)
3138
5. **Follow the commit process** outlined in [commit-process.md](./commit-process.md)

docs/e2e-testing.md

Lines changed: 43 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -180,7 +180,35 @@ Combines both provision and configuration phases in a single LXD VM for comprehe
180180

181181
## 🛠️ Prerequisites
182182

183-
### For E2E Provision Tests
183+
### Automated Setup (Recommended)
184+
185+
The project provides a dependency installer tool that automatically detects and installs required dependencies:
186+
187+
```bash
188+
# Install all required dependencies
189+
cargo run --bin dependency-installer install
190+
191+
# Check which dependencies are installed
192+
cargo run --bin dependency-installer check
193+
194+
# List all dependencies with status
195+
cargo run --bin dependency-installer list
196+
```
197+
198+
The installer supports:
199+
200+
- **cargo-machete** - Detects unused Rust dependencies
201+
- **OpenTofu** - Infrastructure provisioning tool
202+
- **Ansible** - Configuration management tool
203+
- **LXD** - VM-based testing infrastructure
204+
205+
For detailed information, see [`packages/dependency-installer/README.md`](../packages/dependency-installer/README.md).
206+
207+
### Manual Setup
208+
209+
If you prefer manual installation or need to troubleshoot:
210+
211+
#### For E2E Provision Tests
184212

185213
1. **LXD installed and configured**
186214

@@ -195,7 +223,7 @@ Combines both provision and configuration phases in a single LXD VM for comprehe
195223
# Installation instructions in docs/tech-stack/opentofu.md
196224
```
197225

198-
### For E2E Configuration Tests
226+
#### For E2E Configuration Tests
199227

200228
1. **Docker installed**
201229

@@ -210,10 +238,22 @@ Combines both provision and configuration phases in a single LXD VM for comprehe
210238
# Installation instructions in docs/tech-stack/ansible.md
211239
```
212240

213-
### For Full Local Tests (`e2e-tests-full`)
241+
#### For Full Local Tests (`e2e-tests-full`)
214242

215243
Requires **all** of the above: LXD, OpenTofu, Docker, and Ansible.
216244

245+
### Verification
246+
247+
After setup (automated or manual), verify all dependencies are available:
248+
249+
```bash
250+
# Quick check (exit code indicates success/failure)
251+
cargo run --bin dependency-installer check
252+
253+
# Detailed check with logging
254+
cargo run --bin dependency-installer check --verbose
255+
```
256+
217257
## 🐛 Troubleshooting
218258

219259
### Test Environment Cleanup

packages/README.md

Lines changed: 151 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,151 @@
1+
# Torrust Tracker Deployer - Packages
2+
3+
This directory contains reusable Rust workspace packages that support the Torrust Tracker Deployer project. These packages are designed to be modular, maintainable, and potentially reusable across other Torrust projects.
4+
5+
## 📦 Available Packages
6+
7+
### [`dependency-installer/`](./dependency-installer/)
8+
9+
**Purpose**: Dependency detection and installation utilities for development environments
10+
11+
**Key Features**:
12+
13+
- Detects if required development tools are installed (OpenTofu, Ansible, LXD, cargo-machete)
14+
- Installs missing dependencies automatically
15+
- Provides CLI for manual and automated use
16+
- Designed for CI/CD pipelines and automated workflows
17+
- Uses structured logging (tracing) for observability
18+
- Exit-code based success/failure indication for automation
19+
20+
**Use Cases**:
21+
22+
- Setting up development environments for humans and AI agents
23+
- Pre-flight checks in E2E test suites
24+
- CI/CD pipeline dependency validation
25+
- Automated development environment provisioning
26+
27+
**Documentation**: See [packages/dependency-installer/README.md](./dependency-installer/README.md)
28+
29+
### [`linting/`](./linting/)
30+
31+
**Purpose**: Unified linting framework for Rust projects
32+
33+
**Key Features**:
34+
35+
- Supports multiple linters: markdown, YAML, TOML, Rust (clippy + rustfmt), shellcheck
36+
- Pre-built CLI components for easy binary creation
37+
- Extensible architecture for adding new linters
38+
- Uses existing configuration files (`.taplo.toml`, `.yamllint.yml`, etc.)
39+
40+
**Use Cases**:
41+
42+
- Enforcing code quality standards
43+
- Pre-commit validation
44+
- CI/CD linting pipelines
45+
- Standardizing linting across multiple projects
46+
47+
**Documentation**: See [packages/linting/README.md](./linting/README.md)
48+
49+
## 🏗️ Package Architecture
50+
51+
All packages in this directory:
52+
53+
- Are part of a Cargo workspace (defined in root `Cargo.toml`)
54+
- Can be used independently or as library crates
55+
- Follow the project's development principles (observability, testability, user-friendliness)
56+
- Provide both CLI binaries and programmatic APIs
57+
- Use structured logging via the `tracing` crate
58+
- Follow consistent error handling patterns
59+
60+
## 🚀 Using Packages
61+
62+
### As Library Crates
63+
64+
```rust
65+
// Add to your Cargo.toml
66+
[dependencies]
67+
torrust-linting = { path = "packages/linting" }
68+
torrust-dependency-installer = { path = "packages/dependency-installer" }
69+
```
70+
71+
### As CLI Binaries
72+
73+
```bash
74+
# Run the linter
75+
cargo run --bin linter all
76+
77+
# Run the dependency installer
78+
cargo run --bin dependency-installer check
79+
cargo run --bin dependency-installer install
80+
```
81+
82+
## 🎯 Package Design Principles
83+
84+
### Automation-First Design
85+
86+
Packages prioritize **automation and CI/CD workflows**:
87+
88+
- **Exit codes** indicate success/failure (0 = success, non-zero = failure)
89+
- **Structured logging** provides rich context without parsing output
90+
- **Flags for verbosity** (`--verbose`, `--log-level`) control output detail
91+
- **Minimal output** by default, detailed only when needed
92+
93+
### Type Safety
94+
95+
- Use strongly-typed enums and structs
96+
- Leverage Rust's type system for compile-time guarantees
97+
- Avoid stringly-typed APIs
98+
99+
### Error Handling
100+
101+
- Clear, actionable error messages
102+
- Preserve error context with source chains
103+
- Use thiserror for structured error types
104+
105+
### Extensibility
106+
107+
- Easy to add new functionality (linters, dependencies)
108+
- Plugin-like architecture where appropriate
109+
- Trait-based abstractions for flexibility
110+
111+
## 📋 Adding New Packages
112+
113+
When creating new packages:
114+
115+
1. **Create package directory** under `packages/`
116+
2. **Add to workspace** in root `Cargo.toml`:
117+
118+
```toml
119+
[workspace]
120+
members = [
121+
"packages/your-new-package",
122+
# ...
123+
]
124+
```
125+
126+
3. **Create package README** documenting purpose and usage
127+
4. **Update this file** to include the new package in the list above
128+
5. **Follow conventions**:
129+
- Use `tracing` for logging
130+
- Provide both CLI and library interfaces
131+
- Follow project development principles
132+
- Add comprehensive tests
133+
134+
## 🔗 Related Documentation
135+
136+
- [Development Principles](../docs/development-principles.md) - Core principles guiding all packages
137+
- [Error Handling Guide](../docs/contributing/error-handling.md) - Error handling patterns
138+
- [Testing Conventions](../docs/contributing/testing/) - Testing standards
139+
- [E2E Testing Guide](../docs/e2e-testing.md) - How packages integrate with E2E tests
140+
141+
## 💡 Future Packages
142+
143+
Potential future packages that could be added:
144+
145+
- **Configuration Management**: Reusable config loading and validation
146+
- **Template Engine**: Tera template rendering utilities
147+
- **SSH Client**: SSH operations and connectivity checking
148+
- **Infrastructure Clients**: OpenTofu, Ansible, LXD client abstractions
149+
- **Test Utilities**: Common test helpers and fixtures
150+
151+
These packages would further modularize the codebase and improve reusability across the Torrust ecosystem.

packages/dependency-installer/Cargo.toml

Lines changed: 8 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -14,15 +14,20 @@ name = "dependency-installer"
1414
path = "src/bin/dependency-installer.rs"
1515

1616
[dependencies]
17+
async-trait = "0.1"
1718
clap = { version = "4.0", features = [ "derive" ] }
1819
thiserror = "1.0"
20+
tokio = { version = "1.0", features = [ "full" ] }
1921
tracing = "0.1"
2022
tracing-subscriber = { version = "0.3", features = [ "env-filter" ] }
2123

2224
[dev-dependencies]
2325
testcontainers = "0.25"
24-
tokio = { version = "1.0", features = [ "full" ] }
2526

2627
[[test]]
27-
name = "docker_check_command"
28-
path = "tests/docker_check_command.rs"
28+
name = "check_command_docker_integration"
29+
path = "tests/check_command_docker_integration.rs"
30+
31+
[[test]]
32+
name = "install_command_docker_integration"
33+
path = "tests/install_command_docker_integration.rs"

0 commit comments

Comments
 (0)