Skip to content

Commit c7b27ce

Browse files
josecelanojosecelano
committed
docs: [#220] document decision to use single Docker image for sequential E2E testing
Added ADR documenting the architectural decision to use a single Docker image (provisioned-instance) with sequential command execution instead of multiple phase-specific images (configured, released, running). Key points: - Multi-image approach rejected due to high maintenance overhead - Sequential execution faster than building multiple images - Single source of truth easier to maintain and synchronize with code - Trade-offs: No command isolation, no parallel execution (acceptable) Changes: - Created ADR: docs/decisions/single-docker-image-sequential-testing.md - Updated docs/e2e-testing.md to reflect actual implementation - Removed outdated "Future Expansion Architecture" section - Added clear explanation of sequential testing approach - Added comparison table: Docker containers vs LXD VMs - Updated ADR index in docs/decisions/README.md Benefits: - Clear documentation of why we chose sequential approach - Realistic expectations about test isolation and parallelism - Better understanding of maintenance vs feature trade-offs
1 parent 61a712d commit c7b27ce

File tree

3 files changed

+296
-107
lines changed

3 files changed

+296
-107
lines changed

‎docs/decisions/README.md‎

Lines changed: 23 additions & 22 deletions
Original file line numberDiff line numberDiff line change
@@ -4,28 +4,29 @@ This directory contains architectural decision records for the Torrust Tracker D
44

55
## Decision Index
66

7-
| Status | Date | Decision | Summary |
8-
| ------------- | ---------- | --------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------- |
9-
| ✅ Accepted | 2025-12-09 | [Register Command SSH Port Override](./register-ssh-port-override.md) | Add optional --ssh-port argument to register command for non-standard SSH ports |
10-
| ✅ Accepted | 2025-11-19 | [Disable MD060 Table Formatting Rule](./md060-table-formatting-disabled.md) | Disable MD060 to allow flexible table formatting and emoji usage |
11-
| ✅ Accepted | 2025-11-19 | [Test Command as Smoke Test](./test-command-as-smoke-test.md) | Test command validates running services, not infrastructure components |
12-
| ✅ Accepted | 2025-11-13 | [Migration to AGENTS.md Standard](./agents-md-migration.md) | Adopt open AGENTS.md standard for multi-agent compatibility while keeping GitHub redirect |
13-
| ✅ Accepted | 2025-11-11 | [Use ReentrantMutex Pattern for UserOutput Reentrancy](./reentrant-mutex-useroutput-pattern.md) | Use Arc<ReentrantMutex<RefCell<UserOutput>>> to fix same-thread deadlock in issue #164 |
14-
| ❌ Superseded | 2025-11-11 | [Remove UserOutput Mutex](./user-output-mutex-removal.md) | Remove Arc<Mutex<UserOutput>> pattern for simplified, deadlock-free architecture |
15-
| ✅ Accepted | 2025-11-07 | [ExecutionContext Wrapper Pattern](./execution-context-wrapper.md) | Use ExecutionContext wrapper around Container for future-proof command signatures |
16-
| ✅ Accepted | 2025-11-03 | [Environment Variable Prefix](./environment-variable-prefix.md) | Use `TORRUST_TD_` prefix for all environment variables |
17-
| ✅ Accepted | 2025-10-15 | [External Tool Adapters Organization](./external-tool-adapters-organization.md) | Consolidate external tool wrappers in `src/adapters/` for better discoverability |
18-
| ✅ Accepted | 2025-10-10 | [Repository Rename to Deployer](./repository-rename-to-deployer.md) | Rename from "Torrust Tracker Deploy" to "Torrust Tracker Deployer" for production use |
19-
| ✅ Accepted | 2025-10-03 | [Error Context Strategy](./error-context-strategy.md) | Use structured error context with trace files for complete error information |
20-
| ✅ Accepted | 2025-10-03 | [Command State Return Pattern](./command-state-return-pattern.md) | Commands return typed states (Environment<S> → Environment<T>) for compile-time safety |
21-
| ✅ Accepted | 2025-10-03 | [Actionable Error Messages](./actionable-error-messages.md) | Use tiered help system with brief tips + .help() method for detailed troubleshooting |
22-
| ✅ Accepted | 2025-10-01 | [Type Erasure for Environment States](./type-erasure-for-environment-states.md) | Use enum-based type erasure to enable runtime handling and serialization of typed states |
23-
| ✅ Accepted | 2025-09-29 | [Test Context vs Deployment Environment Naming](./test-context-vs-deployment-environment-naming.md) | Rename TestEnvironment to TestContext to avoid conflicts with multi-environment feature |
24-
| ✅ Accepted | 2025-09-10 | [LXD VMs over Containers](./lxd-vm-over-containers.md) | Use LXD virtual machines instead of containers for production alignment |
25-
| ✅ Accepted | 2025-09-09 | [Tera Minimal Templating Strategy](./tera-minimal-templating-strategy.md) | Use Tera with minimal variables and templates to avoid complexity and delimiter conflicts |
26-
| ✅ Accepted | - | [LXD over Multipass](./lxd-over-multipass.md) | Choose LXD containers over Multipass VMs for deployment testing |
27-
| ✅ Resolved | - | [Docker Testing Evolution](./docker-testing-evolution.md) | Evolution from Docker rejection to hybrid approach for split E2E testing |
28-
| ✅ Accepted | - | [Meson Removal](./meson-removal.md) | Remove Meson build system from the project |
7+
| Status | Date | Decision | Summary |
8+
| ------------- | ---------- | ----------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------- |
9+
| ✅ Accepted | 2025-12-10 | [Single Docker Image for Sequential E2E Command Testing](./single-docker-image-sequential-testing.md) | Use single Docker image with sequential command execution instead of multi-image phases |
10+
| ✅ Accepted | 2025-12-09 | [Register Command SSH Port Override](./register-ssh-port-override.md) | Add optional --ssh-port argument to register command for non-standard SSH ports |
11+
| ✅ Accepted | 2025-11-19 | [Disable MD060 Table Formatting Rule](./md060-table-formatting-disabled.md) | Disable MD060 to allow flexible table formatting and emoji usage |
12+
| ✅ Accepted | 2025-11-19 | [Test Command as Smoke Test](./test-command-as-smoke-test.md) | Test command validates running services, not infrastructure components |
13+
| ✅ Accepted | 2025-11-13 | [Migration to AGENTS.md Standard](./agents-md-migration.md) | Adopt open AGENTS.md standard for multi-agent compatibility while keeping GitHub redirect |
14+
| ✅ Accepted | 2025-11-11 | [Use ReentrantMutex Pattern for UserOutput Reentrancy](./reentrant-mutex-useroutput-pattern.md) | Use Arc<ReentrantMutex<RefCell<UserOutput>>> to fix same-thread deadlock in issue #164 |
15+
| ❌ Superseded | 2025-11-11 | [Remove UserOutput Mutex](./user-output-mutex-removal.md) | Remove Arc<Mutex<UserOutput>> pattern for simplified, deadlock-free architecture |
16+
| ✅ Accepted | 2025-11-07 | [ExecutionContext Wrapper Pattern](./execution-context-wrapper.md) | Use ExecutionContext wrapper around Container for future-proof command signatures |
17+
| ✅ Accepted | 2025-11-03 | [Environment Variable Prefix](./environment-variable-prefix.md) | Use `TORRUST_TD_` prefix for all environment variables |
18+
| ✅ Accepted | 2025-10-15 | [External Tool Adapters Organization](./external-tool-adapters-organization.md) | Consolidate external tool wrappers in `src/adapters/` for better discoverability |
19+
| ✅ Accepted | 2025-10-10 | [Repository Rename to Deployer](./repository-rename-to-deployer.md) | Rename from "Torrust Tracker Deploy" to "Torrust Tracker Deployer" for production use |
20+
| ✅ Accepted | 2025-10-03 | [Error Context Strategy](./error-context-strategy.md) | Use structured error context with trace files for complete error information |
21+
| ✅ Accepted | 2025-10-03 | [Command State Return Pattern](./command-state-return-pattern.md) | Commands return typed states (Environment<S> → Environment<T>) for compile-time safety |
22+
| ✅ Accepted | 2025-10-03 | [Actionable Error Messages](./actionable-error-messages.md) | Use tiered help system with brief tips + .help() method for detailed troubleshooting |
23+
| ✅ Accepted | 2025-10-01 | [Type Erasure for Environment States](./type-erasure-for-environment-states.md) | Use enum-based type erasure to enable runtime handling and serialization of typed states |
24+
| ✅ Accepted | 2025-09-29 | [Test Context vs Deployment Environment Naming](./test-context-vs-deployment-environment-naming.md) | Rename TestEnvironment to TestContext to avoid conflicts with multi-environment feature |
25+
| ✅ Accepted | 2025-09-10 | [LXD VMs over Containers](./lxd-vm-over-containers.md) | Use LXD virtual machines instead of containers for production alignment |
26+
| ✅ Accepted | 2025-09-09 | [Tera Minimal Templating Strategy](./tera-minimal-templating-strategy.md) | Use Tera with minimal variables and templates to avoid complexity and delimiter conflicts |
27+
| ✅ Accepted | - | [LXD over Multipass](./lxd-over-multipass.md) | Choose LXD containers over Multipass VMs for deployment testing |
28+
| ✅ Resolved | - | [Docker Testing Evolution](./docker-testing-evolution.md) | Evolution from Docker rejection to hybrid approach for split E2E testing |
29+
| ✅ Accepted | - | [Meson Removal](./meson-removal.md) | Remove Meson build system from the project |
2930

3031
## ADR Template
3132

‎docs/decisions/single-docker-image-sequential-testing.md‎

Lines changed: 188 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,188 @@
1+
# Decision: Single Docker Image for Sequential E2E Command Testing
2+
3+
## Status
4+
5+
✅ Accepted
6+
7+
## Date
8+
9+
2025-12-10
10+
11+
## Context
12+
13+
When designing the E2E testing architecture for deployment workflow tests, we initially planned to create multiple Docker images representing different deployment phases:
14+
15+
- `provisioned-instance` - Post-provision state (base system ready)
16+
- `configured-instance` - Post-configure state (dependencies installed)
17+
- `released-instance` - Post-release state (applications deployed)
18+
- `running-instance` - Post-run state (services started)
19+
20+
This multi-image approach would theoretically allow:
21+
22+
- **Isolated phase testing**: Test individual commands (configure, release, run, test) independently
23+
- **Parallel test execution**: Run E2E tests for different commands in parallel
24+
- **Clear phase boundaries**: Each image captures the exact state after a specific deployment phase
25+
26+
However, implementing and maintaining this architecture presented significant challenges:
27+
28+
1. **High Maintenance Overhead**: Every code change affecting any deployment phase requires updating multiple Docker images
29+
2. **Image Synchronization**: Keeping all phase images in sync with code changes is error-prone and time-consuming
30+
3. **Build Time**: Building multiple Docker images sequentially would be slower than running commands sequentially in a single container
31+
4. **Parallel Execution Overhead**: Even with parallel tests, the Docker build and startup time for multiple images outweighs the benefits
32+
5. **Complexity**: Managing multiple Dockerfiles, build dependencies, and test orchestration adds significant complexity
33+
6. **Duplication**: Much of the image content would be duplicated across phases (base system, users, SSH setup)
34+
35+
The fundamental trade-off is between **test isolation/parallelism** (multiple images) versus **maintainability/simplicity** (single image).
36+
37+
## Decision
38+
39+
We will use a **single Docker image** (`provisioned-instance`) representing the pre-provisioned instance state, and run all deployment commands **sequentially** within that container during E2E tests.
40+
41+
### Implementation Details
42+
43+
**Single Image Approach**:
44+
45+
```text
46+
docker/provisioned-instance/
47+
├── Dockerfile # Ubuntu 24.04 LTS + SSH + torrust user
48+
├── supervisord.conf # Process management
49+
├── entrypoint.sh # Container initialization
50+
└── README.md # Documentation
51+
```
52+
53+
**Sequential Command Execution**:
54+
55+
```rust
56+
// E2E test workflow (simplified)
57+
async fn run_deployment_workflow_tests() -> Result<()> {
58+
// 1. Start single container (provisioned state)
59+
let container = start_provisioned_container().await?;
60+
61+
// 2. Run commands sequentially
62+
run_create_command()?;
63+
run_register_command(container.ip())?;
64+
run_configure_command()?; // Modifies container state
65+
run_release_command()?; // Modifies container state
66+
run_run_command()?; // Modifies container state
67+
run_test_command()?; // Validates container state
68+
69+
// 3. Cleanup
70+
container.stop().await?;
71+
Ok(())
72+
}
73+
```
74+
75+
### Trade-offs Accepted
76+
77+
**✅ Benefits**:
78+
79+
- **Low Maintenance**: Single Dockerfile to maintain - changes propagate automatically
80+
- **Simpler Architecture**: Clear, understandable test flow
81+
- **Faster Overall**: Sequential execution in one container is faster than building/starting multiple images
82+
- **Easy Debugging**: Single container lifecycle to understand and inspect
83+
- **Code Synchronization**: Image changes automatically reflect code changes via Ansible playbooks
84+
85+
**❌ Trade-offs**:
86+
87+
- **No Command Isolation**: Cannot test individual commands independently (must run full sequence)
88+
- **No Test Parallelism**: Cannot run E2E tests for different commands in parallel
89+
- **State Accumulation**: Later commands see state from earlier commands (intentional - tests real workflow)
90+
- **Longer Test Runs**: If one command fails, must re-run entire sequence
91+
92+
## Consequences
93+
94+
### Positive
95+
96+
1. **Reduced Complexity**: Single Dockerfile, single container, single test flow
97+
2. **Better Maintainability**: Code changes automatically tested via playbooks without image rebuilds
98+
3. **Realistic Testing**: Sequential execution matches real deployment workflow exactly
99+
4. **Faster Iteration**: No need to rebuild multiple images during development
100+
5. **Lower CI Resources**: Single container uses fewer resources than multiple containers
101+
6. **Simplified Debugging**: `--keep` flag allows inspection of final container state with all commands applied
102+
103+
### Negative
104+
105+
1. **Test Coupling**: Commands cannot be tested in isolation - must test full workflow
106+
2. **Longer Feedback**: Must run entire sequence to test later commands
107+
3. **No Parallel Speedup**: Cannot leverage parallel test execution for E2E workflow tests
108+
109+
### Risk Mitigation
110+
111+
The negative consequences are mitigated by:
112+
113+
- **Unit Tests**: Individual command logic is tested in isolation via unit tests
114+
- **Integration Tests**: Command interfaces are tested without full E2E overhead
115+
- **Fast Execution**: Sequential execution in Docker is still fast (~48 seconds total)
116+
- **Split Test Suites**: Infrastructure tests run separately, allowing some parallelism at the suite level
117+
118+
## Alternatives Considered
119+
120+
### Alternative 1: Multi-Image Phase Architecture (Original Plan)
121+
122+
**Approach**: Build separate Docker images for each deployment phase (provisioned, configured, released, running).
123+
124+
**Pros**:
125+
126+
- Command isolation - test individual commands independently
127+
- Parallel test execution possible
128+
- Clear phase boundaries
129+
130+
**Cons**:
131+
132+
- High maintenance overhead - must update multiple images for code changes
133+
- Slower build time - building 4 images takes longer than running 4 commands
134+
- Complex orchestration - managing image dependencies and build order
135+
- Image synchronization issues - keeping images in sync with code
136+
- Higher CI resource usage
137+
138+
**Rejected Because**: Maintenance overhead outweighs benefits. Build time for multiple images exceeds sequential execution time.
139+
140+
### Alternative 2: Docker Compose Multi-Service Setup
141+
142+
**Approach**: Use Docker Compose to orchestrate multiple containers representing different phases.
143+
144+
**Pros**:
145+
146+
- Service isolation
147+
- Declarative configuration
148+
- Can leverage Docker Compose features
149+
150+
**Cons**:
151+
152+
- Even higher complexity than multi-image
153+
- Still requires building/maintaining multiple images
154+
- Orchestration overhead
155+
- Harder to debug
156+
157+
**Rejected Because**: Adds orchestration complexity without solving the fundamental maintenance problem.
158+
159+
### Alternative 3: Container Snapshots Between Commands
160+
161+
**Approach**: Start with one image, create container snapshots after each command, test from snapshots.
162+
163+
**Pros**:
164+
165+
- Single base image
166+
- Can jump to any phase via snapshot
167+
- Some test isolation
168+
169+
**Cons**:
170+
171+
- Snapshot management complexity
172+
- Storage overhead for snapshots
173+
- Non-standard testing approach
174+
- Still requires careful state management
175+
176+
**Rejected Because**: Complexity doesn't justify the limited benefits. Snapshots add non-standard workflow.
177+
178+
## Related Decisions
179+
180+
- [Docker Testing Evolution](./docker-testing-evolution.md) - Evolution from Docker rejection to hybrid approach for E2E testing
181+
- [E2E Test Split Architecture](../e2e-testing.md#architecture) - Split between infrastructure and deployment workflow tests
182+
183+
## References
184+
185+
- [E2E Testing Guide - Docker Architecture](../e2e-testing.md#docker-architecture-for-e2e-testing)
186+
- [Provisioned Instance Documentation](../../docker/provisioned-instance/README.md)
187+
- GitHub Actions E2E Deployment Workflow: `.github/workflows/test-e2e-deployment.yml`
188+
- E2E Deployment Workflow Tests: `src/bin/e2e_deployment_workflow_tests.rs`

0 commit comments

Comments
 (0)