Skip to content

Commit 893ac72

Browse files
josecelanojosecelano
committed
docs: [#220] reorganize E2E testing documentation into structured directory
Split the large docs/e2e-testing.md file (667 lines) into a well-organized directory structure for better navigation and maintainability. ## Changes ### New Structure (docs/e2e-testing/) - README.md - Overview, quick start, and documentation index - architecture.md - E2E architecture, Docker strategy, and design decisions - running-tests.md - How to run tests, command options, prerequisites - test-suites.md - Detailed descriptions of each test suite - troubleshooting.md - Common issues, debugging, cleanup procedures - contributing.md - Guidelines for extending E2E tests - advanced.md - Advanced techniques (cross-environment testing, etc.) ### Updated References Updated all references across the codebase to point to the new directory: - AGENTS.md - packages/README.md - docs/contributing/testing/testing-commands.md - docs/contributing/templates.md - docs/decisions/register-ssh-port-override.md - docker/provisioned-instance/README.md - src/testing/e2e/tasks/ (multiple files) ### Benefits 1. **Better Navigation**: Users can jump directly to relevant topics 2. **Easier Maintenance**: Smaller files are easier to update 3. **Clear Separation**: Each file has a single, focused purpose 4. **Cross-linking**: Related documentation properly linked 5. **Scalability**: Easy to add new sections without monolithic file ### Relationship with contributing/testing/ Kept separate from docs/contributing/testing/ because: - Different audiences (test runners vs test writers) - Different purposes (system-level vs code-level testing) - E2E testing important enough for top-level docs/ - Combined would be 2,000+ lines (too large) Cross-linked for users who need both perspectives. BREAKING CHANGE: The single docs/e2e-testing.md file has been removed and replaced with a directory structure at docs/e2e-testing/. All references in documentation and source code have been updated to point to the new location.
1 parent c7b27ce commit 893ac72

20 files changed

+1122
-14
lines changed

‎AGENTS.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -134,6 +134,6 @@ These principles should guide all development decisions, code reviews, and featu
134134
- `cargo run --bin e2e-infrastructure-lifecycle-tests` - Infrastructure provisioning and destruction tests (GitHub runner-compatible)
135135
- `cargo run --bin e2e-deployment-workflow-tests` - Software installation, configuration, release, and run workflow tests (GitHub runner-compatible)
136136
- Pre-commit hook runs the split tests (`e2e-infrastructure-lifecycle-tests` + `e2e-deployment-workflow-tests`) for GitHub Copilot compatibility
137-
- See [`docs/e2e-testing.md`](docs/e2e-testing.md) for detailed information about CI limitations
137+
- See [`docs/e2e-testing/`](docs/e2e-testing/) for detailed information about CI limitations
138138

139139
Follow the project conventions and ensure all checks pass.

‎docker/provisioned-instance/README.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -176,4 +176,4 @@ This container configuration supports the E2E test split architecture:
176176
## Related Documentation
177177

178178
- [Docker Configuration Testing Research](../../docs/research/e2e-docker-config-testing.md)
179-
- [E2E Testing Guide](../../docs/e2e-testing.md)
179+
- [E2E Testing Guide](../../docs/e2e-testing/)

‎docs/contributing/templates.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -344,4 +344,4 @@ Otherwise, use the centralized variables pattern for simplicity.
344344

345345
- **Architecture**: [`docs/technical/template-system-architecture.md`](../technical/template-system-architecture.md) - Understanding the two-phase template system
346346
- **Tera Syntax**: This document (above) - When you DO need dynamic templates with variables
347-
- **Testing**: [`docs/e2e-testing.md`](../e2e-testing.md) - How to run E2E tests to validate your changes
347+
- **Testing**: [`docs/e2e-testing/`](../e2e-testing/) - How to run E2E tests to validate your changes

‎docs/contributing/testing/testing-commands.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -201,4 +201,4 @@ if let Err(e) = run_destroy_command(&context).await {
201201
- Validate state transitions at each step
202202
- Ensure cleanup regardless of test outcome
203203

204-
For detailed E2E testing information, see [`docs/e2e-testing.md`](../../e2e-testing.md).
204+
For detailed E2E testing information, see [`docs/e2e-testing/`](../../e2e-testing/).

‎docs/decisions/register-ssh-port-override.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -184,7 +184,7 @@ Application Layer (RegisterCommandHandler)
184184

185185
- **GitHub Issue**: [#221 - Tracker Slice - Release and Run Commands](https://github.com/torrust/torrust-tracker-deployer/pull/221)
186186
- **Implementation Commit**: `f16d6cd` - feat: [#221] add optional --ssh-port argument to register command
187-
- **E2E Testing Guide**: [docs/e2e-testing.md](../e2e-testing.md)
187+
- **E2E Testing Guide**: [docs/e2e-testing/](../e2e-testing/)
188188
- **Register Command User Guide**: [docs/user-guide/commands/register.md](../user-guide/commands/register.md)
189189
- **Docker Bridge Networking**: <https://docs.docker.com/network/bridge/>
190190
- **GitHub Actions SSH Port Conflict**: SSH service on runners uses port 22 by default

‎docs/e2e-testing/README.md‎

Lines changed: 82 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,82 @@
1+
# E2E Testing Guide
2+
3+
This guide explains how to run and understand the End-to-End (E2E) tests for the Torrust Tracker Deployer project.
4+
5+
## 📖 Documentation Structure
6+
7+
- **[README.md](README.md)** - This overview and quick start guide
8+
- **[architecture.md](architecture.md)** - E2E testing architecture, design decisions, and Docker strategy
9+
- **[running-tests.md](running-tests.md)** - How to run tests, command-line options, and prerequisites
10+
- **[test-suites.md](test-suites.md)** - Detailed description of each test suite and what they validate
11+
- **[troubleshooting.md](troubleshooting.md)** - Common issues, debugging techniques, and cleanup procedures
12+
- **[contributing.md](contributing.md)** - Guidelines for extending E2E tests
13+
- **[advanced.md](advanced.md)** - Advanced techniques including manual testing and cross-environment registration
14+
15+
## 🧪 What are E2E Tests?
16+
17+
The E2E tests validate the complete deployment process using two independent test suites:
18+
19+
1. **E2E Infrastructure Lifecycle Tests** - Test infrastructure provisioning and destruction lifecycle using LXD VMs
20+
2. **E2E Deployment Workflow Tests** - Test software installation and configuration using Docker containers
21+
22+
This split approach ensures reliable testing in CI environments while maintaining comprehensive coverage.
23+
24+
## 🚀 Quick Start
25+
26+
### Run Infrastructure Lifecycle Tests
27+
28+
Test infrastructure provisioning and destruction lifecycle (VM creation, cloud-init, and destruction):
29+
30+
```bash
31+
cargo run --bin e2e-infrastructure-lifecycle-tests
32+
```
33+
34+
### Run Deployment Workflow Tests
35+
36+
Test software installation, configuration, release, and run workflows (Ansible playbooks):
37+
38+
```bash
39+
cargo run --bin e2e-deployment-workflow-tests
40+
```
41+
42+
### Run Full Local Testing
43+
44+
For local development, you can run the complete end-to-end test:
45+
46+
```bash
47+
cargo run --bin e2e-complete-workflow-tests
48+
```
49+
50+
⚠️ **Note**: The `e2e-complete-workflow-tests` binary cannot run on GitHub Actions due to network connectivity issues, but is useful for local validation.
51+
52+
## 🛠️ Quick Prerequisites Setup
53+
54+
The project provides a dependency installer tool that automatically detects and installs required dependencies:
55+
56+
```bash
57+
# Install all required dependencies
58+
cargo run --bin dependency-installer install
59+
60+
# Check which dependencies are installed
61+
cargo run --bin dependency-installer check
62+
```
63+
64+
For detailed prerequisites and manual setup, see [running-tests.md](running-tests.md).
65+
66+
## 📚 Learn More
67+
68+
- **New to E2E testing?** Start with [test-suites.md](test-suites.md) to understand what each test does
69+
- **Running into issues?** Check [troubleshooting.md](troubleshooting.md)
70+
- **Want to understand the architecture?** Read [architecture.md](architecture.md)
71+
- **Adding new tests?** See [contributing.md](contributing.md)
72+
- **Advanced workflows?** Explore [advanced.md](advanced.md)
73+
74+
## 🔗 Related Documentation
75+
76+
For information about writing unit tests and testing conventions, see:
77+
78+
- **[docs/contributing/testing/](../contributing/testing/)** - Unit testing guidelines, conventions, and best practices
79+
- **[docs/contributing/testing/unit-testing.md](../contributing/testing/unit-testing.md)** - Unit test organization and patterns
80+
- **[docs/contributing/testing/coverage.md](../contributing/testing/coverage.md)** - Test coverage guidelines
81+
82+
E2E tests focus on system-level validation of the complete deployment workflow, while unit tests validate individual components in isolation.

‎docs/e2e-testing/advanced.md‎

Lines changed: 216 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,216 @@
1+
# Advanced E2E Testing Techniques
2+
3+
This guide covers advanced testing techniques and workflows for experienced users.
4+
5+
## 🧪 Manual E2E Testing with Cross-Environment Registration
6+
7+
When manually testing the `register` command or the deployment pipeline, you can use a cross-environment technique that avoids manually provisioning VMs.
8+
9+
### The Technique
10+
11+
Use the deployer to provision one environment, then register that VM with a second environment:
12+
13+
```bash
14+
# 1. Create and provision the first environment (owns the VM)
15+
torrust-tracker-deployer --working-dir envs create environment --env-file envs/env-01.json
16+
torrust-tracker-deployer --working-dir envs provision env-01
17+
18+
# 2. Get the instance IP from env-01
19+
cat envs/data/env-01/environment.json | grep instance_ip
20+
# Example output: "instance_ip": "10.140.190.186"
21+
22+
# 3. Create the second environment and register it with env-01's VM
23+
torrust-tracker-deployer --working-dir envs create environment --env-file envs/env-02.json
24+
torrust-tracker-deployer --working-dir envs register env-02 --instance-ip 10.140.190.186
25+
26+
# 4. Test the register workflow (configure, test, destroy)
27+
torrust-tracker-deployer --working-dir envs configure env-02
28+
torrust-tracker-deployer --working-dir envs test env-02
29+
torrust-tracker-deployer --working-dir envs destroy env-02 # VM preserved!
30+
31+
# 5. Clean up the actual VM
32+
torrust-tracker-deployer --working-dir envs destroy env-01 # VM destroyed
33+
```
34+
35+
### Why This Works
36+
37+
- **env-01** has `provision_method: null` (or `Provisioned`) → destroy removes the VM
38+
- **env-02** has `provision_method: Registered` → destroy preserves the VM
39+
40+
### Use Cases
41+
42+
This technique is useful for:
43+
44+
- **Testing register command**: Without needing external infrastructure
45+
- **Verifying destroy behavior**: Confirming registered infrastructure is preserved
46+
- **Testing deployment pipeline**: On registered environments
47+
- **Rapid iteration**: Reuse same VM across multiple test cycles
48+
- **Resource efficiency**: Avoid repeated VM provisioning during development
49+
50+
### Advanced Patterns
51+
52+
#### Multiple Registered Environments
53+
54+
You can register multiple environments to the same VM:
55+
56+
```bash
57+
# Provision one VM
58+
torrust-tracker-deployer provision env-01
59+
60+
# Register multiple test environments to it
61+
torrust-tracker-deployer register env-test-a --instance-ip 10.140.190.186
62+
torrust-tracker-deployer register env-test-b --instance-ip 10.140.190.186
63+
torrust-tracker-deployer register env-test-c --instance-ip 10.140.190.186
64+
65+
# Test different configurations on same VM
66+
torrust-tracker-deployer configure env-test-a
67+
torrust-tracker-deployer configure env-test-b # Different config
68+
torrust-tracker-deployer configure env-test-c # Another config
69+
70+
# Clean up all test environments (VM preserved)
71+
torrust-tracker-deployer destroy env-test-a
72+
torrust-tracker-deployer destroy env-test-b
73+
torrust-tracker-deployer destroy env-test-c
74+
75+
# Finally destroy the VM
76+
torrust-tracker-deployer destroy env-01
77+
```
78+
79+
#### Non-Standard SSH Ports
80+
81+
Test with custom SSH ports:
82+
83+
```bash
84+
# Register with custom SSH port
85+
torrust-tracker-deployer register env-test \
86+
--instance-ip 10.140.190.186 \
87+
--ssh-port 2222
88+
89+
# All subsequent commands use the custom port automatically
90+
torrust-tracker-deployer configure env-test
91+
torrust-tracker-deployer test env-test
92+
```
93+
94+
## 🔧 Custom Template Testing
95+
96+
Test custom templates without modifying the main template directory:
97+
98+
```bash
99+
# Copy templates to a custom location
100+
cp -r templates/ /tmp/my-custom-templates/
101+
102+
# Modify templates as needed
103+
vim /tmp/my-custom-templates/ansible/playbooks/install-docker.yml
104+
105+
# Run tests with custom templates
106+
cargo run --bin e2e-deployment-workflow-tests -- \
107+
--templates-dir /tmp/my-custom-templates
108+
```
109+
110+
## 🐛 Advanced Debugging Techniques
111+
112+
### Inspect Container State During Execution
113+
114+
Use `--keep` flag and connect while tests are paused:
115+
116+
```bash
117+
# Terminal 1: Run test with keep flag
118+
cargo run --bin e2e-deployment-workflow-tests -- --keep
119+
120+
# Terminal 2: While test is running, find container
121+
docker ps
122+
123+
# Terminal 3: Connect and inspect
124+
docker exec -it <container-id> /bin/bash
125+
126+
# Inside container: check logs, validate state, etc.
127+
journalctl -u docker
128+
cat /var/log/cloud-init-output.log
129+
```
130+
131+
### LXD VM Snapshots for Debugging
132+
133+
Create snapshots at specific test stages:
134+
135+
```bash
136+
# During test execution, create snapshot
137+
lxc snapshot torrust-tracker-vm pre-configure
138+
139+
# If test fails, restore to snapshot
140+
lxc restore torrust-tracker-vm pre-configure
141+
142+
# Manually test the failing step
143+
lxc exec torrust-tracker-vm -- /bin/bash
144+
```
145+
146+
### Ansible Verbose Output
147+
148+
Enable verbose Ansible output for debugging:
149+
150+
```bash
151+
# Set environment variable before running tests
152+
export ANSIBLE_VERBOSITY=3
153+
cargo run --bin e2e-deployment-workflow-tests
154+
```
155+
156+
## 📊 Performance Analysis
157+
158+
### Measure Test Execution Time
159+
160+
```bash
161+
# Time complete test run
162+
time cargo run --bin e2e-complete-workflow-tests
163+
164+
# Time individual phases
165+
time cargo run --bin e2e-infrastructure-lifecycle-tests
166+
time cargo run --bin e2e-deployment-workflow-tests
167+
```
168+
169+
### Profile Resource Usage
170+
171+
```bash
172+
# Monitor system resources during test
173+
docker stats # For deployment workflow tests
174+
lxc info torrust-tracker-vm # For infrastructure tests
175+
```
176+
177+
## 🔄 Continuous Integration Testing
178+
179+
### Local CI Simulation
180+
181+
Simulate GitHub Actions environment locally:
182+
183+
```bash
184+
# Use act to run GitHub Actions locally
185+
act -j test-e2e-infrastructure
186+
act -j test-e2e-deployment
187+
```
188+
189+
### Parallel Test Execution
190+
191+
Run independent test suites in parallel:
192+
193+
```bash
194+
# Terminal 1
195+
cargo run --bin e2e-infrastructure-lifecycle-tests
196+
197+
# Terminal 2 (can run simultaneously)
198+
cargo run --bin e2e-deployment-workflow-tests
199+
```
200+
201+
## 🎯 Best Practices
202+
203+
1. **Use split tests for CI**: Always use infrastructure and deployment tests separately in CI
204+
2. **Complete tests locally**: Run complete workflow tests before submitting PRs
205+
3. **Debug with --keep**: Always use `--keep` flag when debugging failed tests
206+
4. **Custom templates**: Test template changes with `--templates-dir` before committing
207+
5. **Cross-environment**: Use cross-environment registration for rapid iteration
208+
6. **Snapshots**: Leverage LXD snapshots for complex debugging scenarios
209+
7. **Cleanup**: Always clean up resources after manual testing
210+
211+
## 🔗 Related Documentation
212+
213+
- [Running Tests](running-tests.md) - Basic test execution
214+
- [Troubleshooting](troubleshooting.md) - Common issues and fixes
215+
- [Architecture](architecture.md) - Understanding the test architecture
216+
- [Contributing](contributing.md) - Extending E2E tests

0 commit comments

Comments
 (0)