@@ -22,6 +22,29 @@ To select the appropriate test type, use this decision table:
2222| ** Crypto** | Encryption, secrets, key management | Fast (ms) |
2323| ** Mutation** | Test quality verification, 70%+ coverage | CI/Release |
2424
25+ ### Test Organization Rationale
26+
27+ Organize tests ** by feature, not by test type** . This improves discoverability and maintainability:
28+
29+ ```
30+ Tests/
31+ ├── Unit/
32+ │ └── Domain/Model/UserTest.php # Tests User model logic
33+ ├── Functional/
34+ │ └── Repository/UserRepositoryTest.php # Tests User persistence
35+ └── E2E/
36+ └── User/Registration.spec.ts # Tests User registration flow
37+ ```
38+
39+ ** Why by feature?**
40+ - Finding tests: "Where are User tests?" → Look in User-related directories
41+ - Understanding coverage: All tests for a feature are co-located
42+ - Maintenance: When changing User logic, related tests are easy to find
43+
44+ ** Why NOT by test type alone?**
45+ - "All unit tests" doesn't help you understand what's tested
46+ - Scattered tests across unrelated features are hard to maintain
47+
2548## Setting Up Test Infrastructure
2649
2750To initialize testing infrastructure for an extension, run:
@@ -42,6 +65,25 @@ To generate a new test file, run:
4265scripts/generate-test.sh < TestType> < ClassName>
4366```
4467
68+ ### Test Verification Workflow (MANDATORY)
69+
70+ After creating or modifying a test, you ** MUST** verify it works by running the test suite:
71+
72+ 1 . ** Run the new test** to confirm it catches the bug or validates the behavior
73+ 2 . ** Verify failure state** (if TDD): The test should FAIL before the fix is applied
74+ 3 . ** Apply the fix** then re-run to confirm the test now PASSES
75+ 4 . ** Run the full suite** to ensure no regressions
76+
77+ ``` bash
78+ # Run only the new test first
79+ Build/Scripts/runTests.sh -s unit -- --filter UserValidatorTest
80+
81+ # After fix is applied, run the full suite
82+ Build/Scripts/runTests.sh -s unit
83+ ```
84+
85+ ** Why this matters:** A test that never fails provides no value. Always verify your test catches the bug before the fix AND passes after.
86+
4587## Running Tests
4688
4789To execute tests via the Docker-based runner, use these commands:
@@ -118,6 +160,60 @@ steps:
118160- run : ddev exec vendor/bin/phpunit
119161` ` `
120162
163+ ## Troubleshooting Test Failures
164+
165+ ### E2E Tests Fail
166+
167+ When E2E tests fail, debug systematically:
168+
169+ | Symptom | Likely Cause | Fix |
170+ |---------|--------------|-----|
171+ | **Timeout on page load** | TYPO3 not started, wrong URL | Check ` TYPO3_BASE_URL` env var, verify `php -S` is running |
172+ | **Element not found** | Page not rendered, JS error | Add `await page.waitForLoadState('networkidle')`, check browser console |
173+ | **Login fails** | Missing fixture, wrong credentials | Verify `be_users.csv` fixture loaded, check password hash |
174+ | **Screenshot shows blank page** | PHP error, 500 response | Check `var/log/typo3_*.log`, enable debug mode |
175+ | **Works locally, fails in CI** | See CI debugging section below | Environment differences |
176+
177+ **Debugging steps:**
178+
179+ 1. **Capture screenshot on failure** (Playwright does this automatically)
180+ 2. **Check Playwright trace** for network requests : ` npx playwright show-trace trace.zip`
181+ 3. **Verify TYPO3 is accessible** : ` curl -I $TYPO3_BASE_URL`
182+ 4. **Check TYPO3 logs** : ` cat .Build/Web/var/log/typo3_*.log`
183+
184+ # ## Tests Pass Locally But Fail in CI
185+
186+ This is a common frustration. Use this checklist :
187+
188+ | Check | Local vs CI Difference | Resolution |
189+ |-------|------------------------|------------|
190+ | **PHP version** | Local may differ from CI matrix | Ensure local PHP matches CI target |
191+ | **Database state** | Local has data, CI starts fresh | Add missing fixtures to test setup |
192+ | **File permissions** | Local user differs from CI runner | Avoid hardcoded paths, use `sys_get_temp_dir()` |
193+ | **Timing** | Local is fast, CI is slow | Add explicit waits, avoid `sleep()` |
194+ | **Environment vars** | Local `.env`, CI lacks it | Define all required vars in CI workflow |
195+ | **Extensions loaded** | Local has extra PHP extensions | Check `php -m` output in CI logs |
196+ | **Filesystem case** | macOS case-insensitive, Linux case-sensitive | Fix `require 'MyClass.php'` vs `myclass.php` |
197+
198+ **CI debugging workflow:**
199+
200+ ` ` ` bash
201+ # 1. Reproduce locally with CI-like conditions
202+ docker run --rm -it php:8.3-cli php -m # Check extensions
203+
204+ # 2. Add debug output to failing test
205+ $this->markTestSkipped('DEBUG: ' . var_export($actualValue, true));
206+
207+ # 3. Check CI logs for environment differences
208+ # Look for: PHP version, loaded extensions, env vars
209+
210+ # 4. Use GitHub Actions debug logging
211+ env:
212+ ACTIONS_STEP_DEBUG: true
213+ ` ` `
214+
215+ **Golden rule:** If tests pass locally but fail in CI, the bug is in your test's assumptions about the environment, not in the CI.
216+
121217# # Using Reference Documentation
122218
123219# ## Core Testing References
0 commit comments