|
| 1 | +--- |
| 2 | +applyTo: 'tests/test_*.py' |
| 3 | +--- |
| 4 | +## For AI Agents and Senior Developers |
| 5 | + |
| 6 | +This document provides comprehensive guidelines for writing high-quality, |
| 7 | +behavior-driven pytest tests for the ArduPilot Methodic Configurator project. |
| 8 | +These guidelines ensure consistency, maintainability, and comprehensive test |
| 9 | +coverage while following industry best practices. |
| 10 | + |
| 11 | +## Core Testing Philosophy |
| 12 | + |
| 13 | +### 1. Senior Developer Mindset |
| 14 | + |
| 15 | +- Write tests that **future developers will thank you for** |
| 16 | +- Focus on **behavior over implementation details** |
| 17 | +- Prioritize **maintainability and readability** |
| 18 | +- Use **minimal, strategic mocking** |
| 19 | +- Apply **DRY principles consistently** |
| 20 | + |
| 21 | +### 2. Behavior-Driven Development (BDD) |
| 22 | + |
| 23 | +- Write tests that describe **user behavior** and **business value**, |
| 24 | + not implementation details |
| 25 | +- Use Given-When-Then structure in test descriptions and code comments |
| 26 | +- Focus on **what** the system should do, not **how** it does it |
| 27 | +- Test names should read like specifications: |
| 28 | + `test_user_can_select_template_by_double_clicking` |
| 29 | + |
| 30 | +### 3. Test Isolation and Independence |
| 31 | + |
| 32 | +- Each test should be completely independent and able to run in any order |
| 33 | +- Use fixtures with appropriate scopes (`function`, `class`, `module`, `session`) |
| 34 | +- Clean up resources properly after tests complete |
| 35 | +- Avoid shared mutable state between tests |
| 36 | + |
| 37 | +All tests MUST follow this structure: |
| 38 | + |
| 39 | +```python |
| 40 | +def test_descriptive_behavior_name(self, fixture_name) -> None: |
| 41 | + """ |
| 42 | + Brief summary of what the test validates. |
| 43 | +
|
| 44 | + GIVEN: Initial system state and preconditions |
| 45 | + WHEN: The action or event being tested |
| 46 | + THEN: Expected outcomes and assertions |
| 47 | + """ |
| 48 | + # Arrange (Given): Set up test data and mocks |
| 49 | + |
| 50 | + # Act (When): Execute the behavior being tested |
| 51 | + |
| 52 | + # Assert (Then): Verify expected outcomes |
| 53 | +``` |
| 54 | + |
| 55 | +## π§ Fixture Guidelines |
| 56 | + |
| 57 | +### Required Fixture Pattern |
| 58 | + |
| 59 | +Create reusable fixtures that eliminate code duplication: |
| 60 | + |
| 61 | +```python |
| 62 | +@pytest.fixture |
| 63 | +def mock_vehicle_provider() -> MagicMock: |
| 64 | + """Fixture providing a mock vehicle components provider with realistic test data.""" |
| 65 | + provider = MagicMock() |
| 66 | + |
| 67 | + # Create properly structured mock data matching real object interfaces |
| 68 | + template_1 = MagicMock() |
| 69 | + template_1.attributes.return_value = ["name", "fc", "gnss"] |
| 70 | + template_1.name = "QuadCopter X" |
| 71 | + template_1.fc = "Pixhawk 6C" |
| 72 | + template_1.gnss = "Here3+" |
| 73 | + |
| 74 | + provider.get_vehicle_components_overviews.return_value = { |
| 75 | + "Copter/QuadX": template_1, |
| 76 | + } |
| 77 | + return provider |
| 78 | + |
| 79 | +@pytest.fixture |
| 80 | +def configured_window(mock_vehicle_provider) -> ComponentWindow: |
| 81 | + """Fixture providing a properly configured window for behavior testing.""" |
| 82 | + with ( |
| 83 | + patch("tkinter.Toplevel"), |
| 84 | + patch.object(BaseWindow, "__init__", return_value=None), |
| 85 | + # Only patch what's necessary for the test |
| 86 | + ): |
| 87 | + window = ComponentWindow(vehicle_components_provider=mock_vehicle_provider) |
| 88 | + window.root = MagicMock() |
| 89 | + return window |
| 90 | +``` |
| 91 | + |
| 92 | +### Fixture Design Rules |
| 93 | + |
| 94 | +1. **One concern per fixture** - Each fixture should have a single responsibility |
| 95 | +2. **Realistic mock data** - Use data that mirrors real system behavior |
| 96 | +3. **Composable fixtures** - Allow fixtures to depend on other fixtures |
| 97 | +4. **Descriptive names** - Fixture names should clearly indicate their purpose |
| 98 | +5. **Minimal scope** - Use the narrowest scope possible (function > class > module > session) |
| 99 | + |
| 100 | +## π Test Structure Standards |
| 101 | + |
| 102 | +### Test Class Organization |
| 103 | + |
| 104 | +```python |
| 105 | +class TestUserWorkflow: |
| 106 | + """Test complete user workflows and interactions.""" |
| 107 | + |
| 108 | + def test_user_can_complete_primary_task(self, configured_window) -> None: |
| 109 | + """ |
| 110 | + User can successfully complete the primary workflow. |
| 111 | + |
| 112 | + GIVEN: A user opens the application |
| 113 | + WHEN: They follow the standard workflow |
| 114 | + THEN: They should achieve their goal without errors |
| 115 | + """ |
| 116 | + # Implementation |
| 117 | +``` |
| 118 | + |
| 119 | +### Test Method Requirements |
| 120 | + |
| 121 | +1. **Descriptive names** - `test_user_can_select_template_by_double_clicking` |
| 122 | +2. **Single behavior focus** - Test one behavior per method |
| 123 | +3. **User-centric language** - Write from the user's perspective |
| 124 | +4. **Complete documentation** - Include summary and GIVEN/WHEN/THEN |
| 125 | + |
| 126 | +### Assertions |
| 127 | + |
| 128 | +- Use **specific assertions** over generic ones |
| 129 | +- **Test behavior outcomes** not implementation details |
| 130 | +- **Group related assertions** logically |
| 131 | +- **Include meaningful failure messages** when helpful |
| 132 | + |
| 133 | +```python |
| 134 | +# Good - Tests behavior |
| 135 | +assert window.selected_template == "Copter/QuadX" |
| 136 | +assert window.close_window.called_once() |
| 137 | + |
| 138 | +# Bad - Tests implementation |
| 139 | +assert mock_method.call_count == 1 |
| 140 | +``` |
| 141 | + |
| 142 | +## π Mocking Strategy |
| 143 | + |
| 144 | +### Minimal Mocking Principle |
| 145 | + |
| 146 | +Only mock what is **absolutely necessary**: |
| 147 | + |
| 148 | +```python |
| 149 | +# Good - Minimal mocking |
| 150 | +def test_template_selection(self, template_window) -> None: |
| 151 | + template_window.tree.selection.return_value = ["item1"] |
| 152 | + template_window._on_selection_change(mock_event) |
| 153 | + assert template_window.selected_template == "expected_value" |
| 154 | + |
| 155 | +# Bad - Over-mocking |
| 156 | +def test_template_selection(self) -> None: |
| 157 | + with patch("tkinter.Tk"), \ |
| 158 | + patch("module.Class1"), \ |
| 159 | + patch("module.Class2"), \ |
| 160 | + patch("module.method1"), \ |
| 161 | + patch("module.method2"): |
| 162 | + # Test lost in mocking complexity |
| 163 | +``` |
| 164 | + |
| 165 | +### Mocking Guidelines |
| 166 | + |
| 167 | +1. **Mock external dependencies** (file system, network, databases) |
| 168 | +2. **Mock UI framework calls** (tkinter widgets, events) |
| 169 | +3. **Don't mock the system under test** - Test real behavior |
| 170 | +4. **Use fixtures for complex mocks** - Keep test methods clean |
| 171 | +5. **Mock return values realistically** - Match expected data types and structures |
| 172 | + |
| 173 | +## ποΈ Project-Specific Patterns |
| 174 | + |
| 175 | +### Frontend Tkinter Testing |
| 176 | + |
| 177 | +```python |
| 178 | +# Use template_overview_window_setup fixture for complex UI mocking |
| 179 | +def test_ui_behavior(self, template_overview_window_setup) -> None: |
| 180 | + """Test UI behavior without full window creation.""" |
| 181 | + |
| 182 | +# Use template_window fixture for component testing |
| 183 | +def test_component_behavior(self, template_window) -> None: |
| 184 | + """Test component behavior with configured window.""" |
| 185 | +``` |
| 186 | + |
| 187 | +### Backend/Logic Testing |
| 188 | + |
| 189 | +```python |
| 190 | +# Use minimal mocking for business logic |
| 191 | +def test_business_logic(self, mock_data_provider) -> None: |
| 192 | + """Test core logic with realistic data.""" |
| 193 | + |
| 194 | +# Mock only external dependencies |
| 195 | +@patch('module.external_api_call') |
| 196 | +def test_integration_behavior(self, mock_api) -> None: |
| 197 | + """Test integration points.""" |
| 198 | +``` |
| 199 | + |
| 200 | +## π Test Categories |
| 201 | + |
| 202 | +### Required Test Types |
| 203 | + |
| 204 | +1. **User Workflow Tests** - Complete user journeys |
| 205 | +2. **Component Behavior Tests** - Individual component functionality |
| 206 | +3. **Error Handling Tests** - Graceful failure scenarios |
| 207 | +4. **Integration Tests** - Component interaction validation |
| 208 | +5. **Edge Case Tests** - Boundary conditions and unusual inputs |
| 209 | + |
| 210 | +### Test Organization |
| 211 | + |
| 212 | +```text |
| 213 | +tests/ |
| 214 | +βββ test_frontend_tkinter_component.py # UI component tests |
| 215 | +βββ test_backend_logic.py # Business logic tests |
| 216 | +βββ test_integration_workflows.py # End-to-end scenarios |
| 217 | +βββ conftest.py # Shared fixtures |
| 218 | +``` |
| 219 | + |
| 220 | +## π Quality Standards |
| 221 | + |
| 222 | +### Test Quality Metrics |
| 223 | + |
| 224 | +- **Coverage**: Aim for 80%+ on core modules |
| 225 | +- **Maintainability**: Tests should be easy to modify |
| 226 | +- **Speed**: Test suite should run in < 2 minutes |
| 227 | +- **Reliability**: Zero flaky tests allowed |
| 228 | + |
| 229 | +### Code Review Checklist |
| 230 | + |
| 231 | +- [ ] Tests follow GIVEN/WHEN/THEN structure |
| 232 | +- [ ] Fixtures used instead of repeated setup |
| 233 | +- [ ] Minimal, strategic mocking applied |
| 234 | +- [ ] User-focused test names and descriptions |
| 235 | +- [ ] All edge cases covered |
| 236 | +- [ ] Error scenarios tested |
| 237 | +- [ ] Performance considerations addressed |
| 238 | + |
| 239 | +## π Example: Complete Test Implementation |
| 240 | + |
| 241 | +```python |
| 242 | +class TestTemplateSelection: |
| 243 | + """Test user template selection workflows.""" |
| 244 | + |
| 245 | + def test_user_can_select_template_by_double_clicking(self, template_window) -> None: |
| 246 | + """ |
| 247 | + User can select a vehicle template by double-clicking on the tree item. |
| 248 | + |
| 249 | + GIVEN: A user views available vehicle templates |
| 250 | + WHEN: They double-click on a specific template row |
| 251 | + THEN: The template should be selected and stored |
| 252 | + AND: The window should close automatically |
| 253 | + """ |
| 254 | + # Arrange: Configure template selection behavior |
| 255 | + template_window.tree.identify_row.return_value = "template_item" |
| 256 | + template_window.tree.item.return_value = {"text": "Copter/QuadX"} |
| 257 | + |
| 258 | + # Act: User double-clicks on template |
| 259 | + mock_event = MagicMock(y=100) |
| 260 | + template_window._on_row_double_click(mock_event) |
| 261 | + |
| 262 | + # Assert: Template selected and workflow completed |
| 263 | + template_window.program_settings_provider.store_template_dir.assert_called_once_with("Copter/QuadX") |
| 264 | + template_window.root.destroy.assert_called_once() |
| 265 | + |
| 266 | + def test_user_sees_visual_feedback_during_selection(self, template_window) -> None: |
| 267 | + """ |
| 268 | + User receives immediate visual feedback when selecting templates. |
| 269 | + |
| 270 | + GIVEN: A user is browsing available templates |
| 271 | + WHEN: They click on a template row |
| 272 | + THEN: The corresponding vehicle image should be displayed immediately |
| 273 | + """ |
| 274 | + # Arrange: Set up selection behavior |
| 275 | + template_window.tree.selection.return_value = ["selected_item"] |
| 276 | + template_window.tree.item.return_value = {"text": "Plane/FixedWing"} |
| 277 | + |
| 278 | + with patch.object(template_window, "_display_vehicle_image") as mock_display: |
| 279 | + # Act: User selects template |
| 280 | + mock_event = MagicMock() |
| 281 | + template_window._on_row_selection_change(mock_event) |
| 282 | + template_window._update_selection() # Simulate callback |
| 283 | + |
| 284 | + # Assert: Visual feedback provided |
| 285 | + mock_display.assert_called_once_with("Plane/FixedWing") |
| 286 | +``` |
| 287 | + |
| 288 | +## π οΈ Development Workflow |
| 289 | + |
| 290 | +### Pre-commit Requirements |
| 291 | + |
| 292 | +1. **Run tests**: `pytest tests/ -v` |
| 293 | +2. **Check coverage**: `pytest --cov=ardupilot_methodic_configurator --cov-report=term-missing` |
| 294 | +3. **Lint with ruff**: `ruff check --fix` |
| 295 | +4. **Type check with mypy**: `mypy .` |
| 296 | +5. **Advanced type check with pyright**: `pyright` |
| 297 | +6. **Style check with pylint**: `pylint $(git ls-files '*.py')` |
| 298 | + |
| 299 | +### Test Execution Commands |
| 300 | + |
| 301 | +```bash |
| 302 | +# Run all tests with verbose output |
| 303 | +pytest tests/ -v |
| 304 | + |
| 305 | +# Run tests with coverage reporting |
| 306 | +pytest tests/ --cov=ardupilot_methodic_configurator --cov-report=html |
| 307 | + |
| 308 | +# Run specific test file |
| 309 | +pytest tests/test_frontend_tkinter_template_overview.py -v |
| 310 | + |
| 311 | +# Run tests matching pattern |
| 312 | +pytest tests/ -k "test_user" -v |
| 313 | + |
| 314 | +# Run tests with performance timing |
| 315 | +pytest tests/ --durations=10 |
| 316 | +``` |
| 317 | + |
| 318 | +### Debugging Failed Tests |
| 319 | + |
| 320 | +```bash |
| 321 | +# Run with detailed output |
| 322 | +pytest tests/test_file.py::test_method -v -s |
| 323 | + |
| 324 | +# Run with pdb debugging |
| 325 | +pytest tests/test_file.py::test_method --pdb |
| 326 | + |
| 327 | +# Run with custom markers |
| 328 | +pytest tests/ -m "slow" -v |
| 329 | +``` |
| 330 | + |
| 331 | +## π Quality Assurance |
| 332 | + |
| 333 | +### Final Validation Steps |
| 334 | + |
| 335 | +Before submitting any test changes: |
| 336 | + |
| 337 | +1. **Verify all tests pass**: `pytest tests/ -v` |
| 338 | +2. **Confirm coverage maintained**: Check coverage report |
| 339 | +3. **Lint compliance**: `ruff check` (must pass) |
| 340 | +4. **Type safety**: `mypy .` (must pass) |
| 341 | +5. **Pyright validation**: `pyright` (should pass) |
| 342 | +6. **Style compliance**: `pylint $(git ls-files '*.py')` (address major issues) |
| 343 | + |
| 344 | +### Success Criteria |
| 345 | + |
| 346 | +- β
All tests pass consistently |
| 347 | +- β
Coverage β₯ 80% on modified modules |
| 348 | +- β
Zero ruff/mypy violations |
| 349 | +- β
Tests follow behavior-driven structure |
| 350 | +- β
Fixtures eliminate code duplication |
| 351 | +- β
User-focused test descriptions |
| 352 | +- β
Minimal, strategic mocking |
| 353 | + |
| 354 | +--- |
| 355 | + |
| 356 | +**Remember**: Great tests are an investment in the future. Write tests that make the codebase more maintainable, not just achieve coverage metrics. |
0 commit comments