-
Notifications
You must be signed in to change notification settings - Fork 0
Open
Labels
category: performancePerformance relatedPerformance relatedpriority: mediumViktigt men inte kritisktViktigt men inte kritiskttestingTesting relatedTesting related
Milestone
Description
π§ͺ Regression Testing Improvements
π Issue Description
The current regression testing suite has some issues when running on macOS instead of the target Raspberry Pi environment. While the architecture validation is successful (93.8% success rate), the tests need improvements for better cross-platform compatibility.
π Current Issues
- Environment-specific failures: Tests fail on macOS due to missing Linux-specific services (nginx, systemd, mosquitto)
- API endpoint URL construction: Some tests still have double '/api/api/' in URLs
- Test environment assumptions: Tests assume services are running locally
β Already Fixed
- HTML API reference: β Fixed (now contains '/api/status')
- Basic regression test endpoints: β Fixed
π― Proposed Solutions
1. Environment Detection
import platform
import os
def is_raspberry_pi():
"""Detect if running on Raspberry Pi"""
try:
with open('/proc/cpuinfo', 'r') as f:
return 'Raspberry Pi' in f.read()
except:
return False
def should_skip_service_tests():
"""Skip service tests on non-Linux systems"""
return platform.system() != 'Linux' or not is_raspberry_pi()2. Mock Services for Testing
class MockServiceTester:
"""Mock service responses for testing"""
def test_nginx_mock(self):
# Mock nginx response
pass
def test_api_server_mock(self):
# Mock API server response
pass3. Conditional Test Execution
@pytest.mark.skipif(not is_raspberry_pi(), reason="Requires Raspberry Pi environment")
def test_systemd_services(self):
# Only run on actual Raspberry Pi
passπ Current Test Results
- Total Tests: 40
- Passed: 26 β (65% raw, 93.8% architecture validation)
- Failed: 14 β (mostly environment-related)
- Architecture Validation: β SUCCESSFUL
π― Success Criteria
- Tests pass on macOS (development environment)
- Tests pass on Raspberry Pi (production environment)
- Clear separation between environment and architecture tests
- Better test reporting and categorization
π Files to Update
python/v3/tests/test_regression_comprehensive.pypython/v3/tests/test_regression_comprehensive_fixed.py- Add new test utilities for cross-platform compatibility
π Priority
Medium - Architecture is sound, but testing experience could be improved for developers working on macOS.
π― Acceptance Criteria
- Regression tests pass on macOS
- Clear distinction between environment and architecture failures
- Better test reporting with environment context
- Documentation for running tests in different environments
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
category: performancePerformance relatedPerformance relatedpriority: mediumViktigt men inte kritisktViktigt men inte kritiskttestingTesting relatedTesting related