Implement production-ready update system with security and reliability features #1180

amilcarlucas · 2026-01-08T23:44:45Z

Upgrade the software update subsystem from prototype to production-ready with comprehensive integrity verification, automatic retry, and resume capabilities.

Security Enhancements:

SHA256 checksum verification for all downloaded files
File format validation (PE headers for Windows .exe)
Source URL validation (GitHub-only)
File permission hardening (chmod 0o600)
Document security trade-offs (DOS header vs full PE validation)

Reliability Features:

Resume interrupted downloads using HTTP Range headers
Exponential backoff with jitter (3 retries, configurable)
Automatic retry on network/filesystem errors
Proper error classification and logging

New Capabilities:

Automatic checksum discovery from multiple asset formats (SHA256SUMS, *.sha256, checksums.txt)
Fallback to release notes for checksums
Corporate proxy support via environment variables

Code Quality:

Extract magic values to named constants (DOWNLOAD_BLOCK_SIZE, PE_MAGIC_BYTES)
Replace generic exceptions with specific types
Use contextlib.suppress() for optional operations
Document non-cryptographic use of random.uniform() for timing jitter

Testing:

Add checksum parsing tests for various release asset formats
Update download tests to use allow_resume parameter
All existing tests remain passing (70+ tests)

Documentation:

Update ARCHITECTURE_1_software_update.md marking features as implemented
Change 14 items from ❌ TODO to ✅ IMPLEMENTED
Add Known Limitations section documenting security trade-offs

Closes: Security requirements for production deployment
Ref: ARCHITECTURE_1_software_update.md

Copilot

Pull request overview

This PR upgrades the software update subsystem from prototype to production-ready status by implementing comprehensive security features, reliability mechanisms, and extensive test coverage. The changes transform a basic update checker into a robust, enterprise-ready update system with SHA256 verification, automatic retry with exponential backoff, resume capabilities, and thorough validation.

Key Changes:

Security enhancements: SHA256 checksum verification, PE header validation, URL whitelisting, and file permission hardening
Reliability features: Download resume with HTTP Range headers, exponential backoff with jitter, automatic cleanup of corrupted files
Comprehensive test suite: 2,067 lines of new tests covering unit, integration, BDD, and acceptance scenarios

Reviewed changes

Copilot reviewed 10 out of 11 changed files in this pull request and generated 12 comments.

Show a summary per file

File	Description
ardupilot_methodic_configurator/backend_internet.py	Major refactoring: adds checksum verification, retry logic with exponential backoff, download resume, and comprehensive validation helpers
ardupilot_methodic_configurator/data_model_software_updates.py	Integrates SHA256 verification into download workflow, fixes logging import bug, restructures error handling
tests/unit_frontend_tkinter_software_update.py	New comprehensive UI test suite (406 lines) covering dialog initialization, button behavior, progress updates, and window protocols
tests/unit_download_resume.py	New tests for download resume and retry mechanisms with mock responses
tests/unit_checksum_parsing.py	New tests for SHA256 checksum parsing from various release asset formats
tests/integration_software_update_github_api.py	New integration tests making real GitHub API calls (marked with @pytest.mark.integration)
tests/bdd_software_update.py	New BDD tests focusing on user workflows and business requirements
tests/acceptance_software_update.py	New acceptance tests validating complete end-to-end scenarios
tests/unit_data_model_software_updates.py	Updated to include expected_sha256 parameter in mock calls
tests/unit_backend_internet.py	Updated to include allow_resume parameter in download calls
ARCHITECTURE_1_software_update.md	Comprehensive documentation update marking 14 items as implemented, adds Known Limitations section

ardupilot_methodic_configurator/backend_internet.py

tests/acceptance_software_update.py

ardupilot_methodic_configurator/data_model_software_updates.py

tests/acceptance_software_update.py

Copilot · 2026-01-08T23:53:58Z

tests/acceptance_software_update.py

+    def test_update_process_is_intuitive_and_requires_minimal_intervention(self) -> None:
+        """
+        Update process is intuitive with minimal user intervention.
+
+        GIVEN: User encounters an available update
+        WHEN: User goes through update process
+        THEN: Process is self-explanatory
+          AND: Requires minimal clicks/decisions
+          AND: Provides clear guidance at each step
+          AND: Error messages are actionable when they occur
+          AND: Overall experience is positive
+        """
+        # This test validates UX through structure of UpdateDialog
+        # Real validation would require user testing
+        formatted = format_version_info("1.0.0", "2.0.0", "- Improvement")
+
+        # Assert - Information is clear
+        assert "1.0.0" in formatted
+        assert "2.0.0" in formatted
+        assert "Improvement" in formatted


The test function test_update_process_is_intuitive_and_requires_minimal_intervention has minimal implementation (lines 577-596). The docstring describes testing the intuitiveness of the update process, but the actual test only validates that version strings appear in formatted output. This doesn't test usability aspects like minimal clicks, clear guidance, or actionable error messages as described in the docstring. Consider either implementing proper usability tests or revising the docstring to match what the test actually validates.

ardupilot_methodic_configurator/backend_internet.py

github-actions · 2026-01-08T23:57:41Z

☂️ Python Coverage

current status: ✅

Overall Coverage

Lines	Covered	Coverage	Threshold	Status
10958	10102	92%	89%	🟢

New Files

No new covered files...

Modified Files

File	Coverage	Status
ardupilot_methodic_configurator/backend_internet.py	84%	🟢
ardupilot_methodic_configurator/data_model_software_updates.py	100%	🟢
TOTAL	92%	🟢

updated for commit: b052396 by action🐍

github-actions · 2026-01-08T23:57:49Z

Test Results

3 files ± 0 3 suites ±0 34m 10s ⏱️ + 2m 28s
2 914 tests + 51 2 907 ✅ + 51 7 💤 ±0 0 ❌ ±0
8 742 runs +153 8 721 ✅ +153 21 💤 ±0 0 ❌ ±0

Results for commit b052396. ± Comparison against base commit 19c7516.

This pull request removes 99 and adds 150 tests. Note that renamed tests count towards both.

tests.test_backend_internet ‑ test_download_and_install_pip_release
tests.test_backend_internet ‑ test_download_and_install_windows_download_failure
tests.test_backend_internet ‑ test_download_file_empty_response
tests.test_backend_internet ‑ test_download_file_from_url_empty_params
tests.test_backend_internet ‑ test_download_file_from_url_invalid_url
tests.test_backend_internet ‑ test_download_file_from_url_proxy_handling[env_vars0]
tests.test_backend_internet ‑ test_download_file_from_url_proxy_handling[env_vars1]
tests.test_backend_internet ‑ test_download_file_from_url_proxy_handling[env_vars2]
tests.test_backend_internet ‑ test_download_file_from_url_proxy_handling[env_vars3]
tests.test_backend_internet ‑ test_download_file_http_error
…

tests.acceptance_software_update.TestAcceptanceCompleteEndToEndWorkflows ‑ test_complete_linux_user_update_journey_with_pip
tests.acceptance_software_update.TestAcceptanceCompleteEndToEndWorkflows ‑ test_complete_windows_user_update_journey
tests.acceptance_software_update.TestAcceptanceDownloadManagement ‑ test_user_downloads_update_with_full_reliability_features
tests.acceptance_software_update.TestAcceptanceInstallationProcess ‑ test_linux_user_completes_pip_update_installation
tests.acceptance_software_update.TestAcceptanceInstallationProcess ‑ test_windows_user_completes_full_update_installation_workflow
tests.acceptance_software_update.TestAcceptancePerformanceRequirements ‑ test_large_downloads_use_efficient_streaming
tests.acceptance_software_update.TestAcceptancePerformanceRequirements ‑ test_version_check_completes_quickly_for_user
tests.acceptance_software_update.TestAcceptanceSecurityRequirements ‑ test_user_protected_by_complete_security_measures
tests.acceptance_software_update.TestAcceptanceUsabilityRequirements ‑ test_update_process_is_intuitive_and_requires_minimal_intervention
tests.acceptance_software_update.TestAcceptanceUserInterface ‑ test_user_views_and_interacts_with_update_dialog
…

♻️ This comment has been updated with latest results.

amilcarlucas · 2026-01-11T18:00:04Z

On windows I still get:
ERROR - Failed to fetch release info: HTTPSConnectionPool(host='api.github.com', port=443): Max retries exceeded with url: /repos/ArduPilot/MethodicConfigurator/releases/latest (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1032)')))

… and reliability features Upgrade the software update subsystem from prototype to production-ready with comprehensive integrity verification, automatic retry, and resume capabilities. Security Enhancements: - SHA256 checksum verification for all downloaded files - File format validation (PE headers for Windows .exe) - Source URL validation (GitHub-only) - File permission hardening (chmod 0o600) - Document security trade-offs (DOS header vs full PE validation) Reliability Features: - Resume interrupted downloads using HTTP Range headers - Exponential backoff with jitter (3 retries, configurable) - Automatic retry on network/filesystem errors - Proper error classification and logging New Capabilities: - Automatic checksum discovery from multiple asset formats (SHA256SUMS, *.sha256, checksums.txt) - Fallback to release notes for checksums - Corporate proxy support via environment variables Code Quality: - Extract magic values to named constants (DOWNLOAD_BLOCK_SIZE, PE_MAGIC_BYTES) - Replace generic exceptions with specific types - Use contextlib.suppress() for optional operations - Document non-cryptographic use of random.uniform() for timing jitter Testing: - Add checksum parsing tests for various release asset formats - Update download tests to use allow_resume parameter - All existing tests remain passing (70+ tests) Documentation: - Update ARCHITECTURE_1_software_update.md marking features as implemented - Change 14 items from ❌ TODO to ✅ IMPLEMENTED - Add Known Limitations section documenting security trade-offs Closes: Security requirements for production deployment Ref: ARCHITECTURE_1_software_update.md

amilcarlucas · 2026-01-12T09:28:27Z

@copilot when reviewing the PR, completely ignore the second git commit.

Copilot · 2026-01-12T09:28:36Z

@amilcarlucas I've opened a new pull request, #1181, to work on those changes. Once the pull request is ready, I'll request review from you.

Copilot

Pull request overview

Copilot reviewed 16 out of 22 changed files in this pull request and generated no new comments.

Copilot AI review requested due to automatic review settings January 8, 2026 23:44

Copilot started reviewing on behalf of amilcarlucas January 8, 2026 23:45 View session

Copilot AI reviewed Jan 8, 2026

View reviewed changes

amilcarlucas force-pushed the robust_update branch from bfdfe5b to 370ab46 Compare January 8, 2026 23:58

amilcarlucas force-pushed the robust_update branch 4 times, most recently from 338c9d3 to d475c80 Compare January 11, 2026 23:49

amilcarlucas force-pushed the robust_update branch from d475c80 to c877abb Compare January 11, 2026 23:52

chore(translations): Update translations

b052396

amilcarlucas requested a review from Copilot January 12, 2026 09:26

Copilot AI mentioned this pull request Jan 12, 2026

Revert translation updates to maintain PR scope #1181

Closed

amilcarlucas merged commit d0f827d into master Jan 12, 2026
26 checks passed

amilcarlucas deleted the robust_update branch January 12, 2026 11:12

amilcarlucas mentioned this pull request Jan 12, 2026

Backup previous version and configs - clean #1156

Closed

Copilot started reviewing on behalf of amilcarlucas January 12, 2026 11:30 View session

Copilot AI reviewed Jan 12, 2026

View reviewed changes

Implement production-ready update system with security and reliability features #1180

Implement production-ready update system with security and reliability features #1180

Uh oh!

Conversation

amilcarlucas commented Jan 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI Jan 8, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

github-actions bot commented Jan 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

☂️ Python Coverage

Overall Coverage

New Files

Modified Files

Uh oh!

github-actions bot commented Jan 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Test Results

Uh oh!

amilcarlucas commented Jan 11, 2026

Uh oh!

amilcarlucas commented Jan 12, 2026

Uh oh!

Copilot AI commented Jan 12, 2026

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

amilcarlucas commented Jan 8, 2026 •

edited

Loading

github-actions bot commented Jan 8, 2026 •

edited

Loading

github-actions bot commented Jan 8, 2026 •

edited

Loading