-
-
Notifications
You must be signed in to change notification settings - Fork 237
ENH: Built-in flight comparison tool (FlightComparator) to validate simulations against external data
#888
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
ENH: Built-in flight comparison tool (FlightComparator) to validate simulations against external data
#888
Conversation
|
Hello! Apologies that this implementation turned out to be quite exhaustive; it wasn't an easy feature to implement properly as it required a dedicated class structure to handle the time interpolation, error metrics, and data standardization correctly. The implementation could probably use some optimization. I completely understand that this is a large PR and might take a while to go over and review. Take your time, and let me know if you need me to break anything down! I also anticipate that with a feature this size, there may be some initial linting issues or edge cases I missed. I am standing by to push follow-up commits to fix those as soon as the checks run. |
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## develop #888 +/- ##
===========================================
+ Coverage 80.27% 80.65% +0.38%
===========================================
Files 104 107 +3
Lines 12769 13289 +520
===========================================
+ Hits 10250 10718 +468
- Misses 2519 2571 +52 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
Gui-FernandesBR
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Great work, I'm happy with the results already.
Please solve all the commts, update the CHANGELOG file and then we will merge it when it's ready.
I don't think this one will take a long time to be finished or reviewed.
The implementation seems pretty clean.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull request overview
This PR introduces a comprehensive Flight Comparison architecture that enables users to validate RocketPy simulations against external data sources (avionics logs, OpenRocket, RAS Aero). The implementation automates data interpolation, error metric calculation, and visualization generation through a new FlightComparator class.
Key Changes:
- New
FlightComparatorclass inrocketpy/simulation/flight_comparator.pyproviding automated comparison capabilities with error metrics (RMSE, MAE, Max Deviation) and key event analysis - Convenience method
Flight.compare()added to enable one-line comparison workflows - Comprehensive test suite with 20+ unit tests covering initialization, data formats, plotting, and edge cases
Reviewed changes
Copilot reviewed 3 out of 4 changed files in this pull request and generated 11 comments.
| File | Description |
|---|---|
rocketpy/simulation/flight_comparator.py |
New FlightComparator class implementing data ingestion, interpolation, metric calculation, and visualization methods |
rocketpy/simulation/flight.py |
Added convenience compare() method to Flight class for easy access to comparison functionality |
tests/unit/simulation/test_flight_comparator.py |
Comprehensive unit tests covering all FlightComparator methods and edge cases |
docs/user/flight_comparator.rst |
Empty documentation file requiring content |
|
I just started a new set of reviews with Copilot. @Monta120 please address (and mark as After that, I believe we will be ready to merge this PR |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull request overview
Copilot reviewed 8 out of 8 changed files in this pull request and generated 11 comments.
tests/integration/simulation/test_flight_comparator_workflow.py
Outdated
Show resolved
Hide resolved
|
@Gui-FernandesBR All the comments have been addressed. |
|
Great work @Monta120 !!! |
FlightComparator) to validate simulations against external data
Pull request type
Checklist
black rocketpy/ tests/) has passed locallypytest tests -m slow --runslow) have passed locallyCHANGELOG.mdhas been updated (if relevant)Current behavior
Currently, users who want to compare RocketPy simulations against real flight data (avionics logs) or other simulators (OpenRocket, RAS Aero) face significant obstacles. They must:
1- Manually extract arrays from RocketPy objects.
2- Write custom code to interpolate their external data to match RocketPy's time steps (or vice versa) to calculate errors.
3- Write custom Matplotlib code to visualize the differences.
New behavior
This PR introduces a built-in Flight Comparison architecture that automates the validation process.
Ingests external data (dictionaries of tuples or RocketPy Functions).
Automatically Interpolates disparate datasets onto a shared time grid (1,000 points) to allow point-by-point comparison.
Calculates error metrics: RMSE (Root Mean Square Error), MAE (Mean Absolute Error), and Max Deviation.
Generates comparison tables for Key Events (Apogee, Max Velocity, Impact Velocity) with relative error percentages.
Comparison Plots: Standardized plots showing Simulation vs. External Data (Top) and Residuals/Error (Bottom).
2D Trajectories: Visual comparison of flight paths (e.g., X vs Z) to analyze drift.
Breaking change
Additional information
Testing: Added exhaustive unit tests (tests/unit/simulation/test_flight_comparator.py) covering initialization, data conversion, metric calculation, and plotting logic.