perf(api): replace JOINs with pre-check in threat score aggregation query#10394
Conversation
|
Please add an entry to the corresponding |
|
✅ Conflict Markers Resolved All conflict markers have been successfully resolved in this pull request. |
There was a problem hiding this comment.
Pull request overview
This PR aims to improve performance of the ThreatScore requirement-statistics aggregation by avoiding expensive JOINs in the main aggregation query and removing COUNT(DISTINCT) now that the query is intended to avoid row-multiplication sources.
Changes:
- Adds a pre-check to short-circuit aggregation when the scan’s provider is deleted, allowing removal of provider/resource JOIN filters in the main query.
- Simplifies the aggregation
Count()calls by removingdistinct=True. - Adds an API changelog entry documenting the query optimization.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
api/src/backend/tasks/jobs/threatscore_utils.py |
Refactors the ORM aggregation to remove resource/provider JOIN filters and adds a provider-deleted pre-check. |
api/CHANGELOG.md |
Documents the ThreatScore aggregation query performance change in an UNRELEASED section. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
You can also share your feedback on Copilot code review. Take the survey.
🔒 Container Security ScanImage: 📊 Vulnerability Summary
4 package(s) affected
|
…eted-provider pre-check
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #10394 +/- ##
===========================================
+ Coverage 56.86% 93.37% +36.50%
===========================================
Files 87 219 +132
Lines 2847 30415 +27568
===========================================
+ Hits 1619 28399 +26780
- Misses 1228 2016 +788
Flags with carried forward coverage won't be shown. Click here to find out more.
🚀 New features to boost your workflow:
|
…eted-provider pre-check - Modify api/CHANGELOG.md entry
💚 All backports created successfully
Questions ?Please refer to the Backport tool documentation and see the Github Action logs for details |
Description
Eliminate 3 unnecessary JOINs (resource_finding_mappings, resources, providers) from the threat score aggregation query by moving the provider
is_deletedcheck to a separate pre-check. Also removesCOUNT(DISTINCT)since the M2M traversal that could produce duplicate rows is gone.Steps to review
Run tests and run a local scan from the UI.
Checklist
API
License
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.