Remove pulls trigger #450
Remove pulls trigger #450
Conversation
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## main #450 +/- ##
==========================================
- Coverage 90.58% 89.97% -0.61%
==========================================
Files 401 324 -77
Lines 12509 9199 -3310
Branches 2103 1633 -470
==========================================
- Hits 11331 8277 -3054
+ Misses 1069 859 -210
+ Partials 109 63 -46
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. |
Swatinem
left a comment
Swatinem
left a comment
There was a problem hiding this comment.
I’m unsure related to the should_write_to_storage changes. I don’t think those are needed necessarily?
shared/django_apps/core/models.py
Outdated
| if self.state != PullStates.OPEN.value: | ||
| # while a pull is OPEN, we check whether to write_to_storage | ||
| # when a pull is no longer OPEN, we no longer want the flare in storage. | ||
| # The nightly cron job cleans up the value in storage (see FlareCleanupTask in worker) | ||
| return False |
There was a problem hiding this comment.
this would rather mean that the flare is being stored in the JSON field of the table.
I’m unsure what would happen if you change this value based on the state. will is copy over data from storage to the database? or will that only happen if you save a new value in the field?
There was a problem hiding this comment.
good call - I think I was thinking that if this came back False it wouldn't be saved anywhere, but you're right, this would just force it onto the db field. Removing.
| except ClientError: | ||
| raise | ||
|
|
||
| def delete_files(self, bucket_name, paths=[]): |
There was a problem hiding this comment.
This default value is dangerous and unnecessary. fixed the occurrences I found across the project.
nora-shap
force-pushed
the
nora/1029
branch
4 times, most recently
from
741151b to
2331c4a
Compare
| if repository: | ||
| self.storage_hash = self.get_archive_hash(repository) | ||
| else: | ||
| self.storage_hash = None |
There was a problem hiding this comment.
Making ArchiveService dependent on a repository was complicating my flare cleanup task.
It seems like repository is only needed to make the hash for reading and writing to the archive - you don't need it to delete.
To make my flare cleanup task much lighter, I removed the requirement for a repository in order to use ArchiveService. I added checks to the ArchiveStorage functions where the hash is used, so that you can't do something like write to storage without a hash.
Part of the quest to reduce locks by eliminating triggers https://github.com/codecov/internal-issues/issues/1029
The trigger is deleted by these changes. The behavior is replaced in a cron job on worker codecov/worker#947.
What the trigger did: every time a pull was updated, if it was transitioning from
OPENto any otherstate, it set theflarefield tonull.flareis an Archive field, so the value is either in_flareon the object or in our Archive storage.Replacement behavior: see
FlareCleanupTaskcodecov/worker#947Once the trigger is removed, when the pull transitions from
OPENto any otherstate, leaveflarealone in order to reduce locks and wait time.The cron job runs overnight, gets all pulls that are non-
OPENand have flare (either in our db or in Archive storage), clears it (either in our db or in Archive storage) soflareisnull.more about
flare: