create db vacuum service to periodically delete old prefect resources

[levzem]

Closes #16054

This PR adds a new LoopService called DBVacuum that allows for the automatic clean up of Prefect resources that are older than a configurable retention period. This is to help keep the Prefect DB from overflowing with old flow runs/task runs/etc.

The DBVacuum deletes the following:

1. flow runs
2. associated logs
3. associated artifacts

Task runs are handled by the DELETE CASCADE rules setup on the flow run table.

The DBVacuum queries a limited amount of parent flow runs (those that do not have a parent_task_run_id) that are older than the retention and deletes them. It then deletes logs that no longer have any flow runs associated with them and finally performs the same deletion for artifacts.

The deletion of a parent flow run will delete child task runs and set their child flow runs parent_task_run_id to null, which will get picked up on the next round of the loop. This gives a guarantee that eventually all flow runs and their sub flow runs will be deleted (along with the associated logs and artifacts).

Checklist

• This pull request references any related issue by including "closes <link to issue>"
  • If no issue exists and your change is not a small fix, please create an issue first.
• If this pull request adds new functionality, it includes unit tests that cover the changes
• If this pull request removes docs files, it includes redirect settings in mint.json.
• If this pull request adds functions or classes, it includes helpful docstrings.

[codspeed-hq]

CodSpeed Performance Report

Merging #19280 will not alter performance

_{Comparing levzem:lev/vacuum (7225337) with main (7fcfb7e)}

Summary

✅ 2 untouched

[levzem]

@stephane-additive fyi

[zzstoatzz]

hi @levzem - i appreciate the PR and have a couple initial thoughts:

• something like this should exist
• we should not introduce it as default on, as that would cause significant changes in behavior for server operators
• we should probably get aligned in a github issue (that proposes a design, not just the original motivation of cleaning up the DB) so as to reduce back and forth on the PR

can you create an issue where you lay out your design here and why? this would be a non-trivial introduction to the server

[levzem]

@zzstoatzz thanks for the quick turnaround.

I agree it should be off by default, that was a copy pasta mistake on my end.

I have tagged you in a design proposal in #16054. Look forward to hearing your thoughts.

[zzstoatzz]

great thank you @levzem - marking this as a draft for the time being! will give feedback on the issue

[github-actions]

This pull request is stale because it has been open 14 days with no activity. To keep this pull request open remove stale label or comment.

[github-actions]

This pull request was closed because it has been stale for 14 days with no activity. If this pull request is important or you have more to add feel free to re-open it.

[zzstoatzz]

hi @levzem - sorry for the delay here

if you're willing, can you check out the new background services implemented in docket?

notably, LoopService has been totally removed, and you can instead use a perpetual docket task

[levzem]

@zzstoatzz thanks! I will return to this PR and migrate it to Docket

[github-actions]

This pull request is stale because it has been open 14 days with no activity. To keep this pull request open remove stale label or comment.

[github-actions]

This pull request was closed because it has been stale for 14 days with no activity. If this pull request is important or you have more to add feel free to re-open it.