Skip to content

Clean up of import history #13776

@daanschipper

Description

@daanschipper

Is your feature request related to a problem? Please describe
We have a job running every 5 minutes which creates a vulnerability report which in turn gets reimported into DefectDojo via /reimport. The findings get correctly deduplicated and deleted with the 'Delete duplications' option, but the import history seems to grow infinitely.

The screenshots show the large number of imports of three tests. The size of these tables seems to grow significantly.

                     table_full_name                      |    size                                                                                                                                                                                                                                                                                                                                                                      
----------------------------------------------------------+------------
 public.watson_searchentry                                | 19 GB
 public.dojo_test_import_finding_action                   | 19 GB
Image

The tests in question have quite a significant number of findings, that also contributes to the size.

Describe the solution you'd like
It should be able to limit the import history, similar to the number of duplicate findings. Or not store empty import history.

Describe alternatives you've considered
The TRACK_IMPORT_HISTORY option, however that option will maybe be removed and has other side effects: #13312.

Additional context
Using version 2.51.2.
Slack thread: https://owasp.slack.com/archives/C2P5BA8MN/p1764060461585449.

From the slack conversation, to lower the size of the above tables (followed with a vacuum in case of postgres):

  • dojo_test_import_finding_action can be safely deleted
  • Run ./manage.py buildwatson to rebuild watson_searchentry

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions