Plan storing of multiple advisories with same AVID

[TG1999]

We will have advisories in future which will have same identifer and source but it's content may change in future due to changes in upstream or we have better our parsing capabilities.

for example:

github_importer_v2/GHSA-1234-567

This advisory can change in the future at upstream or we can start parsing more available data.

So we need to decide how we are going to store and show data in UI and API.

IMO we should do this:

• we shall show latest advisory in both UI and API
• have a history endpoint in API to show all revisions/changes of that AVID
• have a history tab in UI to show all all revisions/changes of that AVID

[DennisClark]

@TG1999 your suggestions make sense to me. Question from a database perspective: when an update to an advisory is discovered, do we then copy our current advisory to an "AVID history" table, and then replace that with the new one? I am also assuming that the history table is indexed by AVID/date to handle multiple entries. You probably have already figured out something like that, so I'm just wanting to clarify those details.

[TG1999]

@DennisClark excellent question

so for example github_importer_v2/GHSA-1234-567 has 2 references today, and if in upstream the same advisory now has 3 references. We will store both. We have a field called unique_content_id which is a hash on the parsed advisory content.

And we have a unique_together on the Advisory like this:

• "datasource_id", "advisory_id", "unique_content_id"

So for say we have these advisories:

• "github_importer_v2", "GHSA-1234-567", "abcd" (assumed hash)
• "github_importer_v2", "GHSA-1234-567", "efgh" (new hash, because of the changes of content)

The unique content hash helps us to identify/differentiate difference in data.

So we will not maintain a AVID history table as of now, rather use "AVID" to get all advisories and sort them on the date of creation

[DennisClark]

@TG1999 thanks for the explanation, which makes sense to me.

[ziadhany]

@TG1999 I agree with you that we should have a history and show the latest advisories, but this can be problematic for some importers like EPSS. EPSS generates about 306,930 advisories, and it gets updated daily with new ones. This could overwhelm the advisory list.

[TG1999]

@ziadhany are all those 300kish advisories are updated daily? Or some of them are updated ?

[ziadhany]

@TG1999 I think a big portion will change, as they return two numerical values in two fields, for example:

cve         "CVE-2019-16759"
epss        "0.968170000"
percentile  "0.999990000"
date        "2022-04-28"

example : https://www.cvedetails.com/epss/CVE-2024-39689/epss-score-history.html

[TG1999]

@ziadhany that's fine. It's not that all the data is updated daily and we shall store all this data and the updations.