CVE and CPE datasets filtered

[J08nY]

The CVE and CPE datasets (and CPE match) may not correspond to what our users expect from them as they are filtered:

sec-certs/src/sec_certs/dataset/cve.py

Lines 179 to 186 in 3d885d0

	def enhance_with_nvd_data(self, data: dict[str, Any]) -> CVEDataset:
	self.last_update_timestamp = datetime.fromisoformat(data["timestamp"])
	for vuln in data["vulnerabilities"]:
	# https://nvd.nist.gov/vuln/vulnerability-status#divNvdStatus
	if vuln["cve"]["vulnStatus"] in {"Analyzed", "Modified"}:
	cve = CVE.from_nist_dict(vuln["cve"])
	self[cve.cve_id] = cve
	return self

sec-certs/src/sec_certs/dataset/cpe.py

Lines 102 to 115 in 3d885d0

	def enhance_with_nvd_data(self, nvd_data: dict[Any, Any]) -> None:
	self.last_update_timestamp = datetime.fromisoformat(nvd_data["timestamp"])
	cpes_to_deprecate: set[str] = set()
	for cpe in nvd_data["products"]:
	if cpe["cpe"]["deprecated"]:
	cpes_to_deprecate.add(cpe["cpe"]["cpeNameId"])
	else:
	new_cpe = CPE.from_nvd_dict(cpe["cpe"])
	self.cpes[new_cpe.uri] = new_cpe
	uris_to_delete = self._find_uris_for_ids(cpes_to_deprecate)
	for uri in uris_to_delete:
	del self[uri]

sec-certs/src/sec_certs/utils/nvd_dataset_builder.py

Lines 341 to 353 in 3d885d0

	if m["matchString"]["status"] == "Inactive":
	inactive_criteria.add(m["matchString"]["matchCriteriaId"])
	else:
	if "matches" in m["matchString"]:
	dataset_to_fill["match_strings"][m["matchString"]["matchCriteriaId"]] = {
	"criteria": m["matchString"]["criteria"],
	"matches": m["matchString"]["matches"],
	}
	for version_key in self._VERSION_KEYS:
	if version_key in m["matchString"]:
	dataset_to_fill["match_strings"][m["matchString"]["matchCriteriaId"]][version_key] = m[
	"matchString"
	][version_key]

This is because having these CVEs and CPEs does not make sense for the purposes of matching them to affected certificates as they are either invalid or miss required information. However, this is quite a suprising thing considering our users may use the CVEDataset or CPEDataset for different purposes. Also, this means that existing CVEs and CPEs do not get a page on the web and a user gets a suprising 404 Not Found.

We could let the dataset have all of the entries and filter them during the matching phase instead. This would likely also require changing the objects so that they contain the information that is being filtered upon (i.e. the CVE/CPE status).