This repository was archived by the owner on Nov 7, 2024. It is now read-only.
JSON format support #1
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…sues Conflicts: .gitignore
Do the download.sh process to get these updates only.
First cut on an experiment to parse the json format from the NSIT json zip files. This is the new format and the xml format is being deprecated by NIST.
TODO edit the hard coded zero to a proper iteration
Reference the source of this code
Input file is displayed, code is able to find a match on a hard coded value.
Place the json data into a python set object for later use of the set interection method for searching specific entries.
Create two sets(). One holds the list of packages the other holds the entire list of data from the NIST JSON database. Use the set.intersection() to do a database join operation to find a match.
Given the sets can be managed with the intersection function, now produce output showing the CVE ID and description of the matching CVE entry.
Formatted output for one CVE with one description then white space. Making it similar to the xml format.
Output format had redundant data. Need to match on version strings exactly is in the works.
Output format much closer to the original xml reports. Contains the CVE ID followed by the description.
The ignore list input file contains CVE identifiers that the user specifically ignores due to mitigation. The output is thus filtered as either <skipped> if the CVE ID is found in the ignore list or it is listed as <warning> if not ignored.
Exact match on the input version number for a specific package. So now when the input contains php-php-5.4.3 the exact match will only find CVE IDs for 5.4.3 and not 5.4.3X
Corrected output format to remove duplicates. Output format is cleaned up.
Testing showed each trial results in a different sorting on the output list. Implement a proper sorting so each iteration results in the same output given the same input.
Once I started the json_cve_parser.py in the Jenkins machine the python interpreter failed indicating a character encoding error. Imported the parser into Notepad, exported it out, and verified UTF-8 format and Unix line endings. Scripts that end with _json are used to run the front end in the Jenkins environment thus searching for CVE issues in the Space, Dosetrac, and Com 3 projects.
Yup the database download issue is resolved. Typo.
The checking side of the process was failing due to the presences of an unexpected file called utfile. Removed it. Need to make the checking program robust so it only examines zip files not all files.
Json format
XML support deprecated by NIST
Adjust the readme to clarify the project. Set file permissions to execute.
Error was thrown: UnicodeEncodeError: 'ascii' codec can't encode character u'\xc2' in position 301: ordinal not in range(128) Turns out the strings are managed as ascii by default so need to encode them ast utf 8 instead.
Google site console verification of ownership
Made the how-to generic
Refactored the basic implementation to use SQLite. Create various SQL tables to store data and form relationships. This provides the ability to do efficient wildcard searching.
CVE definition is now up to 20 characters long according to my observation
Corrected a strange error related to encoding.
Sort the results in a single file
Support NIST JSON format version 1.1
Support NIST JSON 1.1
# Conflicts: # scan_for_vulnerabilities_json.sh
Support for the Space Com 2 WB45 SBOM
CVE-2018-10511 is missing the expected cpe_match field which broke the parser. Added error checking and handling for this possiblity.
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
1 participant
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
JSON supported
NIST deprecated the previous XML format in favor of JSON format.