-
Notifications
You must be signed in to change notification settings - Fork 40
Enrich SBOM strategy #478
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Enrich SBOM strategy #478
Changes from all commits
Commits
Show all changes
219 commits
Select commit
Hold shift + click to select a range
cfad408
Move Sca buildinfo logic to technologies package
attiasas 00bcad9
fix breaking changes
attiasas 7c94bc1
Start removing AuditParams from BomGeneration
attiasas 67214e4
Start moving bom logic to buildinfobom file
attiasas cd62d2c
clean after logic move
attiasas 88c6409
Move test to right place
attiasas f6335ce
Replace AuditParams with BuildInfoBomGeneratorParams
attiasas 942f2f7
fix breaking changes
attiasas f7365b9
add jfrog-ignore
attiasas 9a0c26a
fix static tests
attiasas cf860e9
clean
attiasas 69284a5
fix tests
attiasas 9940c37
fix more path broken tests
attiasas 7404da9
fix last failing tests
attiasas 2f2f8b9
Merge remote-tracking branch 'upstream/dev' into move_buildinfo_sca
attiasas 0cd72bb
remove old Sbom struct in results
attiasas 129c63e
remove unused JasPackageScanType attributes and vals
attiasas 57bef1c
remove old struct and replace with cyclonedx lib, comment all related…
attiasas b1bdb36
Merge remote-tracking branch 'upstream/dev' into sbom_with_cdx
attiasas 3b929ba
add new empty methods, adjust tests to them
attiasas 7348b4c
Add PackageUrl lib and techutils for cdx conversion
attiasas 3b6f617
fix purl tests
attiasas c0fc6b1
Add diff calculation with cdx and fix related tests
attiasas 150a976
Implement Tree to Cdx for source and binary with tests
attiasas 6177413
add cdx utils
attiasas f2e62fc
rename for-each method for looping results
attiasas 4e34736
rename handlers and add for-each sbom component
attiasas 3317abf
use for-each sbom component in table format and implement the method
attiasas e3125b8
implement cdx utils for for-each sbom component
attiasas 7412c05
add tests for cdx utils
attiasas bbe919c
parse sbom table rows
attiasas ab37b52
clean
attiasas af33dec
fix go.mod
attiasas 0ee7248
fix static tests
attiasas c0a787c
fix static tests
attiasas b802ad7
Add SbomGenerator interface
attiasas 3d1d145
add params to PrepareGenerator and implement the interface as BuildIn…
attiasas 77661a2
get dynamic logic from CLI input to params. move params conversion as…
attiasas 18fd51a
prepare BOM generator in audit
attiasas 5ada683
Merge remote-tracking branch 'upstream/dev' into sbom_with_cdx
attiasas e087423
Merge remote-tracking branch 'origin/sbom_with_cdx' into sbom_generat…
attiasas 19712b8
add option to dump cdx content to file
attiasas cbfd55d
move appsConfigModule to be attribute of target and infer multiModule…
attiasas 3757ea1
Encapsulate logic in parallel runner
attiasas 217b402
Add Bom to Tree conversion for dependencies
attiasas 8f1d07d
add cdx utils
attiasas 84c8f72
create Sbom Gen unified func to use with the interface
attiasas 6687dd5
implement Sbom Gen interface for build-info (install) the current def…
attiasas 0884354
separate bom-gen stage from sca-scan stage and use SBOM for deps
attiasas 7718534
fix static tests
attiasas f98d988
fix tests
attiasas 5ba9daf
add new tests for cdx utils
attiasas 4aaf70c
add new tests for common
attiasas b2dbfe0
fix tests
attiasas e87ab43
fix more tests
attiasas 048fa68
fix more tests
attiasas cda5d05
Sbom Scan Strategy
attiasas afc6264
update interface
attiasas 5b7dda5
remove thread id in scan res count log and add to log in dump methods
attiasas e113bea
log bom component count from buildinfo if uniqueDepsWithType
attiasas d1ba913
done implementing usage of SbomScanStrategy interface
attiasas 35fd958
create empty implementation of the interface for scan graph
attiasas 6e96073
integrate the interface to audit logic
attiasas 3d09d61
implement diff with interfaces and cdx
attiasas a477133
implement scan graph interface for source code
attiasas 4458958
add direct dep and cve extraction for CA in Audit
attiasas e58c64b
CR changes
attiasas d70d01c
Merge remote-tracking branch 'origin/sbom_with_cdx' into sbom_generat…
attiasas 5b03d22
Merge remote-tracking branch 'upstream/dev' into sbom_generator_inter…
attiasas cd7826a
add default values to Purl ref or xrayCompId if not provided
attiasas 369cab0
add clean-up to bom gen interface
attiasas 9843c20
log fix and pass Scan type
attiasas 316d434
move location of downloadindexer logic
attiasas 3a4d738
detect component relation in SBOM make sure if component exists but n…
attiasas 60eb1d8
add bom to binary component tree conversion
attiasas 56aa3bb
implement BomGenerator for indexer
attiasas 5b1ed4a
integrate BomGenerator in binary scan
attiasas 114b86e
handle cleanup error in scan cmd
attiasas 87d7407
fix flaky test
attiasas f07adee
remove new flow from sca until docker is supported
attiasas 4cd5f88
Merge remote-tracking branch 'origin/sbom_generator_interface' into s…
attiasas d88585d
add log for windows test
attiasas 7487d4b
remove log for windows test
attiasas 4813938
try get relative with filepath to fix tests
attiasas 4569e46
change log to debug
attiasas b6197cb
renames
attiasas cdb5147
remove old scarunner code and move its tests to the right place
attiasas d63ffea
log better when no bom detected in buildinfo
attiasas d44f399
add scan strategy to scan and refactor JAS scan type detection
attiasas a072edd
inject strategy to all places and tests
attiasas 2f6cb57
try fix tests
attiasas fa20021
try to fix tests
attiasas c342b3f
fix tests
attiasas a5570d8
add todo for binary scan when interface will be used
attiasas 7a9726e
add new flow only cdx with empty func
attiasas 9533cdf
add to convertor new flow with empty impl
attiasas 481e555
fix tests
attiasas a3eadd3
change common parsing interface to parse Id and not split attributes
attiasas dc34887
move xray tool name to utils
attiasas bc4337f
add severity utils for cdx
attiasas e8c00ae
add cdx utils to convert scan response to enriched cdx
attiasas 0bef163
implement ScanResponseToSbom
attiasas d1a29ee
add conversion with new interface method
attiasas abf0e21
implement parsing from new flow in simple-json
attiasas c3fbbb6
return package type if unknown
attiasas 810d75b
implement parsing from new flow in table
attiasas 182fcf2
get cve for CA in new flow
attiasas 34b2046
implement parsing license in simple-json
attiasas 028fcd6
implement parsing violations in simple-json
attiasas 7585088
fix Search to return mutable reference
attiasas 840abc1
pass include license in audit
attiasas 655e31e
clean
attiasas 3b0b89c
implement parsing cdx in summary
attiasas f0b3e9a
implement parsing cdx in sarif
attiasas d2d0fb9
flx spell
attiasas c819940
CR changes
attiasas 69f5c4d
fix tests
attiasas 976962f
Merge remote-tracking branch 'origin/sbom_generator_interface' into s…
attiasas 258fb36
start fixing tests
attiasas 634105d
Add CycloneDx Output Format for command results
attiasas 9a17d00
fix output file names on windows
attiasas d4a0bc6
fix rating search and some tests
attiasas 12c538d
fix bom to flat test
attiasas 3f23bbe
fix tests
attiasas 75cc7fd
CR changes
attiasas 1c9a6dc
Merge remote-tracking branch 'upstream/dev' into sbom_generator_inter…
attiasas 32b257e
Merge remote-tracking branch 'origin/sbom_generator_interface' into s…
attiasas a8bb905
Merge remote-tracking branch 'origin/sca_scan_interface' into cyclone…
attiasas ec0d05a
Add empty CycloneDx parser
attiasas b84fea5
save raw result and output results to output and only print raw to lo…
attiasas b2cfa47
add cdx format to result writer
attiasas f220b35
extract direct or indirect cve from cdx
attiasas 952a3a7
add logs for debug ubuntu test
attiasas 8f91ca7
Merge remote-tracking branch 'origin/sca_scan_interface' into cyclone…
attiasas b3a7adf
Implement cdx parser RESET
attiasas 68db249
parse targets to metadata
attiasas 7e55927
move sarif property parsing to sarifutils
attiasas bb320a5
add iac parsing
attiasas fd297ce
parse Sbom unrelated to SCA, update flags
attiasas fc1d48f
add Sbom parsing
attiasas 6db2797
add sast parsing
attiasas 0bb04c6
Merge remote-tracking branch 'upstream/dev' into sca_scan_interface
attiasas 83f0389
fix static
attiasas 136a4f1
fix exclude logic, add tests
attiasas 6cd0d37
new flow only if not scan graph
attiasas ae4851a
fix new flow bug
attiasas c8ff4f2
fix bug - inject indexer if conditional upload
attiasas 4a4c215
try to fix win tests
attiasas fb76b4f
parse bom metadata
attiasas de2f0ea
Merge remote-tracking branch 'origin/sca_scan_interface' into cyclone…
attiasas beb001f
format and update dep
attiasas deb3736
violations not supported
attiasas 8a334c0
add secrets parsing
attiasas 9325c3b
parse new flow CVE and CA
attiasas 1e5f663
parse old flow CVE and CA
attiasas d05d9c9
parse old flow license
attiasas 55ba468
fix relative and target path handling to show relative in all places …
attiasas 441e8be
refactor to utils
attiasas 12ebce7
add options to New func
attiasas 3adbe28
Merge remote-tracking branch 'upstream/dev' into sca_scan_interface
attiasas 8f68c01
fix option handling in interfaces
attiasas 2d49b4e
add descriptors param to build info
attiasas 1658968
update has findings in sca results
attiasas d5c80af
fix static
attiasas c5ae213
fix relative and path handling in commands
attiasas a79cc7d
Merge remote-tracking branch 'origin/sca_scan_interface' into cyclone…
attiasas b1c45af
dont scan if nothing to scan
attiasas 0ac1d1e
add cdx validations
attiasas baa6b2d
add integration tests for cdx
attiasas a564261
format
attiasas bd3476d
fix test to binary
attiasas 168cc15
Merge remote-tracking branch 'upstream/dev' into sca_scan_interface
attiasas 2704cfb
Merge remote-tracking branch 'origin/sca_scan_interface' into cyclone…
attiasas 14bf41a
make sure no SCA if no components or deps
attiasas f192a36
format
attiasas ec2613c
Merge remote-tracking branch 'origin/sca_scan_interface' into cyclone…
attiasas a735af3
fix static
attiasas 3dad354
CR review
attiasas 8170daa
Merge remote-tracking branch 'origin/sca_scan_interface' into cyclone…
attiasas 91f1380
fix break after merge
attiasas 91a23d8
Add Enrich BOM SCA Scan Strategy
attiasas 897165c
create empty interface impl
attiasas 1af0012
Implement enrich strategy and use in audit
attiasas cbfae0a
update deps
attiasas 326d477
fix tests
attiasas aee13ef
fix tests
attiasas 685a92b
sort affects to eliminate flaky test
attiasas 122bafc
Merge remote-tracking branch 'origin/cyclonedx_output_format' into en…
attiasas c1450c3
CR changes
attiasas 8f54a06
Merge remote-tracking branch 'upstream/dev' into sca_scan_interface
attiasas 55574cb
Merge remote-tracking branch 'origin/sca_scan_interface' into cyclone…
attiasas b6f2457
fix flaky tests
attiasas e5579ee
Merge remote-tracking branch 'upstream/dev' into sca_scan_interface
attiasas f937ef7
Merge remote-tracking branch 'origin/sca_scan_interface' into cyclone…
attiasas 7d5392d
Merge remote-tracking branch 'origin/cyclonedx_output_format' into en…
attiasas 51436f1
Merge remote-tracking branch 'upstream/dev' into cyclonedx_output_format
attiasas 425fc80
Merge remote-tracking branch 'upstream/dev' into cyclonedx_output_format
attiasas 920aa8b
Merge remote-tracking branch 'upstream/dev' into enrich_sbom_strategy
attiasas 1d94ecd
CR changes
attiasas 6561af5
Merge remote-tracking branch 'upstream/dev' into cyclonedx_output_format
attiasas 47a5448
Merge remote-tracking branch 'origin/cyclonedx_output_format' into en…
attiasas cca9dc1
fix tests
attiasas 272ef67
fix some tests
attiasas 16cf0b4
Merge remote-tracking branch 'upstream/dev' into cyclonedx_output_format
attiasas b39b9c4
try to fix tests
attiasas c1f69e5
fix flaky old tests
attiasas 7417c16
try to fix scans_test tests in windows
attiasas 9a80f34
try to fix TestConvertResults
attiasas 3cc8b05
try to fix TestPatchRunsToPassIngestionRules
attiasas 7a8b4a1
Merge remote-tracking branch 'upstream/dev' into cyclonedx_output_format
attiasas 9f59b53
Merge remote-tracking branch 'origin/cyclonedx_output_format' into en…
attiasas 11f35ba
fix after merge
attiasas a1d07cb
fix static
attiasas f2eee54
Merge remote-tracking branch 'origin/cyclonedx_output_format' into en…
attiasas 3f8d4c5
Merge remote-tracking branch 'upstream/dev' into enrich_sbom_strategy
attiasas 89517f7
fix after merge
attiasas 331378a
remove changes after merge
attiasas abe9323
add comment to strategy
attiasas dc3e1ae
Update deps
attiasas File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,66 @@ | ||
| package enrich | ||
|
|
||
| import ( | ||
| "errors" | ||
| "fmt" | ||
|
|
||
| "github.com/CycloneDX/cyclonedx-go" | ||
|
|
||
| "github.com/jfrog/jfrog-cli-core/v2/utils/config" | ||
| "github.com/jfrog/jfrog-client-go/utils/log" | ||
| "github.com/jfrog/jfrog-client-go/xray/services" | ||
|
|
||
| "github.com/jfrog/jfrog-cli-security/sca/scan" | ||
| "github.com/jfrog/jfrog-cli-security/utils/catalog" | ||
| ) | ||
|
|
||
| type EnrichScanStrategy struct { | ||
| serverDetails *config.ServerDetails | ||
| projectKey string | ||
| } | ||
|
|
||
| func NewEnrichScanStrategy() *EnrichScanStrategy { | ||
| return &EnrichScanStrategy{} | ||
| } | ||
|
|
||
| func WithParams(serverDetails *config.ServerDetails, projectKey string) scan.SbomScanOption { | ||
| return func(sss scan.SbomScanStrategy) { | ||
| if ess, ok := sss.(*EnrichScanStrategy); ok { | ||
| ess.serverDetails = serverDetails | ||
| ess.projectKey = projectKey | ||
| } | ||
| } | ||
| } | ||
|
|
||
| func (ess *EnrichScanStrategy) WithOptions(options ...scan.SbomScanOption) scan.SbomScanStrategy { | ||
| for _, option := range options { | ||
| option(ess) | ||
| } | ||
| return ess | ||
| } | ||
|
|
||
| func (ess *EnrichScanStrategy) PrepareStrategy() (err error) { | ||
| catalogManager, err := catalog.CreateCatalogServiceManager(ess.serverDetails, catalog.WithScopedProjectKey(ess.projectKey)) | ||
| if err != nil { | ||
| return fmt.Errorf("failed to create catalog service manager: %w", err) | ||
| } | ||
| catalogVersion, err := catalogManager.GetVersion() | ||
| if err != nil { | ||
| return fmt.Errorf("failed to get catalog version: %w", err) | ||
| } | ||
| log.Debug(fmt.Sprintf("Catalog version: %s", catalogVersion)) | ||
| return | ||
| } | ||
|
|
||
| func (ess *EnrichScanStrategy) SbomEnrichTask(target *cyclonedx.BOM) (enriched *cyclonedx.BOM, violations []services.Violation, err error) { | ||
hadarshjfrog marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| catalogManager, err := catalog.CreateCatalogServiceManager(ess.serverDetails, catalog.WithScopedProjectKey(ess.projectKey)) | ||
| if err != nil { | ||
| return nil, []services.Violation{}, fmt.Errorf("failed to create catalog service manager: %w", err) | ||
| } | ||
| enriched, err = catalogManager.Enrich(target) | ||
| return | ||
| } | ||
|
|
||
| func (ess *EnrichScanStrategy) DeprecatedScanTask(target *cyclonedx.BOM) (techResults services.ScanResponse, err error) { | ||
hadarshjfrog marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| return services.ScanResponse{}, errors.New("EnrichScanStrategy does not support DeprecatedScanTask") | ||
| } | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,45 @@ | ||
| package catalog | ||
|
|
||
| import ( | ||
| "github.com/jfrog/jfrog-cli-core/v2/utils/config" | ||
| "github.com/jfrog/jfrog-cli-core/v2/utils/coreutils" | ||
| "github.com/jfrog/jfrog-client-go/catalog" | ||
| clientconfig "github.com/jfrog/jfrog-client-go/config" | ||
| ) | ||
|
|
||
| // Options for creating an Catalog service manager. | ||
| type CatalogManagerOption func(f *catalog.CatalogServicesManager) | ||
|
|
||
| // Global reference to the project key, used for API endpoints that require it for authentication | ||
| func WithScopedProjectKey(projectKey string) CatalogManagerOption { | ||
| return func(f *catalog.CatalogServicesManager) { | ||
| f.SetProjectKey(projectKey) | ||
| } | ||
| } | ||
|
|
||
| func CreateCatalogServiceManager(serverDetails *config.ServerDetails, options ...CatalogManagerOption) (manager *catalog.CatalogServicesManager, err error) { | ||
| certsPath, err := coreutils.GetJfrogCertsDir() | ||
| if err != nil { | ||
| return | ||
| } | ||
| catalogDetails, err := serverDetails.CreateCatalogAuthConfig() | ||
| if err != nil { | ||
| return | ||
| } | ||
| serviceConfig, err := clientconfig.NewConfigBuilder(). | ||
| SetServiceDetails(catalogDetails). | ||
| SetCertificatesPath(certsPath). | ||
| SetInsecureTls(serverDetails.InsecureTls). | ||
| Build() | ||
| if err != nil { | ||
| return | ||
| } | ||
| manager, err = catalog.New(serviceConfig) | ||
| if err != nil { | ||
| return nil, err | ||
| } | ||
| for _, option := range options { | ||
| option(manager) | ||
| } | ||
| return | ||
| } |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.