Different findings in cbomkit-action vs. sonarcrypto plugin

[san-zrl]

I ran a little experiment to compare the scanning results between cbomkit-action and the sonar plugin.

Test repo was keycloak-test, a fork of keycloak that I created some time ago. Its not quite in sync with the main keycloak repo but recent enough to serve as a test basis.

I used two different variants of cbomkit-action. One variant, called cbomkit-action, uses the currently favoured approach of scanning packages separately and then creating a consolidated cbom by merging the package cboms. The other variant, called cbomkit-action-alternative, runs one scan over all package modules and then extracts package cboms from the overall cbom.

By setting sonar.java.binaries, all three test cases pass class files generated during the build step to the scan function.

Results

• cbomkit-action: 139 findings
• cbomkit-action-alternative: 143 findings
• sonarcrypto plugin: 148 findings

Differences

The different number of findings is due to the different handling of cross-package dependencies.

Case 1: Internal cross -package dependencies

An example is the line below (AbstractX509ClientCertificateAuthenticator.java#L185-L185)

return Hex.encodeHexString(HashUtils.hash(JavaAlgorithm.SHA256, certs[0].getEncoded()));

A SHA156-finding is only generated if JavaAlgorithm and the SHA256 constant can be resolved. In the above case the target type is defined in org.keycloak.crypto.JavaAlgorithm. This is a cross-package dependency on a class file that is created when building the project.

Case 2: External cross -package dependencies

A more challenging case is a cross-package dependency on a class file that is provided by an external dependency. These class files are not generated as part of the build. An example is (HttpClientBuilder.java#L272-L272)

final SSLContext tlsContext = SSLContext.getInstance(SSLSocketFactory.TLS);

Here the scanner must be able to resolve the javax.net.ssl.SSLContext class which is not part of the keycloak repo.

Discussion

Sonarcypto plugin generated findings for example 1 und 2. cbomkit-action-alternative generates a finding for example 1 but not 2. cbomkit-action does not find any of the examples.

Apparently, Sonarcrypto plugin can resolve both internal and external dependencies. cbomkit-action-alternative can only handle internal dependencies. cbomkit-action does currently not resolve any cross package dependencies. It seems that due to the total scan of all packages cbomkit-action-alternative has access to the AST of internal packages (and passing class files via sonar.java.binaries does not make a difference). Sonarcrypto plugin must somehow also see symbols provided by external packages.

[san-zrl]

This is normal behavior since the java scanning result depends on the scanner's ability to resolve symbols defined in dependencies. Added a note to README.md recommending to build java code prior to scanning and to specify additional dependencies (see PR #76).