Shift security left

TODO

This content needs to be incorporated better in the explanation and motivation start of the page:

• DependencyCheck — verify your project dependencies against known security issues
• Find known bad code patterns — a plugin for SpotBugs
• Configuring a pipeline to use secrets (for instance using with github actions)

See also Common report formats for collating concerns.

---

Special thanks to Dan Wallach for ideas and review of this chapter.

Security as a first class concern

If you look at the badges for the project code, you may note that "Snyk security" comes immediately after build status. This is intentional. After the CI build successfully passing, there is probably no other check more important in terms of reliability than the security measures being implemented. These checks ensure that no unexpected or malicious processes are affecting your build. There are many ways that problems can arise and vulnerabilities can surface (these are just a few):

• Users may share web page links internally that could include hashcodes and passwords
• Security bugs in library dependencies often include functional bugs as well
• Corner cases for functional code often include security angles as well

It is our recommendation that you should always ensure that your project addresses security and restricted information as a first-class concern. In addition, the importance and status of security in your work should be communicated to stakeholders (for open source projects, this means the public) so that they can have confidence that your software is reliable as well. Following is our list of some of the more important considerations that we recommend you adopt, but by no means an exhaustive list.

Use secrets wisely

There are many examples of builds needing secrets such as cloud credentials, passwords, sensitive user information, etc. This project uses the OWASP_NVD_API_KEY as an example to both illustrate safe passing of secrets to local and CI builds, and how a containerized build (Earthly) can help you with this.

For Earthly and GitHub, some helpful documentation:

• https://docs.earthly.dev/docs/guides/best-practices#use-secret-not-args-to-pass-secrets-to-the-build
• https://docs.earthly.dev/docs/guides/secrets
• https://docs.github.com/en/actions/security-guides/using-secrets-in-github-actions

The sample build scripts for Gradle and Maven assume an OWASP_NVD_API_KEY environment variable — but work without one, just more slowly when updating CVEs for dependency checking.

The sample Earthfile configuration and sample command-line scripts show how to pass secrets from an environment variable into your containerized build. And the sample GitHub workflow files for CI bring the secret from your GitHub project configuration into the build pipeline.

Tip

Make sure you are passing the DependencyCheck key through from the shell environment to your build scripts, including passing through the containerized build. And update your build.gradle (build.gradle.kts) or pom.xml. You will need to update your CI configuration with this key to get the same results as local devs who have a shell export for the key.

The end result is that OWASP_NVD_API_KEY is handled as a secret when:

• Building locally directly (calling ./gradlew or ./mvnw)
• Building locally with a containerized build (Earthly)
• Building in CI directly in Gradle or Maven
• Building in CI with a containerized build

Private information

You may find that you have images in your code repository that provides revealing information about the individuals that created them. An example is GPS coordinates for a photo. The example code repo scanned the code in all previous commits, and the wiki pages, for "PII" (personal identifiable information). Screenshots from a phone or personal devices are especially prone to leaking details on individuals.

There are tools for checking images in a code repository for PII. This project uses ExifTool to check for image tags that may say more than we intend.

Checking dependencies

This is CRITICAL if you have any direct, indirect, or through-plugin dependencies on Log4j. Beyond your project, you may be impacted by services you call, so check with your organization or external services

DependencyCheck is the current best tool for JVM projects to verify that your project does not rely on external code with known security vulnerabilities (CVEs) from the NVD. That said, DependencyCheck does impact build times. It is smart about caching, but will once a day may take time to download data on any new NVD CVEs, and occasionally the site is down for maintenance. You may consider leaving this check in CI-only if you find local build times overly impacted. Leaving these checks to CI-only is a tradeoff between "shifting security left", and speed for local builds. I lean towards security first; however, you know your circumstances best.

Important

This project fails the build if finding any CVEs for the current version of any dependency.

Your build should fail, too. It is a red flag to you to consider the CVE, what impact the vulnerable dependency has, and if you are comfortable with a vulnerable dependency. It is rarely (if ever) the case you keep a vulnerable version of a dependency. However, there are also "false positives": see config/owasp-suppressions.xml for examples of handling these.

The best current tooling with Gradle or Maven uses the DependencyCheck plugin.

NVD API key

Note

As per the Attribution request from NVD: This product uses the NVD API but is not endorsed or certified by the NVD.

DependencyCheck works out of the box. However, for faster build times (especially when local), consider using an API key for pulls of NVD data (CVE security issues).

Requesting a key is simple: 0. Check if your project already has an NVD API key to use. If not:

1. Go to the linked request page.
2. Fill in the fields.
3. Accept the terms.
4. Check for an email to the address you provided.
5. In the email, use the link and save the API key in the resulting web page.
6. Include the key in your build but do not share the key and do not commit it to source.

Repeat for each separate project.

The sample code uses the OWASP_NVD_API_KEY environment variable and Gradle and Maven automatically pick this up. In single projects you can use the key in your ~/.bashrc file (loaded each time you log in):

export OWASP_NVD_API_KEY=<the key you got from NVD>

Note

I had an email conversation with NVD on how best to use the key, and their recommendation was along the lines of:

• Use separate NVD API keys for each public or shared project.
• For personal projects, it is OK to use a single key for yourself across your personal projects.

Gradle

Read more at Verifying dependencies.

Note

When running a Gradle build, you may see the dependencyCheckAnalyze task take a while. This is as the DependencyCheck plugin pulls down updates for CVEs, the plugin does not notify you of progress. (Contrast with the equivalent Maven plugin which does notify you of progress.)

Maven

Always run with the --strict-checksums (or -C) flag. See Maven Artifact Checksums - What? for more information. This is easy to forget about at the local command line. The .mvn/maven.config file helps this be automatic, and can be checked into your project repository.

An alternative is to declare each repository in your user settings.xml and set the checksum policy to "fail".

(Earthly and GitHub Actions are discussed both above.)

Automate scanning for secrets

One key to shifting security left is avoiding secrets (passwords, private identifiers, etc.) in your source code, commit history, and so on.

GitHub and other repository services offer secrets scanning out of the box: Secret scanning alerts are now available (and free) for all public repositories.

Notes

DependencyCheck may be your slowest quality check in local builds (competing with mutation testing for that ignominious title). Sometimes it may fail when the upstream source for CVEs is offline. If this is a recurring problem for you, consider moving this check into CI. The downside that local work might use an insecure dependency for a while. Checking daily for updated dependencies can lessen this risk:

• Gradle: ./gradlew dependencyUpdates
• Maven: ./mvnw versions:update-properties

For non-Windows platforms, you may see this warning when DependencyCheck runs:

.NET Assembly Analyzer could not be initialized and at least one 'exe' or 'dll' was scanned. The 'dotnet' executable could not be found on the path; either disable the Assembly Analyzer or add the path to dotnet core in the configuration.

In most situations, you are running in a Linux-based Docker container, or using local Linux or Mac command line. In a native Windows project, this is an issue to address, and may be a serious security concern indicating you are missing critical Windows components or updates. For other platforms, this is a nuisance message.

On Gradle when updating to version 7.x.x of DependencyCheck from 6.x.x or earlier, first run ./gradlew dependencyCheckPurge to clear out the local cache schema of CVEs. DependencyCheck moved to schema v2 in 7.x.x from v1 in 6.x.x and earlier, and the 7.0.0 Gradle plugin fails with the older schema version.

Verifying your project to others

Use checksums and signatures: verify what your build and project downloads! When publishing for consumption by others, provide MD5 (checksum) files in your upload: be a good netizen, and help others trust code downloaded from you.

Dependabot

GitHub provides Dependabot (other systems than GitHub may have similar robot tools) which, among other things, can automatically issue PRs to your repository when security issues are discovered. This project uses Dependabot for Gradle and Maven.

NB — Dependabot is more reliable than either the Gradle or Maven plugins for dependencies.

Tips

It is tempting to hard-code the API key into your Gradle or Maven build scripts: avoid this; better is to put it into your CI environment configuration, and help local developers in their setups. (This is a mistake I've made, and now have to figure out how to get the API key out of my git history.)

• To open the report for DependencyCheck, build locally and use the <project root>/build/reports/ (Gradle) or <project root/target/ (Maven) path. The path shown in a Docker build is relative to the interior of the container
• Sometimes you may want to refresh your local cache of the NVD files (DependencyCheck may suggest this):
  • Gradle: ./gradlew dependencyCheckPurge depenedencyCheckUpdate
  • Maven: ./mvnw dependency-check:purge dependency-check:update-only
• See Smart configuration for more discussion on configuring your API key.
• See the "Tips" section of Gradle or Maven.
• With GitHub actions, consider adding a tool such as Dependabot, which automatically files GitHub issues for known dependency vulnerabilities. See earlier in this document for an example.
• You can temporarily disable OWASP dependency checking via -Dowasp.skip=true for either Gradle or Maven, for example if the OWASP site is down for maintenance, and you cannot update the local CVE cache.
• The log4shell security vulnerabilities (CVE-2021-44228, CVE-2021-45046, CVE-2021-45105 are extremely severe. They are so severe, this should be a top priority for you to address regardless of other priorities. Although this project does not use log4j, local testing shows that the DependencyCheck plugin for either Gradle or Maven fails build when you use an older, insecure version of log4j-core indirectly. Note that Gradle 7.3.3+ itself fails your build if it detect a dependency on a vulnerable version of log4j-core.
• The BCEL security vulnerability (CVE-2022-42920) is critical, and should be a top priority to address. In this project, BCEL is used by SpotBugs: you should update to recent versions of SpotBugs that resolve the dependency on BCEL.

Tips

When Dependabot fails in a PR

Dependabot for GitHub and others may fail when running builds on PRs offered. If the problem is not on the suggested branch dependency change, you can fix the issue in your main/master branch, and then go to the PR and ask CI Dependabot to pull and retry. You do this with a comment in the PR:

@dependabot rebase

Note that this asks Depedabot to:

1. Update the PR branch with changes from the main branch.
2. Apply the PR changes against the updated main branch.
3. Rerun checks to show if the PR is easily merged.

Going further

OK, you have built software that you trust. How do you share that trust with others so they also trust your software?

A good overview comes from GitHub: Where does your software (really) come from?.