Black Duck SCA Scan Yocto Script - `bd_scan_yocto_via_sbom.py` v1.2.1

Provision of This Script

This script is provided under the MIT license (see LICENSE file).

It does not extend the licensed functionality of Black Duck Software. It is provided as-is, without warranty or liability.

For comments or issues, please raise a GitHub issue in this repository. Black Duck Support cannot respond to support tickets for this open-source utility. Users are encouraged to engage with the authors to address any identified issues.

Refer to How to Report Issues below for the minimum required information to report issues with this script.

Introduction

Overview of `bd_scan_yocto_via_sbom`

This script is for licensed users of Black Duck Software only. You will need a Black Duck SCA server and an API key to use it.

While Black Duck Detect is the default scan utility for Black Duck SCA and supports Yocto projects, its Yocto scans have limitations. A Detect scan will only identify standard recipes from OpenEmbedded.org and does not cover:

Modified or custom recipes.
Recipes moved to new layers or with changed package versions/revisions.
Custom kernel identification or kernel vulnerability mapping.
Copyright and deep license data.
Snippet or other code scanning within custom recipes.

This utility addresses these gaps by creating a comprehensive Black Duck SCA project from your Yocto build. It achieves this by:

Scanning Yocto project artifacts to generate an SPDX SBOM (Software Bill of Materials), which is then uploaded to your Black Duck server to create a project version (--modes OE_RECIPES)
Filtering and "fixing-up" recipes using data from the OpenEmbedded (OE) APIs to correctly identify local recipes moved to new layers or with different local versions/revisions.
Signature scanning packages and downloaded archives for recipes not matched by OE data (with the option to scan all packages - --modes SIG_SCAN or --modes SIG_SCAN_ALL)
Optionally adding unmatched components via CPE lookup or by creating custom components through a second SBOM upload (--modes CPE_COMPS and/or --modes CUSTOM_COMPS)
Applying patches for locally patched CVEs identified from cve_check data (if available - --modes CVE_PATCHES)
Optionally processing the Linux Kernel to filter out vulnerabilities not within the sources compiled into your specific kernel build (--modes IMAGE_MANIFEST,KERNEL_VULNS)

For guidance on optimizing Yocto project scans with this utility, refer to the Best Practice Recommendations section below.

Note the addition of new --modes parameter in v1.2.0 to control scans to be performed (legacy scan parameters still supported) - see Scan Mode section below.

Understanding Yocto and Why This Script is Needed

Yocto is a powerful, local build system for creating custom Linux images. It's not a package manager but a highly customizable environment. Key characteristics that make comprehensive scanning challenging include:

No Centralized Repository: Unlike other Linux distributions, Yocto downloads software packages from external locations at build time.
Recipe Misconception: Yocto recipe names do not fully identify software packages. Standard recipes are provided as templates but are only local text files in a project and can be modified to download different software from anywhere - with the potential for supply chain hacking.
Extensive Customization: Yocto projects can be heavily modified, with unlimited changes to the build environment, layers, recipes, and the addition of custom recipes and patches.
Hidden Dependencies: Packages within recipes can embed other open-source software not explicitly listed in the Yocto manifest.
Post-Build Analysis: Due to Yocto's dynamic nature, it's only feasible to accurately determine what was built and packaged into an image after the build is complete.

This script employs multiple techniques to reverse-engineer a built Yocto project to identify its included packages, report license information, and map vulnerabilities from the package origins.

The script can also identify and process the Linux kernel, determining its custom configuration and modules to ignore vulnerabilities reported against excluded modules.

Prerequisites

Before running the script, ensure you meet the following requirements:

Yocto v2.1 or newer
Python 3.10 or newer
Black Duck SCA server 2024.7 or newer
Black Duck SCA API Token with either:
- Global Code Scanner and Global Project Manager roles, OR
- Project Code Scanner and BOM Manager roles for an existing project.
- (Also Global Component Manager needed to create custom components if --mode CUSTOM_COMPS specified)
Single-target Bitbake configurations only are supported. Run this utility on one target at a time.
A built Yocto project with access to the build platform. Alternatively, specific outputs from the build can be used, though many script features may be limited so not recommended.

Installation

You have a few options for installing this utility:

Option 1: Install from PyPI (Recommended)

Create a virtual environment:

python3 -m venv venv
source venv/bin/activate # On Windows: .\venv\Scripts\activate

Install the package:

pip3 install bd_scan_yocto_via_sbom --upgrade

Option 2: Build and Install Locally

Clone the repository:

git clone <repository-url>
cd bd_scan_yocto_via_sbom

Create a virtual environment (if you haven't already):

python3 -m venv venv
source venv/bin/activate # On Windows: .\venv\Scripts\activate

Build the utility:
```
python3 -m build
```
Install the package:
```
pip3 install dist/bd_scan_yocto_via_sbom-1.0.X-py3-none-any.whl --upgrade
```
(Replace 1.0.X with the actual version number of the built wheel file.)

Option 3: Clone Repository and Install Prerequisites

Clone the repository:

git clone <repository-url>
cd bd_scan_yocto_via_sbom

Ensure prerequisite packages are installed (see the pyproject.toml file for a list).
```
pip install .
```

How to Run

If you installed the utility as a package:

Set the Bitbake environment (e.g., source oe-init-build-env).
Activate your virtual environment (e.g., source venv/bin/activate).
Run the script:
```
bd-scan-yocto-via-sbom PARAMETERS
```

If you cloned the repository locally:

Set the Bitbake environment (e.g., source oe-init-build-env).
Activate your virtual environment where dependency packages were installed (e.g., source venv/bin/activate).

Run the script:

python3 PATH_TO_REPOSITORY/run.py PARAMETERS

Best Practice Recommendations

For optimal Yocto scan results, consider the following:

Check required scan modes using --modes: - the default scans (if --modes not specified) are OE_RECIPES,SIG_SCAN,CVE_PATCHES (same as --modes DEFAULT) - see Scan Modes below.
Optionally override Bitbake Environment Values: By default, the utility calls Bitbake to extract environment and layer information, and will refer to the latest Yocto build. You can override values including license.manifest, machine, target, download_dir, package_dir, and image_package_type using command-line parameters.
Generate a Recipe Report: Use the --recipe_report REPFILE parameter to create a report of matched and unmatched recipes in the BOM, required for analysis and debugging.
Cache OE Data: The --oe_data_folder FOLDER parameter allows you to cache downloaded OE data (approx. 300MB) and reuse it in subsequent runs, saving download time. OE data doesn't change frequently.
Identify Patched CVEs: Add the cve_check class to your Bitbake local.conf to identify patched CVEs. Ensure PHASE 7 picks up the cve-check file. Optionally, specify the output CVE check file using --cve_check_file FILE if an alternative location is needed. See CVE Patching for more information.
Fuzzy Match Modified Recipes: For recipes modified from standard OE versions, optionally use --max_oe_version_distance X.X.X (e.g., 0.0.1 to 0.0.10) for fuzzy matching against OE recipes. Be cautious, as this can sometimes disable correct matches. It's recommended to create two projects and compare results with and without this parameter. See OE Difference Calculations for more information.
Process Image Manifest: To include the Linux kernel and other packages specified in the image manifest, consider adding --modes IMAGE_MANIFEST. Optionally, specify the image_license.manifest file path (--image_license_manifest FILEPATH) if the latest build is not desired.
Add Components by CPE: Add --modes CPE_COMPS to create packages not matched by other methods through CPE lookup. Note that not all packages have published CPEs.
Process Kernel Vulnerabilities: To ignore kernel vulnerabilities not within compiled kernel sources, add --modes KERNEL_VULNS (assumes IMAGE_MANIFEST mode).
Add Custom Components for Unmatched recipes - USE WITH CAUTION Where OSS recipes are not matched by any other method, and for custom or commercial recipes, you can consider adding --modes CUSTOM_COMPS to create Custom Components for unmatched recipes and add them to the BD project as placeholders. However, note that Custom Components do not have any vulnerabilities mapped, will use the license specified in the license.manifest and cannot be easily deleted once added without deleting the project itself. Also note that all future scans for the same recipe will map to the created Custom Components even when KB updates add new recipes (unless the associated PURL is deleted under 'Management-->Unmatched Components').

Scan Modes

The new --modes option is a comma-delimited list of modes (no spaces), used to control the multiple types of scan supported in the script. It can be used to replace existing scan control parameters and simplify the command line.

Explanation of modes:

DEFAULT - Includes OE_RECIPES,SIG_SCAN,CVE_PATCHES
OE_RECIPES - Map components by OE recipe lookup (set by DEFAULT)
IMAGE_MANIFEST - Process image manifest in addition to standard manifest (usually to add linux kernel)
SIG_SCAN - Scan unmatched recipes/packages using Signature scan (set in DEFAULT mode)
SIG_SCAN_ALL - Scan all recipes/packages using Signature scan
CPE_COMPS - Add unmatched recipes as packages by CPE lookup (where CPEs available)
CUSTOM_COMPS - Create Custom Components for unmatched recipes (note Custom Components are only placeholders for SBOM export - no vulnerability or other data is provided)
CVE_PATCHES - Process locally patched CVEs from cve_check class (set in DEFAULT mode) - See CVE Patching
KERNEL_VULNS - Process kernel modules and mark vulns as unaffected where associated modules do not exist in the kernel
ALL - Includes OE_RECIPES,IMAGE_MANIFEST,SIG_SCAN,CVE_PATCHES,CPE_COMPS,CUSTOM_COMPS,KERNEL_VULNS (but not SIG_SCAN_ALL)

Notes:

DEFAULT is assumed if --modes not specified.
To add scan modes to the default set use, for example --modes DEFAULT,IMAGE_MANIFEST,KERNEL_VULNS.
To add SIG_SCAN_ALL mode (scanning all packages) to ALL scan mode use --modes ALL,SIG_SCAN_ALL.

Mapping of existing scan control parameters to scan modes:

--process_image_manifest = mode IMAGE_MANIFEST
--scan_all_packages = mode SIG_SCAN_ALL
--add_comps_by_cpe = mode CPE_COMPS
--process_kernel_vulns = mode KERNEL_VULNS
--sbom_create_custom_components = mode CUSTOM_COMPS

Specifying legacy parameters in addition to --modes will override scan modes defined in the modes list (including modes in DEFAULT).

Other Scan Options

Improve Recipe Release Identification: Optionally run bitbake -g to create a task-depends.dot file which is specified in --task_depends_dot_file FILE along with -l license.manifest. If -l license.manifest is not also specified, all development dependencies will be processed (not just those within the license manifest).
Signature scan ALL packages (as opposed to only unmatched packages): Add --modes SIG_SCAN_ALL to scan all packages.
Turn on Snippet scanning within Signature scanned packages: Requires --modes SIG_SCAN or --modes SIG_SCAN_ALL - add the parameter --detect_opts detect.blackduck.signature.scanner.snippet.matching=SNIPPET_MATCHING to enable snippet scanning of sources within scanned packages.

Command Line Parameters

usage: bd-scan-yocto-via-sbom [-h] [--blackduck_url BLACKDUCK_URL] [--blackduck_api_token BLACKDUCK_API_TOKEN] [--blackduck_trust_cert] [-p PROJECT] [-v VERSION] <OTHER PARAMETERS>

Create BD-SCA project version from Yocto project

-h, --help: Show help message and exit.

Black Duck Project Parameters - REQUIRED:

--blackduck_url BLACKDUCK_URL: Black Duck server URL (also uses BLACKDUCK_URL environment variable).
--blackduck_api_token BLACKDUCK_API_TOKEN: Black Duck API token (also uses BLACKDUCK_API_TOKEN environment variable).
-p PROJECT, --project PROJECT: Black Duck project to create or update.
-v VERSION, --version VERSION: Black Duck project version to create or update.
-t TARGET, --target TARGET: Yocto target (e.g., 'core-image-sato' - single target configuration only supported).
--blackduck_trust_cert: Trust Black Duck server certificate (also uses BLACKDUCK_TRUST_CERT environment variable) - OPTIONAL.

Scan Mode Configuration Parameter - OPTIONAL:

--modes MODES: A comma-delimited list of scan modes (no spaces) selected from [ALL,DEFAULT,OE_RECIPES,IMAGE_MANIFEST,SIG_SCAN,SIG_SCAN_ALL,CVE_PATCHES,CPE_COMPS,CUSTOM_COMPS,KERNEL_VULNS] - see Scan Modes.

Yocto Project Configuration Parameters - OPTIONAL:

-l LICENSE_MANIFEST, --license_manifest LICENSE_MANIFEST: The most recent Yocto license.manifest file is located by default, usually located in tmp/deploy/licenses/<yocto-image>-<yocto-machine>/license.manifest. Use this option to specify a different build or if the file is in a non-standard location (or if --skip_bitbake is used).
-i IMAGE_LICENSE_MANIFEST, --image_license_manifest IMAGE_LICENSE_MANIFEST: If --modes IMAGE_MANIFEST specified, the latest image manifest will be identified by default from the environment. Specify the path to an alternate image_license.manifest file if the latest build is not desired or the file exists in another location.
--task_depends_dot_file FILE: Specify the path to the task-depends.dot file created by bitbake -g for processing. If -l license.manifest is not also specified, all recipes, including dev dependencies, will be processed (--target is also required).
-b BITBAKE_LAYERS, --bitbake_layers_file BITBAKE_LAYERS: Optionally specify the output of the command bitbake-layers show-recipes stored in a file. Should be used with --skip_bitbake (although it will prevent many other script modes from operating, so it's not recommended).
-c CVE_CHECK_FILE, --cve_check_file CVE_CHECK_FILE: CVE check output file (in .cve or .json format) to mark locally patched CVEs. The most recent file will be located by default; use this parameter to specify an alternate file. Usually located in build/tmp/deploy/images/XXX. Should be determined from the Bitbake environment by default (unless --skip_bitbake is used).
--build_dir BUILD_DIR: Alternate Yocto build folder (defaults to poky/build).
--download_dir DOWNLOAD_DIR: Alternate directory where original OSS source is downloaded (defaults to poky/build/downloads).
--package_dir PACKAGE_DIR: Alternate directory where package files are downloaded (e.g., poky/build/tmp/deploy/rpm/<ARCH>).
--image_package_type IMAGE_PACKAGE_TYPE: Package type used for installing packages (specify one of rpm, deb, or ipx - default rpm).
--kernel_recipe RECIPE_NAME: Define a non-standard kernel recipe name (defaults to 'linux-yocto').

Script Behavior Parameters - OPTIONAL:

--skip_oe_data: Do not download layers/recipes/layers from layers.openembedded.org APIs. These are used to review origin layers and revisions within recipes to ensure more components are matched against the Black Duck KnowledgeBase (KB). Useful where recipes have been moved to new layers against the template recipes provided by OE - (equivalent to removing OE_RECIPES from --modes).
--oe_data_folder OE_DATA_FOLDER: Folder to contain OE data files. If files don't exist, they will be downloaded; if they exist, they will be used without re-downloading. Creates OE data files in the specified folder if they don't already exist. If files exist, they are used without re-downloading. This allows offline usage of OE data or reduces large data transfers if the script is run frequently. RECOMMENDED.
--max_oe_version_distance MAX_OE_VERSION_DISTANCE: When no exact match, use the closest previous recipe version up to the specified distance against OE data. Setting this value allows close (previous) recipe version matching. The value must be in MAJOR.MINOR.PATCH format (e.g., 0.10.0). CAUTION: Setting this value too high may cause components to be matched against older recipes in the OE data, potentially leading to different vulnerability reports. It's generally better to maintain a close relationship between matched versions and project versions. Consider values in the range 0.0.1 to 0.0.10. See OE Difference Calculations.
--skip_sig_scan: Do not signature scan downloads and packages. By default, only recipes not matched from OE data are scanned (equivalent to removing SIG_SCAN from --modes)

Connection & Detect Configuration Parameters - OPTIONAL:

--detect_jar_path DETECT_JAR_PATH: Path to the Detect JAR file.
--detect_opts DETECT_OPTS: Additional Detect options, comma separated list (remove leading -- from Detect options).
--api_timeout: Specify API timeout in seconds (default 60). Used in Detect as --detect.timeout.
--no_unmap: Do not unmap previous code locations (scans) when running the initial scan (default is to unmap).

Legacy Scan Mode Parameters (replaced by --modes):

--process_image_manifest: Process the image_license.manifest file (default location) to include recipes from the core image - equivalent to --modes IMAGE_MANIFEST.
--add_comps_by_cpe: Look up CPEs for packages not mapped in the BOM or discovered by signature scan and add missing packages. Note: not all packages have published CPEs - equivalent to --modes CPE_COMPS.
--process_kernel_vulns: Process compiled kernel sources to ignore vulnerabilities affecting modules not compiled into the custom kernel. Requires --process_image_manifest - equivalent to --modes KERNEL_VULNS.
--scan_all_packages: Signature scan all packages (default: only recipes not matched from OE data are scanned) - equivalent to --modes SIG_SCAN_ALL.
--sbom_create_custom_components: Create Custom Components when all other scan techniques do not match recipes. The license reported in license.manifest will be used. Custom Components are designed for unknown components but lack associated vulnerabilities, deep license data, or copyrights. Caution: This parameter creates new, permanent custom components mapped to PURLs. These PURLs will be matched in all future scans, even if new data is added to the KB for the recipe. Use the Manage --> Unmatched Origins view to disassociate PURLs from custom components if you don't want them matched in future scans. Deleting custom components once created and referenced in projects is not possible unless the projects themselves are removed. Do not use this parameter until you fully understand its implications - equivalent to --modes CUSTOM_COMPS.

General Parameters:

--recipe_report REPFILE: Create a report file with a list of recipes, including those not matched in the BOM.
-o OUTPUT, --output OUTPUT: Specify output SPDX SBOM file. If specified, only the initial SBOM will be created and all other script features will be skipped (use for debug purposes only).
--debug: Enable debug logging mode.
--logfile LOGFILE: Output logging messages to a specified file.

Minimum Required Parameters

To run the script, you must provide:

--blackduck_url BD_URL --blackduck_api_token BD_API -p BD_PROJECT -v BD_VERSION -t YOCTO_TARGET

Optionally, add --blackduck_trust_cert to trust the Black Duck server certificate.

Consider using --oe_data_folder FOLDER with an existing folder to cache OE recipe data on subsequent runs, saving large downloads.

Server credentials can also be specified using the standard environment variables: BLACKDUCK_URL, BLACKDUCK_API_TOKEN, and BLACKDUCK_TRUST_CERT.

Example Distance Calculations for `--max_oe_version_difference`

Recipe version is 3.2.4 - closest previous OE recipe version is 3.2.1: Distance value needed: minimum 0.0.3
Recipe version is 3.2.4 - closest previous OE recipe version is 3.0.1: Distance value needed: minimum 0.2.0
Recipe version is 3.2.4 - closest previous OE recipe version is 2.0.1: Distance value needed: minimum 1.0.0

Note that lower-order values are overridden by higher-order values (e.g., a distance of 1.0.0 is equivalent to 1.999.999).

CVE Patching

The Yocto cve_check class processes Bitbake dependencies within the development environment, generating a list of CVEs identified from the NVD for all packages in that environment.

For patched CVE remediation in your Black Duck project, you need to add the cve_check bbclass to your Yocto build configuration to generate the CVE check log output. Add the following line to your build/conf/local.conf (or similar) file:

INHERIT += "cve-check"

Then, rebuild your project (e.g., using bitbake core-image-sato) to run the CVE check action and generate the required CVE log files without a full rebuild.

The script should automatically locate the cve-check output file (usually under poky/tmp/deploy/images/licenses), but you can also specify the file using the --cve_check_file CVE_CHECK_FILE parameter.

The CVE_PATCHES mode enables this feature which is included in the DEFAULT mode. Remove CVE_PATCHES from the list of modes to disable.

Troubleshooting

Ensure required parameters are specified: Double-check that you've provided all minimum required parameters, including the Yocto target (-t).
Verify prerequisites and build status: Confirm all prerequisites are met and that you have a built Yocto project with a generated license.manifest file - see Prerequisites
Enable debug logging: Use the --debug parameter to turn on detailed logging.
Check code locations in Black Duck: After the scan, verify that your Black Duck project's Source tab shows 2 or 3 separate code locations: one for the initial SBOM import, one for Signature Scan, and optionally one for the second SBOM used to create CPE matching/custom components.
Review Recipe matching information: Specify --recipe_report FILE to create a full list of recipes and examine missing recipes; consider using --max_oe_version_difference X.X.X to enable fuzzy matching of OE recipes, or adding additional scan modes to add missing recipes.
Missing packages: If a specific package appears missing from the project, confirm you are looking for the recipe name, not the package name (See FAQs for an explanation of Yocto recipes vs. packages). The scan modes SIG_SCAN, CPE_COMPS and CUSTOM_COMPS can be specified in --modes to process missing recipes using various techniques (alternatively use --modes ALL).
Linux kernel missing: Consider adding the --modes IMAGE_MANIFEST mode to include packages in the image manifest, which usually includes the Linux kernel. Use --image_license_manifest PATH to specify a different path to the image_license.manifest file if in a non-standard location or the latest build is not desired. Check the Linux Kernel recipe name ('linux-yocto' is expected - use parameter --kernel_recipe mylinux to specify a non-standard recipe name for the kernel). If the kernel still isn't identified due to a custom name format, consider adding the required kernel version manually to the project.

How to Report Issues with This Script

If you encounter issues, please create an issue in GitHub and include the following information:

Your Organization name (to validate your Black Duck license).
A clear description of the issue.
Yocto version in use (obtain with bitbake -e | grep DISTRO_CODENAME).
The license.manifest file for your build (and optionally image_license.manifest is mode IMAGE_MANIFEST is used.
The output of the command bitbake-layers show-recipes in a file.
The log output from this script with --debug enabled (use --logfile LOGFILE to save to a file).
The recipe report file (use --recipe_report REPFILE).

Additional Scan Possibilities (Outside This Script)

For custom C/C++ recipes or recipes built with other languages and package managers, you can combine this script with other scan types:

C/C++ Recipes: The advanced blackduck_c_cpp utility can be used during the build process to identify compiled sources, system includes, and operating system dependencies. You would need to modify the recipe's build command to call blackduck-c-cpp as part of a scanning cycle after configuring it to connect to your Black Duck server.
Package Manager-Based Recipes: For recipes utilizing a package manager, a standard Detect scan in DETECTOR mode can be used to analyze project dependencies separately.

Combining Multiple Scans: Multiple scans can be combined into the same Black Duck project. Ensure you use the Detect parameter --detect.project.codelocation.unmap=false to prevent previous scans from being unmapped.

Release Notes

v1.2.1
- Minor fix for CPE lookup version strings
v1.2.0
- Addition of --modes option to simplify multiple scan mode options:
- Modes string can contain comma separated list (no spaces) from [ALL, OE_RECIPES*, IMAGE_MANIFEST, SIG_SCAN*, SIG_SCAN_ALL, CVE_PATCHES*, CPE_COMPS, CUSTOM_COMPS, KERNEL_VULNS] (modes marked with * run by default)
- Note that legacy options to control scan modes are still supported
v1.1.4
- Minor bug fix for sbom_create_custom_components and more
v1.1.3
- Minor fix to enforce --process_image_manifest if --process_kernel_vulns specified (as image manifest required to locate kernel image package and extract kernel modules).
v1.1.2
- Improved custom component creation by SBOM import: added to the second SBOM import (which runs when --add_comps_by_cpe is specified).
- Moved SBOM custom component creation step to after Signature scan alongside optional addition of packages using CPE - to ensure custom packages only added if truly unmatched.
- Removed date/time from SBOM document name and signature scan temporary folder names, allowing rescanning without unmapping (required for Detect 11). Default unmapping will be removed when Detect 11 is implemented.
- Removed asyncio as dependency (bundled in Python 3.4+)
v1.1.1
- Minor fix for cve_check output file identification.
v1.1.0
- Added --add_comps_by_cpe parameter to look for unmatched packages via CPE.
- Changed vulnerability patching to use asynchronous updates for improved speed.
- Added custom kernel source scanning to ignore vulnerabilities affecting uncompiled kernel modules (requires --process_kernel_vulns and process_image_manifest).

FAQs

Why can't I rely on the license data provided by Yocto in the license.manifest file? The licenses reported by Bitbake come directly from the recipe files. However, the true applicable license for each package is the one declared in its origin repository, which may differ. Furthermore, most open-source licenses require the full license text and copyrights in the distribution. Many OSS packages also embed other OSS with different licenses, sometimes with re-licensing restrictions. Black Duck uses licenses from the origin packages, supports full license text and copyrights, and offers optional deep license analysis to identify embedded licenses within packages. Licenses from recipes are used when Custom Components are created using --modes CUSTOM_COMPS.
Can this utility be used on a Yocto image without access to the build environment? Yes, potentially. Use the parameters --skip_bitbake -l LIC_MANIFEST_FILE --bitbake_layers_file LAYERS_FILE, where LIC_MANIFEST_FILE is the path to your license.manifest file and LAYERS_FILE contains the output of bitbake-layers show-recipes. However, several scan modes will be disabled due to the lack of build artefacts (including Signature scanning of packages --modes SIG_SCAN, CVE patching --modes CVE_PATCHES and kernel vulnerability applicability --modes KERNEL_VULNS).
Why can't I simply use the cve-check class provided by Yocto to determine unpatched vulnerabilities? The cve-check class processes all recipes in the build and attempts to associate CVEs from the NVD using CPEs. This often leads to a large number of false positive CVEs, as it reports all packages (including build dependencies) rather than just those in the distributed image and the CPE is a wildcard often associating many vulnerabilities which are false positive. Furthermore, the CPE association data from the NVD is frequently inaccurate with no earliest affected version meaning that newer vulnerabilities are shown for all previous verisons of packages. Black Duck Security Advistories are expert-curated to reduce false positives including marking CVEs as ignored where they should not apply.
Why couldn't I just use the create-spdx class provided by Yocto to export a full SBOM? The Yocto create-spdx class generates SPDX JSON files for packages, including data like file lists and hashes. However, many SPDX fields (e.g., license text, copyrights) are often blank (NO-ASSERTION), and packages are not identified by PURL, stopping import into other tools (including Black Duck).
I cannot see a specific package in the Black Duck project. Black Duck reports recipes in the Yocto project, not individual packages. Multiple packages can be combined into a single recipe and are generally considered part of that recipe's main component. Use the --recipe_report FILE parameter to list matched and missing recipes, and conider using other scan modes (--modes ALL or --modes CPE_COMPS,CUSTOM_COMPS) to add missing recipes. Note also that Signature scan (--modes SIG_SCAN or --modes SIG_SCAN_ALL) can identify missing recipes as well as embedded OSS within recipes.
I cannot see the Linux kernel in the Black Duck project. Consider using the --modes IMAGE_MANIFEST (optionally with --image_license_manifest PATH) to process packages in the image manifest, which usually includes the Linux kernel. If the kernel still cannot be identified due to a custom name format, consider adding the required kernel version manually to the project.
I am using another Yocto wrapper like KAS (https://kas.readthedocs.io/) and cannot run Bitbake, or the script fails for some other reason. Use the parameters --skip_bitbake -l LIC_MANIFEST_FILE --bitbake_layers_file LAYERS_FILE, where LIC_MANIFEST_FILE is the path to your license.manifest file and LAYERS_FILE contains the output of bitbake-layers show-recipes. Note that this will disable several features, including CVE patching, kernel vulnerability identification, and package scanning.
I want to scan all recipes, including development dependencies, as opposed to only those in the delivered image. Run the command bitbake -g to create a task-depends.dot file, then use the parameter --task_depends_dot_file FILE, where FILE is the path to the generated file.

Name		Name	Last commit message	Last commit date
Latest commit History 160 Commits
.github/workflows		.github/workflows
bd_scan_yocto		bd_scan_yocto
LICENSE		LICENSE
README.md		README.md
pyproject.toml		pyproject.toml
run.py		run.py

License

blackducksoftware/bd_scan_yocto_via_sbom

Folders and files

Latest commit

History

Repository files navigation

Black Duck SCA Scan Yocto Script - bd_scan_yocto_via_sbom.py v1.2.1