Merge remote-tracking branch 'origin/1682-left-pane-file-tree' into 1682-left-pane-file-tree

Author: aayushkdev

CHANGELOG.rst

@@ -1,6 +1,17 @@
 Changelog
 =========
 
+v35.3.0 (unreleased)
+--------------------
+
+- Enhanced scorecard compliance support with:
+  * New ``scorecard_compliance_alert`` in project ``extra_data``.
+  * ``/api/projects/{id}/scorecard_compliance/`` API endpoint.
+  * Scorecard compliance integration in ``check-compliance`` management command.
+  * UI template support for scorecard compliance alert.
+  * ``evaluate_scorecard_compliance()`` pipe function for compliance evaluation.
+  https://github.com/aboutcode-org/scancode.io/pull/1800
+
 v35.2.0 (2025-08-01)
 --------------------
 

docs/installation.rst

@@ -55,6 +55,40 @@ create an **environment file**, and **build the Docker image**::
     As the ``docker-compose`` v1 command is officially deprecated by Docker, you will
     only find references to the ``docker compose`` v2 command in this documentation.
 
+.. note::
+    If you intend to run an Android deploy to develop project, ``Java``, ``jadx
+    v1.5.0`` and ``android-inspector`` must be installed in the Docker image by
+    adding the following lines to the ``Dockerfile`` and rebuilding the Docker
+    image:
+
+    Add at line 65 after `apt-get` command::
+
+        # Install Java and utilities to install jadx
+        RUN apt-get update \
+        && apt-get install -y --no-install-recommends \
+            openjdk-17-jre-headless \
+            unzip \
+            wget
+
+        # Download and extract jadx
+        RUN wget https://github.com/skylot/jadx/releases/download/v1.5.0/jadx-1.5.0.zip \
+        && unzip -d /usr jadx-1.5.0.zip
+
+        # Remove jadx archive and installed utilities
+        RUN apt-get remove -y unzip wget \
+        && apt-get clean \
+        && rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/* \
+        && rm jadx-1.5.0.zip
+
+    Add at end of file::
+
+        # Install android-inspector
+        RUN pip install --no-cache-dir android-inspector
+
+    Rebuild the image::
+
+        docker compose build
+
 Run the App
 ^^^^^^^^^^^
 
@@ -293,6 +327,20 @@ For the :ref:`pipeline_collect_strings_gettext` pipeline, `gettext <https://www.
 
         brew install gettext
 
+For the Android deploy to develop pipeline, `jadx <https://github.com/skylot/jadx>` and `Java <https://openjdk.org/index.html>`_ are needed.
+
+    * On **Linux** install it using::
+
+        # Ensure that you are in the scancode.io directory
+        sudo apt-get install openjdk-21-jre # Install Java 21
+        wget https://github.com/skylot/jadx/releases/download/v1.5.0/jadx-1.5.0.zip # Download jadx v1.5.0
+        unzip -qd jadx-1.5.0 jadx-1.5.0.zip # Extract jadx-1.5.0.zip
+        export PATH=$PATH:`pwd`/jadx-1.5.0/bin/jadx:`pwd`/jadx-1.5.0/lib # add jadx-1.5.0 binary and libraries to your path
+
+    * On **MacOS** install it using Homebrew::
+
+        brew install jadx
+
 Clone and Configure
 ^^^^^^^^^^^^^^^^^^^
 
@@ -326,6 +374,11 @@ Clone and Configure
 
     make envfile
 
+ * If you intend to run an Android deploy to develop project, install the pipeline::
+
+    source .venv/bin/activate
+    pip install android-inspector
+
 Database
 ^^^^^^^^
 

docs/policies.rst

@@ -91,6 +91,43 @@ Accepted values for the alert level:
 - ``warning``
 - ``error``
 
+Creating Scorecard Thresholds Files
+-----------------------------------
+
+A valid scorecard thresholds file is required to **enable OpenSSF Scorecard compliance features**.
+
+The scorecard thresholds file, by default named ``policies.yml``, is a **YAML file** with a
+structure similar to the following:
+
+.. code-block:: yaml
+
+    scorecard_score_thresholds:
+      9.0: ok
+      7.0: warning
+      0: error
+
+- In the example above, the keys ``9.0``, ``7.0``, and ``0`` are numeric threshold values
+  representing **minimum scorecard scores**.
+- The values ``error``, ``warning``, and ``ok`` are the **compliance alert levels** that
+  will be triggered if the project's scorecard score meets or exceeds the
+  corresponding threshold.
+- The thresholds must be listed in **strictly descending order**.
+
+How it works:
+
+- If the scorecard score is **9.0 or above**, the alert is **``ok``**.
+- If the scorecard score is **7.0 to 8.9**, the alert is **``warning``**.
+- If the scorecard score is **below 7.0**, the alert is **``error``**.
+
+You can adjust the threshold values and alert levels to match your organization's
+security compliance requirements.
+
+Accepted values for the alert level:
+
+- ``ok``
+- ``warning``
+- ``error``
+
 App Policies
 ------------
 

docs/rest-api.rst

@@ -518,6 +518,31 @@ Data:
         "license_clarity_compliance_alert": "warning"
     }
 
+.. _rest_api_scorecard_compliance:
+
+Scorecard Compliance
+^^^^^^^^^^^^^^^^^^^^
+
+This action returns the **scorecard compliance alert** for a project.
+
+The scorecard compliance alert is a single value (``ok``, ``warning``, or ``error``)
+that summarizes the project's **OpenSSF Scorecard security compliance status**,
+based on the thresholds defined in the ``policies.yml`` file.
+
+``GET /api/projects/6461408c-726c-4b70-aa7a-c9cc9d1c9685/scorecard_compliance/``
+
+Data:
+    - ``scorecard_compliance_alert``: The overall scorecard compliance alert
+      for the project.
+
+      Possible values: ``ok``, ``warning``, ``error``.
+
+.. code-block:: json
+
+    {
+        "scorecard_compliance_alert": "warning"
+    }
+
 Reset
 ^^^^^
 

docs/tutorial_license_policies.rst

@@ -128,6 +128,51 @@ The ``license_clarity_compliance_alert`` value (e.g., ``"error"``, ``"warning"``
 is computed automatically based on the thresholds you configured and reflects the
 overall license clarity status of the scanned codebase.
 
+Scorecard Compliance Thresholds and Alerts
+------------------------------------------
+
+ScanCode.io also supports **OpenSSF Scorecard compliance thresholds**, allowing you to enforce
+minimum security standards for open source packages in your codebase. This is managed
+through the ``scorecard_score_thresholds`` section in your ``policies.yml`` file.
+
+Defining Scorecard Thresholds
+-----------------------------
+
+Add a ``scorecard_score_thresholds`` section to your ``policies.yml`` file, for example:
+
+.. code-block:: yaml
+
+    scorecard_score_thresholds:
+      9.0: ok
+      7.0: warning
+      0: error
+
+Scorecard Compliance in Results
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+When you run a the addon pipeline fetch_scores with scorecard thresholds defined in your
+``policies.yml``, the computed scorecard compliance alert is included in the project's
+``extra_data`` field.
+
+For example:
+
+.. code-block:: json
+
+    "extra_data": {
+      "md5": "d23df4a4",
+      "sha1": "3e9b61cc98c",
+      "size": 3095,
+      "sha256": "abacfc8bcee59067",
+      "sha512": "208f6a83c83a4c770b3c0",
+      "filename": "cuckoo_filter-1.0.6.tar.gz",
+      "sha1_git": "3fdb0f82ad59",
+      "scorecard_compliance_alert": "warning"
+    }
+
+The ``scorecard_compliance_alert`` value (e.g., ``"error"``, ``"warning"``, or ``"ok"``)
+is computed automatically based on the thresholds you configured and reflects the
+overall security compliance status of the OpenSSF Scorecard scores for packages in the scanned codebase.
+
 Run the ``check-compliance`` command
 ------------------------------------
 

pyproject.toml

@@ -60,7 +60,7 @@ dependencies = [
   "extractcode[full]==31.0.0",
   "commoncode==32.3.0",
   "Beautifulsoup4[chardet]==4.13.4",
-  "packageurl-python==0.17.2",
+  "packageurl-python==0.17.3",
   # Workaround issue https://github.com/aboutcode-org/scancode.io/issues/1795
   "fingerprints==1.2.3",
   "normality==2.6.1",

scanpipe/api/views.py

@@ -497,6 +497,22 @@ def license_clarity_compliance(self, request, *args, **kwargs):
         clarity_alert = project.get_license_clarity_compliance_alert()
         return Response({"license_clarity_compliance_alert": clarity_alert})
 
+    @action(detail=True, methods=["get"])
+    def scorecard_compliance(self, request, *args, **kwargs):
+        """
+        Retrieve the scorecard compliance alert for a project.
+
+        This endpoint returns the scorecard compliance alert stored in the
+        project's extra_data.
+
+        Example:
+        GET /api/projects/{project_id}/scorecard_compliance/
+
+        """
+        project = self.get_object()
+        scorecard_alert = project.get_scorecard_compliance_alert()
+        return Response({"scorecard_compliance_alert": scorecard_alert})
+
 
 class RunViewSet(mixins.RetrieveModelMixin, viewsets.GenericViewSet):
     """Add actions to the Run viewset."""

scanpipe/management/commands/check-compliance.py

@@ -77,7 +77,12 @@ def check_compliance(self, fail_level):
         clarity_alert = self.project.get_license_clarity_compliance_alert()
         has_clarity_issue = clarity_alert not in (None, "ok")
 
-        total_issues = count + (1 if has_clarity_issue else 0)
+        scorecard_alert = self.project.get_scorecard_compliance_alert()
+        has_scorecard_issue = scorecard_alert not in (None, "ok")
+
+        total_issues = (
+            count + (1 if has_clarity_issue else 0) + (1 if has_scorecard_issue else 0)
+        )
 
         if total_issues and self.verbosity > 0:
             self.stderr.write(f"{total_issues} compliance issues detected.")
@@ -92,6 +97,10 @@ def check_compliance(self, fail_level):
                 self.stderr.write("[license clarity]")
                 self.stderr.write(f" > {clarity_alert.upper()}")
 
+            if has_scorecard_issue:
+                self.stderr.write("[scorecard compliance]")
+                self.stderr.write(f" > {scorecard_alert.upper()}")
+
         return total_issues > 0
 
     def check_vulnerabilities(self):

scanpipe/models.py

@@ -1554,6 +1554,13 @@ def get_license_clarity_compliance_alert(self):
         """
         return self.extra_data.get("license_clarity_compliance_alert")
 
+    def get_scorecard_compliance_alert(self):
+        """
+        Return the scorecard compliance alert value for the project,
+        or None if not set.
+        """
+        return self.extra_data.get("scorecard_compliance_alert")
+
     def get_license_policy_index(self):
         """Return the policy license index for this project instance."""
         if policies_dict := self.get_policies_dict():

scanpipe/pipelines/deploy_to_develop.py

@@ -76,6 +76,7 @@ def steps(cls):
             cls.map_javascript,
             cls.map_javascript_symbols,
             cls.map_javascript_strings,
+            cls.get_symbols_from_binaries,
             cls.map_elf,
             cls.map_macho,
             cls.map_winpe,
@@ -197,6 +198,14 @@ def map_javascript_strings(self):
         """Map deployed JavaScript, TypeScript to its sources using string literals."""
         d2d.map_javascript_strings(project=self.project, logger=self.log)
 
+    def get_symbols_from_binaries(self):
+        """Extract symbols from Elf, Mach0 and windows binaries for mapping."""
+        d2d.extract_binary_symbols(
+            project=self.project,
+            options=self.selected_groups,
+            logger=self.log,
+        )
+
     @optional_step("Elf")
     def map_elf(self):
         """Map ELF binaries to their sources using dwarf paths and symbols."""
@@ -215,8 +224,9 @@ def map_winpe(self):
 
     @optional_step("Go")
     def map_go(self):
-        """Map Go binaries to their sources using paths."""
+        """Map Go binaries to their sources using paths and symbols."""
         d2d.map_go_paths(project=self.project, logger=self.log)
+        d2d.map_go_binaries_with_symbols(project=self.project, logger=self.log)
 
     @optional_step("Rust")
     def map_rust(self):