Merge branch 'develop' into rename_match_spdx_expression_3838

Signed-off-by: Ayan Sinha Mahapatra <[email protected]>

Author: AyanSinhaMahapatra

CHANGELOG.rst

@@ -9,10 +9,13 @@ v33.0.0 (next next, roadmap)
   - OpenWRT packages.
   - Yocto/BitBake .bb recipes.
 
+
 - Fallback packages for non-native dependencies of SCTK.
 - Dependencies for
 - Support for copyright detection objects.
 
+- We can now collect packages from a Go binary using go-inspector (Linux-only)
+
 - A new field in packages with the license category for the
   detected license expression and also an API function to
   compute license categories from license expressions.
@@ -34,6 +37,22 @@ v33.0.0 (next next, roadmap)
   of these in other summary plugins.
   See https://github.com/nexB/scancode-toolkit/issues/1745
 
+- Update link references of ownership from nexB to aboutcode-org
+  See https://github.com/aboutcode-org/scancode-toolkit/issues/3885
+
+- New and updated licenses, including support for newly released
+  SPDX license list versions:
+  - SPDX License List 3.25.0:
+    This release of the SPDX license list had 9 new licenses
+    and exceptions, and out of them 5 were present as licenses
+    and 2 were present as rules already. There were 2 new
+    license/exception texts added, and also 1 license was deprecated.
+    For more details see https://github.com/aboutcode-org/scancode-toolkit/pull/3897
+
+- New and improved copyright detection with many false positive removed
+  and refined detection added.
+
+- Fix Python ``SyntaxWarning`` in textcode module.
 
 v32.2.1 - 2024-07-02
 ---------------------
@@ -155,7 +174,7 @@ Changes in Output Data Structure:
       file-level ``package_data``
     * ``license_detections`` and  ``other_license_detections`` in
       codebase level ``packages``
-  
+
   - On using the CLI option ``--license-text-diagnostics`` there is
     now a new license match attribute ``matched_text_diagnostics``
     with the matched text and highlighted diagnostics, instead of
@@ -164,7 +183,7 @@ Changes in Output Data Structure:
   - A new ``reference_matches`` attribute is added to codebase-level
     ``license_detections`` which is same as the ``matches`` attribute
     in other license detections.
-  
+
   - We now have SPDX license expressions everywhere we have
     ScanCode license expressions for ease of use and adopting
     SPDX everywhere. A new attribute ``license_expression_spdx``
@@ -212,7 +231,7 @@ Changes in Output Data Structure:
   and https://github.com/nexB/scancode-toolkit/issues/3443
   Also improve debian manifests parsing and purl parsing from
   filenames. Support for https://github.com/nexB/purldb/issues/245
-  Bumps debian-inspector to v31.1.0 
+  Bumps debian-inspector to v31.1.0
 
 - Bump commoncode to v31.0.3
 
@@ -870,6 +889,8 @@ Package detection:
 
 - For Pypi packages, python_requires is treated as a package dependency.
 
+- Update JavaScript package handlers to handle aliases in npm and yarn manifests.
+
 
 License Clarity Scoring Update:
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

README.rst

@@ -89,7 +89,7 @@ Why use ScanCode?
   InstallShield installers, iOS apps, ISO images, Apache IVY, JBoss Sar,
   R CRAN, Apache Maven, Meteor, Mozilla extensions, MSI installers,
   JavaScript npm packages, package-lock.json, yarn.lock, NSIS Installers,
-  NugGet, OPam, Cocoapods, Python PyPI setup.py, setup.cfg, and 
+  NuGet, OPam, Cocoapods, Python PyPI setup.py, setup.cfg, and 
   several related lockfile formats, semi structured README
   files such as README.android, README.chromium, README.facebook, README.google,
   README.thirdparty, RPMs, Shell Archives, Squashfs images, Java WAR, Windows
@@ -246,6 +246,7 @@ See the NOTICE file and the .ABOUT files that document the origin and license of
 the third-party code used in ScanCode for more details.
 
 
+
 .. |azure| image:: https://dev.azure.com/nexB/scancode-toolkit/_apis/build/status/nexB.scancode-toolkit?branchName=develop
     :target: https://dev.azure.com/nexB/scancode-toolkit/_build/latest?definitionId=1&branchName=develop
     :alt: Azure tests status (Linux, macOS, Windows)
@@ -261,3 +262,142 @@ the third-party code used in ScanCode for more details.
 .. |release-github-actions| image:: https://github.com/nexB/scancode-toolkit/actions/workflows/scancode-release.yml/badge.svg?event=push
     :target: https://github.com/nexB/scancode-toolkit/actions/workflows/scancode-release.yml
     :alt: Release tests
+
+
+Acknowledgements, Funding, Support and Sponsoring
+--------------------------------------------------------
+
+This project is funded, supported and sponsored by:
+
+- Generous support and contributions from users like you!
+- the European Commission NGI programme
+- the NLnet Foundation 
+- the Swiss State Secretariat for Education, Research and Innovation (SERI)
+- Google, including the Google Summer of Code and the Google Seasons of Doc programmes
+- Mercedes-Benz Group
+- Microsoft and Microsoft Azure
+- AboutCode ASBL
+- nexB Inc. 
+
+
+
+|europa|   |dgconnect| 
+
+|ngi|   |nlnet|   
+
+|aboutcode|  |nexb|
+
+
+This project was funded through the NGI0 Discovery Fund, a fund established by NLnet with financial
+support from the European Commission's Next Generation Internet programme, under the aegis of DG
+Communications Networks, Content and Technology under grant agreement No 825322.
+
+|ngidiscovery| https://nlnet.nl/project/vulnerabilitydatabase/
+
+
+This project was funded through the NGI0 Entrust Fund, a fund established by NLnet with financial
+support from the European Commission's Next Generation Internet programme, under the aegis of DG
+Communications Networks, Content and Technology under grant agreement No 101069594. 
+
+|ngizeroentrust| https://nlnet.nl/project/Back2source/
+
+
+This project was funded through the NGI0 Core Fund, a fund established by NLnet with financial
+support from the European Commission's Next Generation Internet programme, under the aegis of DG
+Communications Networks, Content and Technology under grant agreement No 101092990.
+
+|ngizerocore| https://nlnet.nl/project/Back2source-next/
+
+
+This project was funded through the NGI0 Core Fund, a fund established by NLnet with financial
+support from the European Commission's Next Generation Internet programme, under the aegis of DG
+Communications Networks, Content and Technology under grant agreement No 101092990. 
+
+|ngizerocore| https://nlnet.nl/project/FastScan/
+
+
+This project was funded through the NGI0 Commons Fund, a fund established by NLnet with financial
+support from the European Commission's Next Generation Internet programme, under the aegis of DG
+Communications Networks, Content and Technology under grant agreement No 101135429. Additional
+funding is made available by the Swiss State Secretariat for Education, Research and Innovation
+(SERI).
+
+|ngizerocommons| |swiss| https://nlnet.nl/project/MassiveFOSSscan/
+
+This project was funded through the NGI0 Entrust Fund, a fund established by NLnet with financial
+support from the European Commission's Next Generation Internet programme, under the aegis of DG
+Communications Networks, Content and Technology under grant agreement No 101069594. 
+
+|ngizeroentrust| https://nlnet.nl/project/purl2sym/
+
+
+.. |nlnet| image:: https://nlnet.nl/logo/banner.png
+    :target: https://nlnet.nl
+    :height: 50
+    :alt: NLnet foundation logo
+
+.. |ngi| image:: https://ngi.eu/wp-content/uploads/thegem-logos/logo_8269bc6efcf731d34b6385775d76511d_1x.png
+    :target: https://ngi.eu35
+    :height: 50
+    :alt: NGI logo
+
+.. |nexb| image:: https://nexb.com/wp-content/uploads/2022/04/nexB.svg
+    :target: https://nexb.com
+    :height: 30
+    :alt: nexB logo
+
+.. |europa| image:: https://ngi.eu/wp-content/uploads/sites/77/2017/10/bandiera_stelle.png
+    :target: http://ec.europa.eu/index_en.htm
+    :height: 40
+    :alt: Europa logo
+
+.. |aboutcode| image:: https://aboutcode.org/wp-content/uploads/2023/10/AboutCode.svg
+    :target: https://aboutcode.org/
+    :height: 30
+    :alt: AboutCode logo
+
+.. |swiss| image:: https://www.sbfi.admin.ch/sbfi/en/_jcr_content/logo/image.imagespooler.png/1493119032540/logo.png
+    :target: https://www.sbfi.admin.ch/sbfi/en/home/seri/seri.html
+    :height: 40
+    :alt: Swiss logo
+
+.. |dgconnect| image:: https://commission.europa.eu/themes/contrib/oe_theme/dist/ec/images/logo/positive/logo-ec--en.svg
+    :target: https://commission.europa.eu/about-european-commission/departments-and-executive-agencies/communications-networks-content-and-technology_en
+    :height: 40
+    :alt: EC DG Connect logo
+
+.. |ngizerocore| image:: https://nlnet.nl/image/logos/NGI0_tag.svg
+    :target: https://nlnet.nl/core
+    :height: 40
+    :alt: NGI Zero Core Logo
+
+.. |ngizerocommons| image:: https://nlnet.nl/image/logos/NGI0_tag.svg
+    :target: https://nlnet.nl/commonsfund/
+    :height: 40
+    :alt: NGI Zero Commons Logo
+
+.. |ngizeropet| image:: https://nlnet.nl/image/logos/NGI0PET_tag.svg
+    :target: https://nlnet.nl/PET
+    :height: 40
+    :alt: NGI Zero PET logo
+
+.. |ngizeroentrust| image:: https://nlnet.nl/image/logos/NGI0Entrust_tag.svg
+    :target: https://nlnet.nl/entrust
+    :height: 38
+    :alt: NGI Zero Entrust logo
+
+.. |ngiassure| image:: https://nlnet.nl/image/logos/NGIAssure_tag.svg
+    :target: https://nlnet.nl/image/logos/NGIAssure_tag.svg
+    :height: 32
+    :alt: NGI Assure logo
+
+.. |ngidiscovery| image:: https://nlnet.nl/image/logos/NGI0Discovery_tag.svg
+    :target: https://nlnet.nl/discovery/
+    :height: 40
+    :alt: NGI Discovery logo
+
+
+
+
+
+

azure-pipelines.yml

@@ -75,6 +75,33 @@ jobs:
                   tests/licensedcode/test_detection_validate.py \
                   -k TestValidateLicenseExtended5
 
+
+              license_validate_ignorables_1: |
+                venv/bin/pytest -n 3 -vvs --test-suite=validate \
+                  tests/licensedcode/test_detection_validate.py \
+                  -k TestValidateLicenseIgnorableClues1
+
+              license_validate_ignorables_2: |
+                venv/bin/pytest -n 3 -vvs --test-suite=validate \
+                  tests/licensedcode/test_detection_validate.py \
+                  -k TestValidateLicenseIgnorableClues2
+
+              license_validate_ignorables_3: |
+                venv/bin/pytest -n 3 -vvs --test-suite=validate \
+                  tests/licensedcode/test_detection_validate.py \
+                  -k TestValidateLicenseIgnorableClues3
+
+              license_validate_ignorables_4: |
+                venv/bin/pytest -n 3 -vvs --test-suite=validate \
+                  tests/licensedcode/test_detection_validate.py \
+                  -k TestValidateLicenseIgnorableClues4
+
+              license_validate_ignorables_5: |
+                venv/bin/pytest -n 3 -vvs --test-suite=validate \
+                  tests/licensedcode/test_detection_validate.py \
+                  -k TestValidateLicenseIgnorableClues5
+
+
               license_cache: |
                 venv/bin/pytest -n 3 -vvs --test-suite=all \
                   tests/licensedcode/test_zzzz_cache.py --reruns 2

docs/source/cli-reference/basic-options.rst

@@ -108,7 +108,7 @@
                             "license_expression": "apache-2.0",
                             "rule_identifier": "apache-2.0_65.RULE",
                             "rule_relevance": 100,
-                            "rule_url": "https://github.com/nexB/scancode-toolkit/tree/develop/src/licensedcode/data/rules/apache-2.0_65.RULE",
+                            "rule_url": "https://github.com/aboutcode-org/scancode-toolkit/tree/develop/src/licensedcode/data/rules/apache-2.0_65.RULE",
                             "matched_text": "License: Apache-2.0"
                         }
                     ],
@@ -587,19 +587,19 @@
 
     A scan example using the ``--license-url-template TEXT`` option ::
 
-        scancode -clpieu --json-pp output.json samples --license-url-template https://github.com/nexB/scancode-toolkit/tree/develop/src/licensedcode/data/licenses/{}.LICENSE
+        scancode -clpieu --json-pp output.json samples --license-url-template https://github.com/aboutcode-org/scancode-toolkit/tree/develop/src/licensedcode/data/licenses/{}.LICENSE
 
     In a normal scan, reference url for "ZLIB License" is as follows::
 
         "reference_url": "https://scancode-licensedb.aboutcode.org/zlib",
 
     After using the option in the following manner::
 
-        ``--license-url-template https://github.com/nexB/scancode-toolkit/tree/develop/src/licensedcode/data/licenses/{}.LICENSE``
+        ``--license-url-template https://github.com/aboutcode-org/scancode-toolkit/tree/develop/src/licensedcode/data/licenses/{}.LICENSE``
 
-    the reference URL changes to this `zlib.LICENSE file <https://github.com/nexB/scancode-toolkit/blob/develop/src/licensedcode/data/licenses/zlib.LICENSE>`_::
+    the reference URL changes to this `zlib.LICENSE file <https://github.com/aboutcode-org/scancode-toolkit/blob/develop/src/licensedcode/data/licenses/zlib.LICENSE>`_::
 
-        "reference_url": "https://github.com/nexB/scancode-toolkit/blob/develop/src/licensedcode/data/licenses/zlib.LICENSE",
+        "reference_url": "https://github.com/aboutcode-org/scancode-toolkit/blob/develop/src/licensedcode/data/licenses/zlib.LICENSE",
 
     The reference URL changes for all detected licenses in the scan, across the scan result file.
 
@@ -691,7 +691,7 @@
                             "license_expression": "unknown-license-reference",
                             "rule_identifier": "lead-in_unknown_30.RULE",
                             "rule_relevance": 100,
-                            "rule_url": "https://github.com/nexB/scancode-toolkit/tree/develop/src/licensedcode/data/rules/lead-in_unknown_30.RULE",
+                            "rule_url": "https://github.com/aboutcode-org/scancode-toolkit/tree/develop/src/licensedcode/data/rules/lead-in_unknown_30.RULE",
                             "matched_text": "dual-licensed under [`
                         },
                         {
@@ -704,7 +704,7 @@
                             "license_expression": "wtfpl-2.0",
                             "rule_identifier": "spdx_license_id_wtfpl_for_wtfpl-2.0.RULE",
                             "rule_relevance": 50,
-                            "rule_url": "https://github.com/nexB/scancode-toolkit/tree/develop/src/licensedcode/data/rules/spdx_license_id_wtfpl_for_wtfpl-2.0.RULE",
+                            "rule_url": "https://github.com/aboutcode-org/scancode-toolkit/tree/develop/src/licensedcode/data/rules/spdx_license_id_wtfpl_for_wtfpl-2.0.RULE",
                             "matched_text": "WTFPL"
                         },
                         {
@@ -717,7 +717,7 @@
                             "license_expression": "wtfpl-2.0",
                             "rule_identifier": "wtfpl-2.0_27.RULE",
                             "rule_relevance": 100,
-                            "rule_url": "https://github.com/nexB/scancode-toolkit/tree/develop/src/licensedcode/data/rules/wtfpl-2.0_27.RULE",
+                            "rule_url": "https://github.com/aboutcode-org/scancode-toolkit/tree/develop/src/licensedcode/data/rules/wtfpl-2.0_27.RULE",
                             "matched_text": "www.wtfpl.net/"
                         },
                         {
@@ -730,7 +730,7 @@
                             "license_expression": "mit",
                             "rule_identifier": "mit_64.RULE",
                             "rule_relevance": 100,
-                            "rule_url": "https://github.com/nexB/scancode-toolkit/tree/develop/src/licensedcode/data/rules/mit_64.RULE",
+                            "rule_url": "https://github.com/aboutcode-org/scancode-toolkit/tree/develop/src/licensedcode/data/rules/mit_64.RULE",
                             "matched_text": "MIT`](https://opensource.org/licenses/MIT)."
                         }
                     ],

docs/source/cli-reference/help-text-options.rst

@@ -79,7 +79,7 @@ The Following Help Text is displayed, i.e. This is the help text for Scancode Ve
       --csv FILE              [DEPRECATED] Write scan output as CSV to FILE. The
                               --csv option is deprecated and will be replaced by new
                               CSV and tabular output formats in the next ScanCode
-                              release. Visit https://github.com/nexB/scancode-
+                              release. Visit https://github.com/aboutcode-org/scancode-
                               toolkit/issues/3043 to provide inputs and feedback.
       --html FILE             Write scan output as HTML to FILE.
       --custom-output FILE    Write scan output to FILE formatted with the custom
@@ -321,7 +321,7 @@ The Following Text is displayed, i.e. This is the available plugins for Scancode
     required_plugins:
     options:
       help_group: output formats, name: csv: --csv
-        help: [DEPRECATED] Write scan output as CSV to FILE. The --csv option is deprecated and will be replaced by new CSV and tabular output formats in the next ScanCode release. Visit https://github.com/nexB/scancode-toolkit/issues/3043 to provide inputs and feedback.
+        help: [DEPRECATED] Write scan output as CSV to FILE. The --csv option is deprecated and will be replaced by new CSV and tabular output formats in the next ScanCode release. Visit https://github.com/aboutcode-org/scancode-toolkit/issues/3043 to provide inputs and feedback.
     doc: None
 
   --------------------------------------------