Merge branch 'main' into minecode-pipeline-fixes

Author: JonoYang

README.rst

@@ -1,33 +1,31 @@
-==========
-The purldb
-==========
+======
+PurlDB
+======
 
-This purldb consists of these main tools:
+PurlDB provides tools to create and update a database of package metadata keyed by PURL (Package URL) and an API for the PURL data. PurlDB is an `AboutCode project <https://aboutcode.org>`_.
 
-- PackageDB that is the reference model (based on ScanCode toolkit) that contains package data with
-  PURL (Package URLs) being a first class citizen.
-- MineCode that contains utilities to mine package repositories
-- MatchCode that contains utilities to index package metadata and resources for matching
-- MatchCode.io that provides package matching functionalities for codebases
-- ClearCode that contains utilities to mine Clearlydefined for package data
-- purldb-toolkit CLI utility and library to use the PurlDB, its API and various related libraries.
 
-These are designed to be used first for reference such that one can query for packages by purl and
-validate purl existence.
+Why Use PurlDB?
+===============
 
-In the future, the collected packages will be used as reference for dependency resolution, as a
-reference knowledge base for all package data, as a reference for vulnerable range resolution and
-more.
+Take advantage of PurlDB to get fast pre-scanned metadata for popular software
+packages, integrated with other AboutCode projects.
 
-Documentation
--------------
+Getting Started
+===============
 
-See https://aboutcode.readthedocs.io/projects/PURLdb/en/latest/ for PurlDB
-documentation.
+Instructions to get you up and running on your local machine are at `Getting Started <https://purldb.readthedocs.io/en/latest/>`_ .
 
+The PurlDB documentation also provides:
+
+- prerequisites for installing the software.
+- identification of the various tools that contribute to the PurlDB project, including `PackageDB <https://github.com/aboutcode-org/purldb/tree/main/packagedb>`_ , `MineCode <https://github.com/aboutcode-org/purldb/tree/main/minecode>`_, `MatchCode <https://github.com/aboutcode-org/purldb/tree/main/matchcode>`_, `purldb-toolkit <https://github.com/aboutcode-org/purldb-toolkit>`_ and others.
+- guidelines for contributing to code development.
+- how to get started with the REST API.
+- guidelines for contributing to code development.
 
 Acknowledgements, Funding, Support and Sponsoring
---------------------------------------------------------
+=================================================
 
 This project is funded, supported and sponsored by:
 

clearcode/cdutils.py

@@ -34,7 +34,7 @@
 from packageurl import PackageURL
 
 """
-ClearlyDefined utlities.
+ClearlyDefined utilities.
 """
 
 TRACE_FETCH = False
@@ -532,7 +532,7 @@ def str2coord(s):
     segments = s.strip(splitter).split(splitter)
     if is_urn or is_url:
         segments = segments[1:]
-    # ignore extra segments for now beyond the 5 fisrt (such as the PR of a curation)
+    # ignore extra segments for now beyond the 5 first (such as the PR of a curation)
     segments = segments[:5]
 
     fields = (

clearcode/store_scans.py

@@ -34,8 +34,8 @@
 
 """
 The input is a bunch of scans from ClearlyDefined and
-the output is a bunch of git repositories with commited and
-pushed scans such that we balance the scans roughly evenly accross
+the output is a bunch of git repositories with committed and
+pushed scans such that we balance the scans roughly evenly across
 different repositories.
 
 The primary reason for multiple repositories is size of a single
@@ -127,7 +127,7 @@ def get_cd_item_by_purl_hash(cd_items):
 def add_scancode_scan(repo, purl, scancode_scan):
     """
     Save and commit scancode scan for purl to git repo.
-    Return true if we commited else false
+    Return true if we committed else false
     """
     purl_data_dir = get_or_create_dir_for_purl(purl=purl, repo=repo)
     scancode_scan_path = purl_data_dir / "scancode-toolkit-scan.json"

clearindex/management/commands/run_clearindex.py

@@ -161,7 +161,7 @@ def handle(self, *args, **options):
 def map_definition(cditem):
     """
     Map a CD definition. Return the Package created from a mapped CD definition
-    or None if a Package could not be created or an Exception has occured.
+    or None if a Package could not be created or an Exception has occurred.
     """
     try:
         with transaction.atomic():
@@ -328,7 +328,7 @@ def str2coord(s):
     segments = s.strip(splitter).split(splitter)
     if is_urn or is_url:
         segments = segments[1:]
-    # ignore extra segments for now beyond the 5 fisrt (such as the PR of a curation)
+    # ignore extra segments for now beyond the 5 first (such as the PR of a curation)
     segments = segments[:5]
 
     fields = (

docker-compose.yml

@@ -19,7 +19,7 @@ services:
     restart: always
 
   traefik:
-    image: "traefik:v2.10"
+    image: "traefik:v3.6.8"
     container_name: "traefik"
     hostname: "traefik"
     ports:

docs/source/_static/theme_overrides.css

@@ -24,3 +24,50 @@ div.rst-content {
         margin-left: 0px;
     }
 }
+
+.row {
+    clear: both;
+}
+
+.column img {
+    border: 1px solid gray;
+}
+
+@media only screen and (min-width: 1000px),
+only screen and (min-width: 500px) and (max-width: 768px) {
+
+    .column {
+        padding-left: 5px;
+        padding-right: 5px;
+        float: left;
+    }
+
+    .column3 {
+        width: calc(33.3% - 10px);
+    }
+
+    .column2 {
+        width: calc(50% - 11px);
+        position: relative;
+    }
+
+    .column2:before {
+        padding-top: 61.8%;
+        content: "";
+        display: block;
+        float: left;
+    }
+
+    .top-left {
+        border-right: 1px solid var(--color-background-border);
+        border-bottom: 1px solid var(--color-background-border);
+    }
+
+    .top-right {
+        border-bottom: 1px solid var(--color-background-border);
+    }
+
+    .bottom-left {
+        border-right: 1px solid var(--color-background-border);
+    }
+}

docs/source/changelog.rst

@@ -1 +1,3 @@
+:orphan:
+
 .. include:: ../../CHANGELOG.rst

docs/source/conf.py

@@ -9,16 +9,16 @@
 # If extensions (or modules to document with autodoc) are in another directory,
 # add these directories to sys.path here. If the directory is relative to the
 # documentation root, use os.path.abspath to make it absolute, like shown here.
-#
-# import os
-# import sys
-# sys.path.insert(0, os.path.abspath('.'))
+import os
+import sys
 
+os.environ.setdefault("DJANGO_SETTINGS_MODULE", "purldb_project.settings")
+sys.path.insert(0, os.path.abspath("../../."))
 
 # -- Project information -----------------------------------------------------
 
 project = "PurlDB"
-copyright = "nexB Inc. and others."
+copyright = "AboutCode, nexB Inc. and others."
 author = "AboutCode.org authors and contributors"
 
 
@@ -31,28 +31,18 @@
     "sphinx.ext.intersphinx",
     "sphinx_reredirects",
     "sphinx_rtd_theme",
-    "sphinx_rtd_dark_mode",
     "sphinx.ext.extlinks",
     "sphinx_copybutton",
     "sphinx.ext.autodoc",
     "sphinx_rtd_dark_mode",  # For the Dark Mode
 ]
 
 
-# Redirects for olds pages
-# See https://documatt.gitlab.io/sphinx-reredirects/usage.html
-redirects = {}
-
 # This points to aboutcode.readthedocs.io
 # In case of "undefined label" ERRORS check docs on intersphinx to troubleshoot
 # Link was created at commit - https://github.com/aboutcode-org/aboutcode/commit/faea9fcf3248f8f198844fe34d43833224ac4a83
 
 intersphinx_mapping = {
-    "aboutcode": ("https://aboutcode.readthedocs.io/en/latest/", None),
-    "scancode-workbench": (
-        "https://scancode-workbench.readthedocs.io/en/develop/",
-        None,
-    ),
     "scancode.io": (
         "https://scancodeio.readthedocs.io/en/latest/",
         None,
@@ -66,14 +56,13 @@
 # List of patterns, relative to source directory, that match files and
 # directories to ignore when looking for source files.
 # This pattern also affects html_static_path and html_extra_path.
-exclude_patterns = []
+exclude_patterns = ["_build", "Thumbs.db", ".DS_Store"]
 
 
 # -- Options for HTML output -------------------------------------------------
 
 # The theme to use for HTML and HTML Help pages.  See the documentation for
 # a list of builtin themes.
-#
 html_theme = "sphinx_rtd_theme"
 
 # Add any paths that contain custom static files (such as style sheets) here,
@@ -85,7 +74,7 @@
 
 html_context = {
     "display_github": True,
-    "github_user": "nexB",
+    "github_user": "aboutcode-org",
     "github_repo": "purldb",
     "github_version": "main",  # branch
     "conf_py_path": "/docs/source/",  # path in the checkout to the docs root

docs/source/contributing.rst

@@ -1,3 +1,5 @@
+:orphan:
+
 .. _contributing:
 
 Contributing to PurlDB
@@ -106,7 +108,7 @@ See also :ref:`testing` for module-specific testing commands.
 
   **You must include a "Signed-off-by" to your commit messages**::
 
-    Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
+    Signed-off-by: Philippe Ombredanne <pombredanne@aboutcode.org>
 
 4. **Your code is now ready to be pushed as a PR**
 
@@ -124,7 +126,7 @@ overlooked. We value any suggestions to improve
 
 .. tip::
     Our documentation is treated like code. Make sure to check our
-    `writing guidelines <https://scancode-toolkit.readthedocs.io/en/stable/contribute/contrib_doc.html>`_
+    `writing guidelines <https://scancode-toolkit.readthedocs.io/en/stable/getting-started/contribute/contributing-docs.html>`_
     to help guide new users.
 
 Other Ways
@@ -140,7 +142,7 @@ questions, and interact with us and other community members on
 Helpful Resources
 -----------------
 
-- Review our `comprehensive guide <https://scancode-toolkit.readthedocs.io/en/stable/contribute/index.html>`_
+- Review our `comprehensive guide <https://scancode-toolkit.readthedocs.io/en/stable/getting-started/contribute/index.html>`_
   for more details on how to add quality contributions to our codebase and documentation
 - Check this free resource on `how to contribute to an open source project on github <https://egghead.io/courses/how-to-contribute-to-an-open-source-project-on-github>`_
 - Follow `this wiki page <https://aboutcode.readthedocs.io/en/latest/contributing/writing_good_commit_messages.html>`_

docs/source/funding.rst

@@ -1,3 +1,5 @@
+:orphan:
+
 Funding
 =======