Update the custom pipeline file (#250)

* Update the custom pipeline file

      * Updating the text content
      * Updating the text content in the scanpipe-concepts file

Signed-off-by: Hanan Younes <[email protected]>

* Add a minor update to the custom pipeline file

      * Fixing a line too long issue

Signed-off-by: Hanan Younes <[email protected]>

* Minor edit to the updated documentation #237

Co-authored-by: Thomas Druez <[email protected]>

Author: hyounes4560

docs/custom-pipelines.rst

@@ -3,29 +3,35 @@
 Custom Pipelines
 ================
 
-- A pipeline is a **Python class** that lives in a Python module as a ``.py`` **file**.
+A Pipeline is a Python script that performs code analysis by executing a
+sequence of steps.
+
+- A pipeline is a **Python class** that lives in a Python module as a ``.py``
+  **file**.
 - A pipeline class **always inherits** from the ``Pipeline`` base class
-  :ref:`pipeline_base_class`, or from another existing pipeline class, such as the
-  :ref:`built_in_pipelines`.
-- It **defines steps** using the ``steps`` classmethod.
+  :ref:`pipeline_base_class`, or from other existing pipeline classes, such as
+  the :ref:`built_in_pipelines`.
+- It **defines steps** - execution order of the steps - using the ``steps``
+  classmethod.
 
 See :ref:`pipelines_concept` for more details.
 
 Pipeline registration
 ---------------------
 
 Built-in pipelines are located in :guilabel:`scanpipe/pipelines/` directory and
-registered during the ScanCode.io installation.
+are registered during the ScanCode.io installation.
 
-Custom pipelines can be added as Python files ``.py`` in the directories defined in
-the :ref:`scancodeio_settings_pipelines_dirs` setting and will be automatically
-registered at runtime.
+Whereas custom pipelines are added as Python files ``.py`` in the directories
+defined in the :ref:`scancodeio_settings_pipelines_dirs` setting. Custom
+pipelines are registered at runtime.
 
 Create a Pipeline
 -----------------
 
-Create a new Python file ``my_pipeline.py`` in the and make sure the directory is
-registered in the :ref:`scancodeio_settings_pipelines_dirs` setting.
+Create a new Python file ``my_pipeline.py``, and make sure to include the full
+path of the new pipeline directory in the :ref:`scancodeio_settings_pipelines_dirs`
+setting.
 
 .. code-block:: python
 
@@ -48,14 +54,15 @@ registered in the :ref:`scancodeio_settings_pipelines_dirs` setting.
 
 
 .. tip::
-    Have a look in the :guilabel:`scanpipe/pipelines/` directory for more pipeline
+    You can view the :guilabel:`scanpipe/pipelines/` directory for more pipeline
     examples.
 
 Modify existing Pipelines
 -------------------------
 
-Any existing pipeline can be reused as a base and customized.
-You may want to override existing steps, add new ones, and remove some.
+Existing pipelines are flexible and can be reused as a base for custom pipelines
+, i.e. be customized. For instance, you can override existing steps, add new
+ones, or remove any of them.
 
 .. code-block:: python
 
@@ -87,14 +94,14 @@ You may want to override existing steps, add new ones, and remove some.
             pass
 
 
-Report step example
--------------------
+Custom Pipeline example
+-----------------------
 
-Example of a custom pipeline based on the built-in :ref:`pipeline_scan_codebase` one
-with an extra reporting step.
+The example below shows a custom pipeline that is based on the built-in
+:ref:`pipeline_scan_codebase` pipeline with an extra reporting step.
 
-Add the following content to a Python file and register its directory in the
-:ref:`scancodeio_settings_pipelines_dirs`.
+Add the following code snippet to a Python file and register the path of
+the file's directory in the :ref:`scancodeio_settings_pipelines_dirs`.
 
 .. code-block:: python
 

docs/scanpipe-command-line.rst

@@ -1,7 +1,7 @@
 .. _scanpipe_command_line:
 
-Management Commands
-===================
+Command Line Interface
+======================
 
 The main entry point is the :guilabel:`scanpipe` command which is available
 directly when you are in the activated virtualenv or at this path:

docs/scanpipe-concepts.rst

@@ -8,12 +8,13 @@ Project
 
 A **project** encapsulates the analysis of software code:
 
-- it has a **workspace** which is a directory that contains the software code files under
-  analysis
-- it is related to one or more **code analysis pipelines** scripts to automate its analysis
-- it tracks ``Codebase Resources`` e.g. its **code files and directories**
-- it tracks ``Discovered Packages`` e.g. its the **system and application packages** origin and
-  license discovered in the codebase
+- It has a **workspace**, which is a directory that contains the software code
+  files under analysis.
+- It makes use of one or more **code analysis pipelines** scripts to automate
+  the code analysis process.
+- It tracks ``Codebase Resources``, i.e. its **code files and directories**
+- It tracks ``Discovered Packages``, i.e. **system and application packages**
+  origin and license discovered in the codebase.
 
 In the database, **a project is identified by its unique name**.
 
@@ -25,78 +26,82 @@ In the database, **a project is identified by its unique name**.
 Project workspace
 -----------------
 
-A project workspace is the root directory where **all the project files are stored**.
+A project workspace is the root directory where **a project's files are stored**.
 
-The following directories exists under this workspace directory:
+The following directories exist under the workspace directory:
 
-- :guilabel:`input/` contains all the original uploaded and input files used of the project.
-  For instance, it could be a codebase archive.
-- :guilabel:`codebase/` contains the files and directories (aka. resources) tracked as
-  CodebaseResource records in the database.
-- :guilabel:`output/` contains all output files created by the pipelines: reports,
-  scan results, etc.
-- :guilabel:`tmp/` is a scratch pad for temporary files generated during the pipelines runs.
+- :guilabel:`input/` contains all uploaded files used as the input of a project,
+  such as a codebase archive.
+- :guilabel:`codebase/` contains files and directories - i.e. resources -
+  tracked as CodebaseResource records in the database.
+- :guilabel:`output/` contains any output files created by the pipelines,
+  including reports, scan results, etc.
+- :guilabel:`tmp/` is a scratch pad for temporary files generated during
+  pipelines runs.
 
 .. _pipelines_concept:
 
 Pipelines
 ---------
 
-A pipeline is a Python script that contains a series of steps from start to end
-to execute in order to **perform a code analysis**.
+A pipeline is a Python script that contains a series of steps, which are
+executed sequentially to **perform a code analysis**.
 
-It usually starts from the uploaded input files, and may extract these then
-generates ``CodebaseResource`` records in the database accordingly.
+It usually starts with the uploaded input files, which might need to be
+extracted first. Then, it generates ``CodebaseResource`` records in the database
+accordingly.
 
 Those resources can then be **analyzed, scanned, and matched** as needed.
 Analysis results and reports are eventually posted at the end of a pipeline run.
 
-All pipelines are located in the ``scanpipe.pipelines`` module.
-Each pipeline consist of a Python script including one subclass of the ``Pipeline`` class.
+All :ref:`built_in_pipelines` are located in the ``scanpipe.pipelines`` module.
+Each pipeline consists of a Python script and includes one subclass of the
+``Pipeline`` class.
 Each step is a method of the ``Pipeline`` class.
-The execution order of the steps is declared through the ``steps`` class attribute
-which is a sequence of steps to execute.
+The execution order of the steps - or the sequence of steps execution - is
+declared through the ``steps`` class attribute.
+
+.. tip::
+    Refer to :ref:`custom_pipelines` for adding pipelines to ScanCode.io.
 
 .. note::
     One or more pipelines can be assigned to a project as a sequence.
 
-
 Codebase Resources
 ------------------
 
 A project ``Codebase Resources`` are records of its **code files and directories**.
 ``CodebaseResource`` is a database model and each record is identified by its path
 under the project workspace.
 
-Some of the ``CodebaseResource`` interesting attributes are:
+The following are some of the ``CodebaseResource`` attributes:
 
-- a **status** used to track the analysis status for this resource.
-- a **type** (such as file, directory or symlink)
-- various attributes to track detected **copyrights**, **license expressions**,
+- A **status**, which is used to track the analysis status for this resource.
+- A **type**, such as a file, a directory or a symlink
+- Various attributes to track detected **copyrights**, **license expressions**,
   **copyright holders**, and **related packages**.
 
 .. note::
-    In general the attributes and their names are the same that are used in
-    `ScanCode-toolkit <https://github.com/nexB/scancode-toolkit>`_ for files.
-
+    Please note that `ScanCode-toolkit <https://github.com/nexB/scancode-toolkit>`_
+    use the same attributes and attribute names for files.
 
 Discovered Packages
 -------------------
 
 A project ``Discovered Packages`` are records of the **system and application packages**
-discovered in its code.
+discovered in the code unedr analysis.
 ``DiscoveredPackage`` is a database model and each record is identified by its ``Package URL``.
-``Package URL`` is a grassroot efforts to create informative identifiers for software
-packages such as Debian, RPM, npm, Maven, or PyPI packages.
-See https://github.com/package-url for details.
+``Package URL`` is a fundamental effort to create informative identifiers for
+software packages, such as Debian, RPM, npm, Maven, or PyPI packages.
+See https://github.com/package-url for more details.
 
-Some of the ``DiscoveredPackage`` interesting attributes are:
+The following are some of the ``DiscoveredPackage`` attributes:
 
-- type, name, version (all Package URL attributes)
-- homepage_url, download_url and other URLs
-- checksums (such as SHA1, MD5)
-- copyright, license_expression, declared_license
+- A type, name, version (all Package URL attributes)
+- A homepage_url, download_url, and other URLs
+- Checksums, such as SHA1, MD5
+- Copyright, license_expression, and declared_license
 
 .. note::
-    In general the attributes and their names are the same that are used in
-    `ScanCode-toolkit <https://github.com/nexB/scancode-toolkit>`_ for packages.
+    Please note that `ScanCode-toolkit <https://github.com/nexB/scancode-toolkit>`_
+    use the same attributes and attribute names for packages.