Skip to content

Commit 1285a9a

Browse files
authored
Improve the scan codebase cl tutorail (#239)
* Improve the scan codebase cl tutorail * Updating the text content to follow the docker cl tutorial * Updating the warning text in the user interface file * Adding minor updates to the scancodeio introduction file Signed-off-by: Hanan Younes <[email protected]>
1 parent 46b8489 commit 1285a9a

File tree

3 files changed

+90
-27
lines changed

3 files changed

+90
-27
lines changed

docs/introduction.rst

Lines changed: 7 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -32,16 +32,16 @@ that are adapted to a software project's unique context and technology platform
3232
can be difficult. This will require deploying and running multiple specialized
3333
tools and merge their results with a consistent workflow. Moreover,
3434
when reusing thousands of open source packages is becoming commonplace,
35-
code scans pipelines need to be scripted as code and running on servers backed
35+
code scans pipelines need to be scripted as code is running on servers backed
3636
by a shared database, not on a laptop.
3737

3838
For instance, when you analyze Docker container images, there could be hundreds
3939
to thousands of system packages, such as Debian, RPM, Alpine, and application
4040
packages, including npm, PyPI, Rubygems, Maven, installed in an image
41-
side-by-side with your own code. Taking care of all these can be
41+
side-by-side with your own code. Taking care of all this can be
4242
an extremely hard task, and that's when **ScanCode.io** comes into play to help
4343
organizing these complex code analysis as scripted pipelines and store their
44-
results in a uniform database for automated code analysis.
44+
results in a database for automated code analysis.
4545

4646

4747
What is ScanPipe?
@@ -73,9 +73,10 @@ are planning to start a new one, consider the following questions:
7373
2. **Complexity**: Does the project use many third-party components or technologies?
7474
3. **Reproducibility**: Is it important that the results are reproducible, traceable, and auditable?
7575

76-
If you answered "yes" to any of the above, keep reading - ScanPipe can help you.
77-
If the answer is "no" to all of the above, which is a valid scenario, e.g., when you
78-
are doing small-scale analysis, ScanPipe may provide only limited benefit for you.
76+
If you answered **"yes"** to any of the above, keep reading - ScanPipe can help
77+
you. If the answer is **"no"** to all of the above, which is a valid scenario,
78+
e.g., when you are doing small-scale analysis, ScanPipe may provide only limited
79+
benefit for you.
7980

8081
The first set of available pipelines helps automate the analysis of Docker
8182
container images and virtual machine (VM) disk images that often harbor

docs/tutorial-2.rst

Lines changed: 81 additions & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -2,36 +2,67 @@
22

33
Scan Codebase (command line)
44
============================
5+
The focus of this tutorial is to guide you through scanning a codebase package
6+
using ScanCode.io.
7+
8+
.. note::
9+
This tutorial assumes you have a current version of ScanCode.io installed
10+
locally on your machine. If you do not have it installed,
11+
see our :ref:`installation` guide for instructions.
512

613
Requirements
714
------------
15+
Before you follow the instructions in this tutorial, you need to:
816

9-
- **ScanCode.io is installed**, see :ref:`installation`
10-
- **Shell access** on the machine where ScanCode.io is installed
17+
- Install **ScanCode.io** locally
18+
- Download the following **package archive** and save it to your home directory: `asgiref-3.3.0-py3-none-any.whl <https://files.pythonhosted.org/packages/c0/e8/578887011652048c2d273bf98839a11020891917f3aa638a0bc9ac04d653/asgiref-3.3.0-py3-none-any.whl>`_
19+
- Have **Shell access** on the machine where ScanCode.io is installed
1120

21+
Instructions
22+
------------
1223

13-
Before you start
14-
----------------
24+
- Open a shell in the ScanCode.io installation directory and activate the
25+
virtual environment - **virtualenv**:
1526

16-
Download the following package archive save this in your home directory:
17-
`asgiref-3.3.0-py3-none-any.whl <https://files.pythonhosted.org/packages/c0/e8/578887011652048c2d273bf98839a11020891917f3aa638a0bc9ac04d653/asgiref-3.3.0-py3-none-any.whl>`_
27+
.. code-block:: console
1828
29+
$ source bin/activate
1930
20-
Step-by-step
21-
------------
31+
.. code-block:: console
2232
23-
- Open a shell in the ScanCode.io installation directory and activate the virtualenv::
33+
>> (scancodeio) $
2434
25-
$ source bin/activate
35+
- Create a new project named ``asgiref``:
2636

27-
- The following command will create a new project named ``asgiref``,
28-
add the archive as an input for the project,
29-
add the ``scan_codebase`` pipeline, and execute it::
37+
.. code-block:: console
3038
31-
$ scanpipe create-project asgiref \
32-
--input-file ~/asgiref-3.3.0-py3-none-any.whl \
33-
--pipeline scan_codebase \
34-
--execute
39+
$ scanpipe create-project asgiref
40+
41+
.. code-block:: console
42+
43+
>> Project asgiref created with work directory projects/asgiref-072c89db
44+
45+
- Add the package archive to the project workspace's :guilabel:`input/`
46+
directory:
47+
48+
.. code-block:: bash
49+
50+
$ scanpipe add-input --project asgiref --input-file ~/asgiref-3.3.0-py3-none-any.whl
51+
52+
.. code-block:: console
53+
54+
>> File(s) copied to the project inputs directory:
55+
- asgiref-3.3.0-py3-none-any.whl
56+
57+
- Add the ``scan_codebase`` pipeline to your project:
58+
59+
.. code-block:: console
60+
61+
$ scanpipe add-pipeline --project asgiref scan_codebase
62+
63+
.. code-block:: console
64+
65+
>> Pipeline(s) added to the project
3566
3667
.. note::
3768
The content of the :guilabel:`input/` directory will be copied in the
@@ -41,5 +72,36 @@ Step-by-step
4172
:guilabel:`codebase/` directory in which case the ``--input`` option can be
4273
omitted.
4374

44-
- The scan results as JSON and CSV will be available in the project
45-
:guilabel:`output/` directory.
75+
- Run the ``scan_codebase`` pipeline on your project. The pipeline execution
76+
progress is shown within the following command's output:
77+
78+
.. code-block:: bash
79+
80+
$ scanpipe execute --project asgiref
81+
82+
.. code-block:: console
83+
84+
>> Pipeline scan_codebase run in progress..
85+
2021-07-12 17:45:53.85 Pipeline [scan_codebase] starting
86+
2021-07-12 17:45:53.85 Step [copy_inputs_to_codebase_directory] starting
87+
2021-07-12 17:45:53.86 Step [copy_inputs_to_codebase_directory] completed in 0.00 seconds
88+
2021-07-12 17:45:53.86 Step [run_extractcode] starting
89+
[...]
90+
2021-07-12 17:46:01.61 Pipeline completed
91+
92+
- Finally, you can view your scan results in JSON or CSV file formats inside
93+
the project's :guilabel:`output/` directory.
94+
95+
.. tip::
96+
The ``inputs`` and ``pipelines`` can be provided at the same time when
97+
calling the ``create-project`` command. For instance, the following command
98+
will create a new project named ``asgiref``, add the package archive as the
99+
project input, add the ``scan_codebase`` pipeline to the project, and
100+
execute it:
101+
102+
.. code-block:: bash
103+
104+
$ scanpipe create-project asgiref \
105+
--input-file ~/asgiref-3.3.0-py3-none-any.whl \
106+
--pipeline scan_codebase \
107+
--execute

docs/user-interface.rst

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -90,8 +90,8 @@ to your project by clicking the **"Add inputs"** and **"Add pipeline"** buttons.
9090
.. image:: images/UI-4-new.png
9191

9292
.. warning::
93-
You will not be able to add more inputs once a pipeline has been run on the project.
94-
You can add and run more pipelines though.
93+
You will not be able to add any extra inputs once a pipeline has been run on
94+
the project. However, you still can add and run extra pipelines as needed!
9595

9696
Within each project, you can view your project details, review the results of
9797
the pipeline execution, or download the output files.

0 commit comments

Comments
 (0)