Skip to content

Commit 790adc9

Browse files
pombredannepombredanne
committed
Improve README
 * this is a significant rework of the REDAME and there is more doc come for v3.0 documentation Signed-off-by: Philippe Ombredanne <[email protected]>
1 parent b68d378 commit 790adc9

File tree

1 file changed

+138
-117
lines changed

1 file changed

+138
-117
lines changed

‎README.rst‎

Lines changed: 138 additions & 117 deletions
Original file line numberDiff line numberDiff line change
@@ -2,171 +2,192 @@
22
ScanCode toolkit
33
================
44

5+
A typical software project often reuses hundreds of third-party packages.
6+
License and origin information is often scattered, not easy to find and not
7+
normalized: ScanCode discovers and normalizes this data for you.
8+
9+
ScanCode is a suite of command line utilities to reliably scan a codebase for
10+
license, copyright, package manifests and direct dependencies and other
11+
interesting origin and licensing information discovered in source and binary
12+
code files.
13+
14+
ScanCode is used by several projects and organizations such as the Eclipse
15+
Foundation, Here.com Open Source Review Toolkit, ClearlyDefined and RedHat
16+
Fabric8 analytics.
17+
18+
ScanCode provides comprehensive scan results that you can save as JSON, HTML,
19+
CSV or SPDX. And you can use the companion `AboutCode Manager GUI app
20+
<https://github.com/nexB /aboutcode-manager>`_ to review, search and display
21+
scan results, statistics and graphics.
22+
23+
ScanCode is programed primarily in Python (with some C/C++ when performance is
24+
critical). License and copyright detection use multiple techniques borrowed from
25+
NLP, ML and information retrieval such as feature extraction, probablistic
26+
searches using inverted indexes, multi-patterns automatons and multiple local
27+
sequence alignments for comprehensive, accurate and reasonably fast scanning.
28+
ScanCode is easily extensible with plugins to contribute new and improved
29+
scanner, data summarization and outputs.
30+
31+
As a command line application returning JSON, ScanCode is easy to integrate in
32+
a code analysis pipeline and Ci/CD.
33+
34+
We are continuously working on new features, such as detecting more package
35+
manifests or improving scanning accuracy and performance and welcome
36+
contributions.
37+
38+
See our roadmap for upcoming features:
39+
https://github.com/nexB/scancode-toolkit/wiki/Roadmap
540

641
Build and tests status
742
======================
843

9-
+-------+--------------------------------------------------------------------------------------+-----------------------------------------------------------------------------+-----------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------+
10-
|Branch | **Coverage** | **Linux (Travis)** | **MacOSX (Travis)** | **Windows (AppVeyor)** |
11-
+=======+======================================================================================+=============================================================================+=============================================================================+===============================================================================================+
12-
| |.. image:: https://codecov.io/gh/nexB/scancode-toolkit/branch/master/graph/badge.svg |.. image:: https://api.travis-ci.org/nexB/scancode-toolkit.png?branch=master |.. image:: https://api.travis-ci.org/nexB/scancode-toolkit.png?branch=master |.. image:: https://ci.appveyor.com/api/projects/status/4webymu0l2ip8utr/branch/master?png=true |
13-
|Master | :target: https://codecov.io/gh/nexB/scancode-toolkit/branch/master | :target: https://travis-ci.org/nexB/scancode-toolkit | :target: https://travis-ci.org/nexB/scancode-toolkit | :target: https://ci.appveyor.com/project/nexB/scancode-toolkit |
14-
| | :alt: Linux Master branch test coverage | :alt: Linux Master branch tests status | :alt: MacOSX Master branch tests status | :alt: Windows Master branch tests status |
15-
+-------+--------------------------------------------------------------------------------------+-----------------------------------------------------------------------------+-----------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------+
16-
| |.. image:: https://codecov.io/gh/nexB/scancode-toolkit/branch/develop/graph/badge.svg |.. image:: https://api.travis-ci.org/nexB/scancode-toolkit.png?branch=develop|.. image:: https://api.travis-ci.org/nexB/scancode-toolkit.png?branch=develop|.. image:: https://ci.appveyor.com/api/projects/status/4webymu0l2ip8utr/branch/develop?png=true|
17-
|Develop| :target: https://codecov.io/gh/nexB/scancode-toolkit/branch/develop | :target: https://travis-ci.org/nexB/scancode-toolkit | :target: https://travis-ci.org/nexB/scancode-toolkit | :target: https://ci.appveyor.com/project/nexB/scancode-toolkit |
18-
| | :alt: Linux Develop branch test coverage | :alt: Linux Develop branch tests status | :alt: MacOSX Develop branch tests status | :alt: Windows Develop branch tests status |
19-
+-------+--------------------------------------------------------------------------------------+-----------------------------------------------------------------------------+-----------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------+
20-
21-
22-
ScanCode is a suite of utilities used to scan a codebase for license,
23-
copyright, package manifests and dependencies and other interesting
24-
information that can be discovered in source and binary code files.
25-
26-
A typical software project often reuses hundreds of third-party
27-
packages. License and origin information is often scattered, not
28-
easy to find and not normalized: ScanCode discovers and normalizes
29-
this data for you.
30-
31-
ScanCode provides accurate scan results and the line position
32-
where each result is found. The results can be formatted as JSON or
33-
HTML. ScanCode provides a simple HTML app for quick visualization of
34-
scan results (see screenshot below), but you will have more robust analysis
35-
options if you use AboutCode Manager to view a scan. AboutCode Manager is
36-
a desktop application available on Linux, OSX or Windows - go to
37-
https://github.com/nexB/aboutcode-manager to learn more or to download
38-
AboutCode Manager.
39-
40-
We are continuously working on new features, such as detecting more
41-
package manifests or improving scanning accuracy and performance.
42-
This is made easier by the recent addition of a plugin architecture.
43-
44-
See the roadmap for upcoming features:
45-
https://github.com/nexB/scancode-toolkit/wiki/Roadmap
46-
47-
.. image:: samples/screenshot.png
44+
+-------+--------------+-----------------+--------------+
45+
|Branch | **Coverage** | **Linux/macOS** | **Windows** |
46+
+=======+==============+=================+==============+
47+
|Master | |master-cov| | |master-posix| | |master-win| |
48+
+-------+--------------+-----------------+--------------+
49+
|Develop| |devel-cov| | |devel-posix| | |devel-win| |
50+
+-------+--------------+-----------------+--------------+
4851

4952

5053
Quick Start
5154
===========
5255

53-
For Windows, please go to the
54-
`Comprehensive Installation <https://github.com/nexB/scancode-toolkit/wiki/Comprehensive-Installation>`_
55-
section instead.
56+
Install Python 2.7 then download and extract the latest ScanCode release
57+
https://github.com/nexB/scancode-toolkit/releases/
5658

57-
Make sure you have Python 2.7 installed:
58-
* Download and install Python 2.7 32 bits for Windows
59-
https://www.python.org/ftp/python/2.7.13/python-2.7.13.msi
60-
* Download and install Python 2.7 for Mac
61-
https://www.python.org/ftp/python/2.7.13/python-2.7.13-macosx10.6.pkg
59+
Then run `./scancode -h` for help.
6260

63-
On Linux install Python 2.7 "devel" and a few extra packages:
64-
65-
* ``sudo apt-get install python-dev bzip2 xz-utils zlib1g libxml2-dev libxslt1-dev``
66-
for Ubuntu 12.04, 14.04 and 16.04
6761

68-
* ``sudo apt-get install python-dev libbz2-1.0 xz-utils zlib1g libxml2-dev libxslt1-dev``
69-
for Debian and Debian-based distros
70-
71-
* ``sudo yum install python-devel zlib bzip2-libs xz-libs libxml2-devel libxslt-devel``
72-
for RPM distros
73-
74-
* ``sudo dnf install python-devel zlib bzip2-libs xz-libs libxml2-devel libxslt-devel``
75-
for Fedora 22 and later
62+
Installation
63+
============
7664

77-
* See the Comprehensive Installation for additional details and other
78-
Linux installations: https://github.com/nexB/scancode-toolkit/wiki/Comprehensive-Installation
65+
Pre-requisites:
7966

80-
Next, download and extract the latest ScanCode release from::
67+
* On Windows, please follow the `Comprehensive Installation instructions
68+
<https://github.com/nexB/scancode-toolkit/wiki/Comprehensive-Installation>`_.
69+
Make sure you use Python 2.7 32 bits from
70+
https://www.python.org/ftp/python/2.7.14/python-2.7.14.msi
8171

82-
https://github.com/nexB/scancode-toolkit/releases/
72+
* On macOS, install Python 2.7 from
73+
https://www.python.org/ftp/python/2.7.14/python-2.7.14-macosx10.6.pkg
8374

84-
Open a terminal, extract the downloaded release archive, then `cd` to
85-
the extracted directory and run this command to display the command
86-
help. ScanCode will self-configure if needed::
75+
Next, download and extract the latest ScanCode release from
76+
https://github.com/nexB/scancode-toolkit/releases/
8777

88-
./scancode --help
78+
* On Linux install the Python 2.7 "devel" and these packages using your
79+
distribution package manager:
8980

90-
Run a sample scan saved to the `samples.html` file::
81+
* On Ubuntu 14.04 and 16.04 use:
82+
``sudo apt-get install python-dev bzip2 xz-utils zlib1g libxml2-dev libxslt1-dev``
9183

92-
./scancode --html-app samples.html samples
84+
* On Debian and Debian-based distros use:
85+
``sudo apt-get install python-dev libbz2-1.0 xz-utils zlib1g libxml2-dev libxslt1-dev``
9386

94-
Then open `samples.html` in your web browser to view the scan results.
87+
* On RPM distros use:
88+
``sudo yum install python-devel zlib bzip2-libs xz-libs libxml2-devel libxslt-devel``
9589

96-
See more command examples::
90+
* On Fedora 22 and later use:
91+
``sudo dnf install python-devel zlib bzip2-libs xz-libs libxml2-devel libxslt-devel``
9792

98-
./scancode --examples
93+
* See also the `Comprehensive Installation instructions
94+
<https://github.com/nexB/scancode-toolkit/wiki/Comprehensive-Installation>`_
95+
for additional instructions.
9996

10097

101-
Support
102-
=======
98+
Next, download and extract the latest ScanCode release from
99+
https://github.com/nexB/scancode-toolkit/releases/
103100

104-
If you have a problem, a suggestion or found a bug, please enter a ticket at:
105-
https://github.com/nexB/scancode-toolkit/issues
101+
Open a terminal window and then `cd` to the extracted ScanCode directory and run
102+
this command to display help. ScanCode will self-configure if needed::
106103

107-
For other questions, discussions, and chats, we have:
104+
./scancode --help
108105

109-
- a mailing list at https://lists.sourceforge.net/lists/listinfo/aboutcode-discuss
106+
You can run an example scan printed on screen as JSON::
110107

111-
- an official Gitter channel at https://gitter.im/aboutcode-org/discuss
112-
Gitter also has an IRC bridge at https://irc.gitter.im/
113-
114-
- an official #aboutcode IRC channel on freenode (server chat.freenode.net)
115-
for scancode and other related tools. Note that this receives
116-
notifactions from repos so it can be a tad noisy. You can use your
117-
favorite IRC client or use the web chat at
118-
https://webchat.freenode.net/
108+
./scancode --clip --json-pp - samples
119109

110+
See more command examples::
120111

121-
About archives
122-
==============
112+
./scancode --examples
123113

124-
All code must be extracted before running ScanCode since ScanCode will
125-
not extract files from tarballs, zip files, etc. However, the ScanCode
126-
Toolkit comes with a utility called extractcode that does recursive
127-
archive extraction. For example, this command will recursively extract
128-
the mytar.tar.bz2 tarball in the mytar.tar.bz2-extract directory::
129114

130-
./extractcode mytar.tar.bz2
115+
Archives extraction
116+
===================
131117

118+
The archives that exist in a codebase must be extracted before running a scan:
119+
ScanCode does not extract files from tarballs, zip files, etc. as part of the
120+
scan. The bundled utility `extractcode` is a mostly-universal archive extractor.
121+
For example, this command will recursively extract the mytar.tar.bz2 tarball in
122+
the mytar.tar.bz2-extract directory::
132123

133-
Source code
134-
===========
124+
./extractcode mytar.tar.bz2
135125

136-
* https://github.com/nexB/scancode-toolkit.git
137-
* https://github.com/nexB/scancode-thirdparty-src.git
138126

127+
Documentation & FAQ
128+
===================
139129

140-
License
141-
=======
130+
https://github.com/nexB/scancode-toolkit/wiki
142131

143-
* Apache-2.0 with an acknowledgement required to accompany the scan output.
144-
* Public domain CC-0 for reference datasets.
145-
* Multiple licenses (GPL2/3, LGPL, MIT, BSD, etc.) for third-party components.
132+
See also https://aboutcode.org for related companion projects and tools.
146133

147-
See the NOTICE file for more details.
148134

135+
Support
136+
=======
149137

150-
Documentation & FAQ
151-
===================
138+
If you have a problem, a suggestion or found a bug, please enter a ticket at:
139+
https://github.com/nexB/scancode-toolkit/issues
152140

153-
https://github.com/nexB/scancode-toolkit/wiki
141+
For discussions and chats, we have:
154142

143+
* an official Gitter channel for web-based chats at https://gitter.im/aboutcode-org/discuss
144+
Gitter is also accessible via an IRC bridge at https://irc.gitter.im/
155145

156-
Basic Usage
157-
===========
146+
* an official `#aboutcode` IRC channel on freenode (server chat.freenode.net).
147+
This channel receives build and commit notifactions and can be a tad noisy.
148+
You can use your favorite IRC client or use the web chat at
149+
https://webchat.freenode.net/
158150

159-
Run this command for a list of options (On Windows use `scancode`
160-
instead of `./scancode`)::
151+
* a mailing list at https://lists.sourceforge.net/lists/listinfo/aboutcode-discuss
161152

162-
./scancode --help
163153

164-
Run this command for a list of command line examples::
154+
Source code and downloads
155+
=========================
165156

166-
./scancode --examples
157+
* https://github.com/nexB/scancode-toolkit.git
158+
* https://github.com/nexB/scancode-toolkit/releases
159+
* https://pypi.org/project/scancode-toolkit/
160+
* https://github.com/nexB/scancode-thirdparty-src.git
167161

168-
To run a license scan on sample data, first run this::
169162

170-
./scancode -l --html-app samples.html samples
163+
License
164+
=======
171165

172-
Then open samples.html in your web browser to see the results.
166+
* Apache-2.0 with an acknowledgement required to accompany the scan output.
167+
* Public domain CC-0 for reference datasets.
168+
* Multiple licenses (GPL2/3, LGPL, MIT, BSD, etc.) for third-party components.
169+
170+
See the NOTICE file and the .ABOUT files that document the origin and license of
171+
the third-party code used in ScanCode for more details.
172+
173+
174+
.. |master-cov| image:: https://codecov.io/gh/nexB/scancode-toolkit/branch/master/graph/badge.svg
175+
:target: https://codecov.io/gh/nexB/scancode-toolkit/branch/master
176+
:alt: Master branch test coverage (Linux)
177+
.. |devel-cov| image:: https://codecov.io/gh/nexB/scancode-toolkit/branch/develop/graph/badge.svg
178+
:target: https://codecov.io/gh/nexB/scancode-toolkit/branch/develop
179+
:alt: Develop branch test coverage (Linux)
180+
181+
.. |master-posix| image:: https://api.travis-ci.org/nexB/scancode-toolkit.png?branch=master
182+
:target: https://travis-ci.org/nexB/scancode-toolkit
183+
:alt: Linux Master branch tests status
184+
.. |devel-posix| image:: https://api.travis-ci.org/nexB/scancode-toolkit.png?branch=develop
185+
:target: https://travis-ci.org/nexB/scancode-toolkit
186+
:alt: Linux Develop branch tests status
187+
188+
.. |master-win| image:: https://ci.appveyor.com/api/projects/status/4webymu0l2ip8utr/branch/master?png=true
189+
:target: https://ci.appveyor.com/project/nexB/scancode-toolkit
190+
:alt: Windows Master branch tests status
191+
.. |devel-win| image:: https://ci.appveyor.com/api/projects/status/4webymu0l2ip8utr/branch/develop?png=true
192+
:target: https://ci.appveyor.com/project/nexB/scancode-toolkit
193+
:alt: Windows Develop branch tests status

0 commit comments

Comments
 (0)