Merge pull request #1 from nexB/develop

Updating my fork

Author: akugarg

.gitignore

@@ -85,3 +85,6 @@ tcl
 
 *.orig
 /release*
+/00-*.txt
+/z-todo-licenses-*
+

AUTHORS.rst

@@ -1,24 +1,94 @@
 The following organizations or individuals have contributed to ScanCode:
 
+- Abhigyan Kumar Singh @Abhigyankrsingh
+- Abhishek Kumar @Abhishek-Dev09
+- Aditya Viki @adityaviki
+- Agni Bhattacharyya @PyAgni
+- Akanksha Garg @akugarg
+- Alex Blekhman @a-tinsmith
+- Armijn Hemmel @armijnhemel
+- Arnaud Jeansen @ajeans
+- Arnav Mandal @arnav-mandal1234
+- Aryan Kenchappagol @aryanxk02
+- Avi Aryan @aviaryan
+- Aviral Verma @avirlrma
+- Ayan Sinha Mahapatra @AyanSinhaMahapatra
+- Ayush Jain @aj4ayushjain
+- Bruno Oliveira @nicoddemus
+- Carmen Bianca Bakker @carmenbianca
+- Chaitya Shah @Chaitya62
 - Chin-Yeung Li @chinyeungli
+- Clement Poulain @FOSS117
+- Daniel Eder @daniel-eder
+- Dan Kegel @dankegel
+- Dennis Clark @DennisClark
+- Divyansh Sharma @Divyansh2512
+- Duncan Howe @Duncan-Howe
+- Felix Kauselmann @selmf
+- François Granade @farialima
+- Frank Viernau @fviernau
+- Gaurang Rao @Gaupeng
+- Hanif Ali @hanif-ali
+- Horie Issei @is2ei
+- James Ward @jamesward
+- Jelmer Vernooij @jelmer
 - Jillian Daguil @jdaguil
+- Jiri Popelka @jpopelka
+- Johannes Najjar @joshovi
+- Jose Nazario @paralax
+- Lemo Shi @lemoshi
 - Li Ha @linexb
+- Mankaran Singh @MankaranSingh
+- Martin Petkov @MartinPetkov
+- Maximilian Huber @maxhbr 
+- Michael Herzog @mjherzog
+- Michael Rupprecht @michaelrup
+- Mrinal Paliwal @mnpw
 - nexB Inc. @nexB
+- Nirmal Sarswat @vivonk
+- Nisha Kumar @nishakm
+- Nishchith Shetty @inishchith
+- Nitish Sharma @nitish81299
+- Paul Gier @pgier
 - Philippe Ombredanne @pombredanne
-- Sebastian Roth @ened
-- Steven Esser @majurg
-- Sebastian Schuberth @sschuberth
+- Pi Delport @PiDelport
+- Pierre Lapointe @pierrelapointe
+- Pierre Tardy @tardyp
+- Pranamika Pandey @Pihu1998
+- Pratik Dey @Pratikrocks
+- Qingmin Duanmu @qduanmu
 - Rakesh Balusa @balusarakesh
-- Francois Granade @farialima
-- Savino Sguera @savinos
-- Yash Nisar @yash-nisar
-- Nirmal Sarswat @vivonk
-- Tushar Mittal @techytushar
-- Martin Petkov @MartinPetkov
-- Mrinal Paliwal @mriiinal
-- Mankaran Singh @MankaranSingh
+- Ranvir Singh @singh1114
 - Ravi Jain @JRavi2
-- Gaurang Rao @Gaupeng
-- Tushar Goel @TG1999
+- Richard Fontana @richardfontana
 - Richard Menzies @WizardOhio24
-- Daniel Eder @daniel-eder
+- Ritiek Malhotra @ritiek
+- Rohit Potter @rpotter12
+- Sankha Das @sankha555
+- Saravanan G @SaravananOffl
+- Sarita Singh @itssingh
+- Savino Sguera @savinos
+- Sebastian Roth @ened
+- Sebastian Schuberth @sschuberth
+- Shankhadeep Dey @Iamshankhadeep
+- Sharikzama @ZamaSharik
+- Shivam Chauhan @chashiv
+- Shivam Sandbhor @sbs2001
+- Steven Esser @majurg
+- Sushant Gupta @susg
+- Thomas Druez @tdruez
+- Thomas Steenbergen @tsteenbe
+- Thorsten Harter @ThorstenHarter
+- Till Jaeger @LeChasseur 
+- Tobias Furuholm @furuholm
+- Tushar Goel @TG1999
+- Tushar Mittal @techytushar
+- Tushar Upadhyay @tushar912
+- Van Lindberg @VanL
+- Vibhu Agarwal @Vibhu-Agarwal
+- Viktor Tiulpin @tiulpin
+- Vinay Kumar Singh @Vinay0001 
+- Virag Umathe @viragumathe5
+- Yash D. Saraf @yashdsaraf
+- Yash Nisar @yash-nisar
+- Yash Sharma @yasharmaster

CHANGELOG.rst

@@ -1,21 +1,147 @@
 Changelog
 =========
 
+v21.4.x (next)
+--------------
 
-v21.3.x (next)
-------------
+Breaking API changes:
+~~~~~~~~~~~~~~~~~~~~~
 
+ - The data structure of the JSON output has changed for copyrights, authors
+   and holders: we now use proper name for attributes and not a generic "value".
 
-Misc.:
+ - The data structure of the JSON output has changed for licenses: we now
+   return match details once for each matched license expression rather than
+   once for each license in a matched expression. There is a new top-level
+   "licenses" attributes that contains the data details for each detected
+   licenses only once. This data can contain the reference license text
+   as an option.
+
+ - The data structure of the JSON output has changed for packages: we now
+   return "package_manifests" package information at the manifest file-level
+   rather than "packages". There is a a new top-level "packages" attribute
+   that contains each package instace that can be aggregating data from
+   multiple manifests for a single package instance.
+
+
+Ouputs:
+~~~~~~~
+
+ - Add new YAML-formatted output. This is exactly the same data structure as for
+   the JSON output
+
+
+License scanning:
+~~~~~~~~~~~~~~~~~
+
+ - Add new command line option to filter ignorable copyrights when included
+   in licenses.
+
+
+
+v21.3.31
+--------
+
+This is a major version with no breaking API changes. Heads-up: the next version
+will bring up some significant API changes summarized above.
+
+
+Security:
+~~~~~~~~~
+
+ - Update dependency versions for security fixes.
+
+
+License scanning:
+~~~~~~~~~~~~~~~~~
+
+ - Add 22 new licenses and update 71 existing licenses
+
+ - Update licenses to include the SPDX license list 3.12
+
+ - Improve license detection accuracy with over 2,300 new and updated license
+   detection rules
+
+ - Undeprecate the regexp license and deprecate the hs-regexp-orig license
+
+ - Improve license db initial load time with caching for faster scancode
+   start time
+
+ - Add experimental SCANCODE_LICENSE_INDEX_CACHE environment variable to point
+   to an alternative directory where the license index cache is stored (as
+   opposed to store this as package data.)
+
+ - Ensure that license short names are not more than 50 characters long
+
+ - Thank you to:
+    - Dennis Clark @DennisClark
+    - Chin-Yeung Li @chinyeungli
+    - Armijn Hemmel @armijnhemel
+    - Sarita Singh @itssingh
+    - Akanksha Garg @akugarg
+
+
+Copyright scanning:
+~~~~~~~~~~~~~~~~~~~
+
+ - Detect SPDX-FileCopyrightText as defined by the FSFE Reuse project
+   Thank you to Daniel Eder @daniel-eder
 
  - Fix bug when using the --filter-clues command line option
    Thank you to Van Lindberg @VanL
 
+ - Allow calling copyright detection from text lines to ease integration
+   Thank you to Jelmer Vernooij @jelmer
+
+
+Package scanning:
+~~~~~~~~~~~~~~~~~
+
+ - Add support for installed RPMs detection internally (not wired to scans)
+   Thank you to Chin-Yeung Li @chinyeungli
+
+ - Improve handling of Debian copyright files with faster and more
+   accurate license detection
+   Thank you to Thomas Druez @tdruez 
+   
+ - Add new built-in support for installed_files report. Only available when
+   used as a library.
+
+ - Improve support for RPM, npm, Debian, build scripts (Bazel) and Go packages
+   Thank you to:
+   - Divyansh Sharma @Divyansh2512
+   - Jonothan Yang @JonoYang
+   - Steven Esser @majurg
+
+ - Add new support to collect information from semi-structured Readme files
+   and related metadata files. 
+   Thank you to Jonothan Yang @JonoYang and Steven Esser @majurg
+
+
+Ouputs:
+~~~~~~~
+
+ - Add new Debian copyright-formatted output.
+   Thank you to Jelmer Vernooij @jelmer
+   
+ - Fix bug in --include where directories where not skipped correctly
+   Thank you to Pierre Tardy @tardyp
+
+
+Misc. and documentation improvements:
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+ - Update the way tests assertions are made
+   Thank you to Aditya Viki @adityaviki
+
+ - Thank you to Aryan Kenchappagol @aryanxk02
+
 
 v21.2.25
 --------
 
 Installation:
+~~~~~~~~~~~~~
 
  - Resolve reported installation issues on macOS, Windows and Linux
  - Stop using extras for a default wheel installation
@@ -25,6 +151,7 @@ Installation:
    Thank you to Viktor Tiulpin @tiulpin
 
 License scanning:
+~~~~~~~~~~~~~~~~~
 
  - Over 150 new and updated licenses
  - Support the latest SPDX license list v3.11
@@ -33,7 +160,7 @@ License scanning:
  - Fix license cache handling issues
 
 Misc.:
-
+~~~~~~
  - Update extractcode, typecode and their native dependencies for better support
    of latests versions of macOS.
 
@@ -42,12 +169,14 @@ v21.2.9
 -------
 
 Security:
+~~~~~~~~~
 
  - Update vulnerable LXML to version 4.6.2 to fix
    https://nvd.nist.gov/vuln/detail/CVE-2020-27783
    This was detected thanks to https://github.com/nexb/vulnerablecode
 
 Operating system support:
+~~~~~~~~~~~~~~~~~~~~~~~~~
 
  - Drop support for Python 2  #295
  - Drop support for 32 bits on Windows #335
@@ -57,6 +186,7 @@ Operating system support:
  - Add deprecation message for native Windows support #2366
 
 License scanning:
+~~~~~~~~~~~~~~~~~
 
  - Improve license detection accuracy with over 8400 new license detection rules
    added or updated
@@ -70,17 +200,20 @@ License scanning:
    statement (that typically may be reported as some unknown license)
 
 Package scanning:
+~~~~~~~~~~~~~~~~~
 
  - Add detection of package-installed files
  - Add analysis of system package installed databases for Debian, OpenWRT and
    Alpine Linux packages
  - Add support for Alpine Linux, Debian, OpenWRT.
 
 Copyright scanning:
+~~~~~~~~~~~~~~~~~~~
 
  - Improve detection with minor grammar fixes
 
 Misc.:
+~~~~~~
 
  - Adopt a new calendar date-based versioning for scancode-toolkit version numbers
  - Update thirdparty dependencies and built-in plugins
@@ -97,6 +230,7 @@ v3.2.3 (2020-10-27)
 -------------------
 
 Notable changes:
+~~~~~~~~~~~~~~~~
 
  - Collect Windows executable metadata #652
  - Fix minor bugs
@@ -107,6 +241,7 @@ v3.2.2rc3 (2020-09-21)
 ----------------------
 
 Notable changes:
+~~~~~~~~~~~~~~~~
 
  - Use commoncode, typecode and extractcode as external standalone packages #2233
 
@@ -115,6 +250,7 @@ v3.2.1rc2 (2020-09-11)
 ----------------------
 
 Minor bug fixes:
+~~~~~~~~~~~~~~~~
 
  - Do not fail if Debian status is missing #2224
  - Report correct detected license text in binary #2226 #2227

NOTICE

@@ -34,6 +34,7 @@ including:
 - https://github.com/nexB/thirdparty-packages/pypi/
 - https://github.com/nexB/scancode-plugins/
 - https://github.com/nexB/scancode-thirdparty-src/
+- https://github.com/nexB/thirdparty-packages/
 
 You may also contact us to request the source code by email at [email protected] or
 by postal mail at:

configure

@@ -52,7 +52,7 @@ export THIRDPARTY_LINKS="https://thirdparty.aboutcode.org/pypi"
 # requirements used by default or with --dev.
 # note the use of constraints with -c 
 REQUIREMENTS="--editable . --constraint requirements.txt"
-DEV_REQUIREMENTS="--editable .[dev] --constraint requirements.txt --constraint requirements-dev.txt"
+DEV_REQUIREMENTS="--editable .[dev] --editable .[packages] --constraint requirements.txt --constraint requirements-dev.txt"
 
 # default supported Python version
 if [[ "$CONFIGURE_SUPPORTED_PYTHON" == "" ]]; then