Fixed #519 - Remove the ASCII Limitation

 * Note that I still tend to keep ASCII-only for field name
 * This commit also reove the test pipeline for python3.6

Signed-off-by: Chin Yeung Li <[email protected]>

Author: chinyeungli

.travis.yml

@@ -10,7 +10,6 @@ dist: xenial
 
 language: python
 python:
-  - "3.6"
   - "3.7"
   - "3.8"
   - "3.9"

CHANGELOG.rst

@@ -1,11 +1,15 @@
 ==============================
 Changelog
 2023-xx-xx
-    Release 7.3.0
+    Release 8.0.0
 
     * Fixed the transform code for xlsx and json
     * Remove irrelevant error for attrib
     * Add support to identify worksheet name for XLSX input
+    * The severity error level for "contains illegal name characters (or empty spaces) and is ignored" is changed from ERROR to WARNING
+    * Remove the limitation to ASCII only
+    * Drop support for python3.6
+    * Update valid chatacters for file/path name
 
 
 2022-10-24

Dockerfile

@@ -6,7 +6,7 @@
 # See https://aboutcode.org for more information about nexB OSS projects.
 #
 
-FROM python:3.6-slim-buster 
+FROM python:3.7-slim-buster
 
 RUN apt-get update \
  && apt-get install -y bash bzip2 xz-utils zlib1g libxml2-dev libxslt1-dev libgomp1 libpopt0\
@@ -27,7 +27,7 @@ RUN bash -c "source ./configure"
 # Add aboutcode to path
 #ENV PATH=$HOME/aboutcode-toolkit:$PATH
 
-# Set entrypoint to be the aboutcode command, allows to run the generated docker image directly with the aboutcode arguments: 
+# Set entrypoint to be the aboutcode command, allows to run the generated docker image directly with the aboutcode arguments:
 # `docker run (...) <containername> <about arguments>`
 # Example: docker run --rm --name "aboutcode" -v ${PWD}:/project -v /tmp/result:/result aboutcode-toolkit attrib /project /result/c.html
 ENTRYPOINT ["./bin/about"]

README.rst

@@ -21,7 +21,7 @@ In addition, this tool is able to generate attribution notices and
 identify redistributable source code used in your project to help you comply
 with open source licenses conditions.
 
-This version of the AboutCode Toolkit follows the ABOUT specification version 3.2.3 at:
+This version of the AboutCode Toolkit follows the ABOUT specification version 3.3.0 at:
 https://aboutcode-toolkit.readthedocs.io/en/latest/specification.html
 
 
@@ -39,7 +39,7 @@ Build and tests status
 
 REQUIREMENTS
 ------------
-The AboutCode Toolkit is tested with Python 3.6.2 or above only on Linux, Mac and Windows.
+The AboutCode Toolkit is tested with Python 3.7 or above only on Linux, Mac and Windows.
 You will need to install a Python interpreter if you do not have one already
 installed.
 
@@ -69,11 +69,6 @@ To install all the needed dependencies in a virtualenv, run (on posix):
 or on windows:
     configure
 
-Note
-~~~~
-   For MacOS users, it's a known issue the Python36 may case SSL Certificates error if the Certificates is not up to date.
-   A solution is to run: `sudo /Applications/Python\\ 3.6/Install\\ Certificates.command` to upgrade the outdated certificates.
-
 ACTIVATE the VIRTUALENV
 -----------------------
 To activate the virtualenv, run (on posix):

about.ABOUT

@@ -1,6 +1,6 @@
 about_resource: .
 name: AboutCode-toolkit
-version: 7.3.0
+version: 8.0.0
 author: Jillian Daguil, Chin Yeung Li, Philippe Ombredanne, Thomas Druez
 copyright: Copyright (c) nexB Inc.
 description: |

azure-pipelines.yml

@@ -11,15 +11,15 @@ jobs:
       parameters:
           job_name: ubuntu18_cpython
           image_name: ubuntu-18.04
-          python_versions: ['3.6', '3.7', '3.8', '3.9', '3.10']
+          python_versions: ['3.7', '3.8', '3.9', '3.10']
           test_suites:
               all: venv/bin/pytest -n 2 -vvs
 
     - template: etc/ci/azure-posix.yml
       parameters:
           job_name: ubuntu20_cpython
           image_name: ubuntu-20.04
-          python_versions: ['3.6', '3.7', '3.8', '3.9', '3.10']
+          python_versions: ['3.7', '3.8', '3.9', '3.10']
           test_suites:
               all: venv/bin/pytest -n 2 -vvs
 
@@ -53,7 +53,7 @@ jobs:
       parameters:
           job_name: win2019_cpython
           image_name: windows-2019
-          python_versions: ['3.6', '3.7', '3.8', '3.9', '3.10']
+          python_versions: ['3.7', '3.8', '3.9', '3.10']
           test_suites:
               all: venv\Scripts\pytest -n 2 -vvs
 

docs/source/home.rst

@@ -20,7 +20,7 @@ In addition, this tool is able to generate attribution notices and
 identify redistributable source code used in your project to help you comply
 with open source licenses conditions.
 
-This version of the AboutCode Toolkit follows the ABOUT specification version 3.2.3 at:
+This version of the AboutCode Toolkit follows the ABOUT specification version 3.3.0 at:
 https://aboutcode-toolkit.readthedocs.io/en/latest/specification.html
 
 
@@ -38,7 +38,7 @@ Build and tests status
 
 REQUIREMENTS
 ------------
-The AboutCode Toolkit is tested with Python 3.6.2 or above only on Linux, Mac and Windows.
+The AboutCode Toolkit is tested with Python 3.7 or above only on Linux, Mac and Windows.
 You will need to install a Python interpreter if you do not have one already
 installed.
 
@@ -69,13 +69,6 @@ To install all the needed dependencies in a virtualenv, run (on posix):
 or on windows:
     configure
 
-.. note::
-   For MacOS users, it's a known issue the Python36 may case SSL Certificates error
-   if the Certificates is not up to date.
-
-   A solution is to run: `sudo /Applications/Python\\ 3.6/Install\\ Certificates.command`
-   to upgrade the outdated certificates.
-
 ACTIVATE the VIRTUALENV
 -----------------------
 To activate the virtualenv, run (on posix):

docs/source/specification.rst

@@ -1,7 +1,7 @@
 .. _specification:
 
 ===============================
-ABOUT File Specification v3.2.3
+ABOUT File Specification v3.3.0
 ===============================
 
 Purpose
@@ -57,8 +57,7 @@ The meaning of this ABOUT file is:
 Specification
 =============
 
-An ABOUT file is an ASCII YAML formatted text file. Note that while Unicode characters
-are not supported in an ABOUT file proper, external files can contain UTF-8 Unicode.
+An ABOUT file is an YAML formatted text file.
 The key for the licenses field and the license_expression are dejacode license key.
 
 ABOUT file name
@@ -67,11 +66,9 @@ ABOUT file name
 An ABOUT file name can use a limited set of characters and is suffixed with a
 ".ABOUT" extension using any combination of uppercase and lowercase characters.
 
-A file name can contain only these US-ASCII characters:
+A file name can contain any characters and digits with the following exception and condition:
 
--   digits from 0 to 9
--   uppercase and lowercase letters from A to Z
--   the following symbols: ``"_", "-", "+", ".", "(", ")", "~", "[", "]", "{", "}", "@", "%"``
+-   the following symbols are not supported: ``", #, &, ', *, \, :, ;, <, >, =, ?, /, ^, `, |``
 -   The case of a file name is not significant. On case-sensitive file systems
     (such as on Linux), a tool must report an error if two ABOUT files stored in the same
     directory have the same lowercase file name. This is to ensure that ABOUT files can be
@@ -81,7 +78,7 @@ A file name can contain only these US-ASCII characters:
 Lines of text
 -------------
 
-An ABOUT file contains lines of US-ASCII text. Lines contain field names/values pairs.
+An ABOUT file contains lines of text. Lines contain field names/values pairs.
 The standard line ending is the LF character. The line ending characters can be any LF,
 CR or CR/LF and tools must normalize line endings to LF when processing an ABOUT file.
 Empty lines and lines containing only white spaces that are not part of a field value
@@ -109,7 +106,7 @@ The field value is separated from the field name by a ":" colon. The ":" colon
 can be followed by one or more spaces that must be ignored. This also applies
 to trailing white spaces: they must be ignored.
 
-The field value is composed of one or more lines of plain US-ASCII printable text.
+The field value is composed of one or more lines of plain printable text.
 
 When a field value is a long string, additional continuation lines must start with
 at least one space. In this case, the first space of an additional continuation
@@ -169,9 +166,7 @@ for long texts or to reference a common text in multiple ABOUT files such as a
 common license text. In this case the field name is suffixed with "_file" and the
 field value must be a path pointing to the file that contains the actual value of the
 field. This path must be a POSIX path relative to the path of the ABOUT file. The file
-content must be UTF-8-encoded text. This is in contrast with field values contained
-directly in an ABOUT file that must be US-ASCII- encoded text and allows to support
-non-ASCII text content.
+content must be UTF-8-encoded text.
 
 For example, the full license text for a component is often stored in a separate file named COPYING:
 

etc/scripts/gen_requirements.py

@@ -34,7 +34,7 @@ def gen_requirements():
         type=pathlib.Path,
         required=True,
         metavar="DIR",
-        help="Path to the 'site-packages' directory where wheels are installed such as lib/python3.6/site-packages",
+        help="Path to the 'site-packages' directory where wheels are installed such as lib/python3.7/site-packages",
     )
     parser.add_argument(
         "-r",

etc/scripts/gen_requirements_dev.py

@@ -36,7 +36,7 @@ def gen_dev_requirements():
         type=pathlib.Path,
         required=True,
         metavar="DIR",
-        help='Path to the "site-packages" directory where wheels are installed such as lib/python3.6/site-packages',
+        help='Path to the "site-packages" directory where wheels are installed such as lib/python3.7/site-packages',
     )
     parser.add_argument(
         "-d",