Merge pull request #783 from nexB/712-695-large-data-file

License detection improvements

Author: pombredanne

src/licensedcode/__init__.py

@@ -51,6 +51,8 @@
 MIN_MATCH_LENGTH = 4
 MIN_MATCH_HIGH_LENGTH = 3
 
+# FIXME: we should consider the length of two rules and two matches when considering MAX_DIST
+# eventually this should be skipped early right during the matching too
 # maximum distance between two matches to merge
 MAX_DIST = 120
 

src/licensedcode/data/licenses/adapt-1.0.LICENSE

@@ -243,30 +243,30 @@ EXHIBIT A (to the Adaptive Public License)
 
 PART 1: INITIAL CONTRIBUTOR AND DESIGNATED WEB SITE
 
-The Initial Contributor is:	____________________________________________________
+The Initial Contributor is:	 
 
 [Enter full name of Initial Contributor]
 
-Address of Initial Contributor:	________________________________________________
- 	________________________________________________
- 	________________________________________________
+Address of Initial Contributor:	 
+ 	 
+ 	 
 
 [Enter address above]
 
-The Designated Web Site is:	__________________________________________________
+The Designated Web Site is:	 
 
 [Enter URL for Designated Web Site of Initial Contributor]
 NOTE: The Initial Contributor is to complete this Part 1, along with Parts 2, 3, and 5, and, if applicable, Parts 4 and 6.
 
 PART 2: INITIAL WORK
 
-The Initial Work comprises the computer program(s) distributed by the Initial Contributor having the following title(s): _______________________________________________.
+The Initial Work comprises the computer program(s) distributed by the Initial Contributor having the following title(s):  .
 
-The date on which the Initial Work was first available under this License: _________________
+The date on which the Initial Work was first available under this License:  
 
 PART 3: GOVERNING JURISDICTION
 
-For the purposes of this License, the Governing Jurisdiction is _________________________________________________. 
+For the purposes of this License, the Governing Jurisdiction is  . 
 [Initial Contributor to Enter Governing Jurisdiction here]
 
 PART 4: THIRD PARTIES
@@ -288,7 +288,7 @@ The default definition of "THIRD PARTY" is the definition set forth in paragraph
 
 PART 5: NOTICE
 
-THE LICENSED WORK IS PROVIDED UNDER THE TERMS OF THE ADAPTIVE PUBLIC LICENSE ("LICENSE") AS FIRST COMPLETED BY: ______________________ [Insert the name of the Initial Contributor here]. ANY USE, PUBLIC DISPLAY, PUBLIC PERFORMANCE, REPRODUCTION OR DISTRIBUTION OF, OR PREPARATION OF DERIVATIVE WORKS BASED ON, THE LICENSED WORK CONSTITUTES RECIPIENT'S ACCEPTANCE OF THIS LICENSE AND ITS TERMS, WHETHER OR NOT SUCH RECIPIENT READS THE TERMS OF THE LICENSE. "LICENSED WORK" AND "RECIPIENT" ARE DEFINED IN THE LICENSE. A COPY OF THE LICENSE IS LOCATED IN THE TEXT FILE ENTITLED "LICENSE.TXT" ACCOMPANYING THE CONTENTS OF THIS FILE. IF A COPY OF THE LICENSE DOES NOT ACCOMPANY THIS FILE, A COPY OF THE LICENSE MAY ALSO BE OBTAINED AT THE FOLLOWING WEB SITE: ___________________________________________________ [Insert Initial Contributor's Designated Web Site here]
+THE LICENSED WORK IS PROVIDED UNDER THE TERMS OF THE ADAPTIVE PUBLIC LICENSE ("LICENSE") AS FIRST COMPLETED BY:   [Insert the name of the Initial Contributor here]. ANY USE, PUBLIC DISPLAY, PUBLIC PERFORMANCE, REPRODUCTION OR DISTRIBUTION OF, OR PREPARATION OF DERIVATIVE WORKS BASED ON, THE LICENSED WORK CONSTITUTES RECIPIENT'S ACCEPTANCE OF THIS LICENSE AND ITS TERMS, WHETHER OR NOT SUCH RECIPIENT READS THE TERMS OF THE LICENSE. "LICENSED WORK" AND "RECIPIENT" ARE DEFINED IN THE LICENSE. A COPY OF THE LICENSE IS LOCATED IN THE TEXT FILE ENTITLED "LICENSE.TXT" ACCOMPANYING THE CONTENTS OF THIS FILE. IF A COPY OF THE LICENSE DOES NOT ACCOMPANY THIS FILE, A COPY OF THE LICENSE MAY ALSO BE OBTAINED AT THE FOLLOWING WEB SITE:   [Insert Initial Contributor's Designated Web Site here]
 
 Software distributed under the License is distributed on an "AS IS" basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License for the specific language governing rights and limitations under the License.
 

src/licensedcode/data/licenses/adobe-eula.yml

@@ -5,3 +5,4 @@ category: Proprietary
 owner: Adobe Systems
 homepage_url: http://www.adobe.com/products/eulas/
 notes: This is a generic license for a large number of Adobe EULAs that are very long and proprietary.
+minimum_coverage: 70

src/licensedcode/data/licenses/autoit-eula.LICENSE

@@ -3,7 +3,7 @@ AutoIt
 Author : Jonathan Bennett and the AutoIt Team
 WWW : https://www.autoitscript.com/site/autoit/
 Email : support at autoitscript dot com
-________________________________________________________
+ 
 END-USER LICENSE AGREEMENT FOR THIS SOFTWARE
 This End-User License Agreement ("EULA") is a legal agreement between you (either an individual or a single entity) and the mentioned author of this Software for the software product identified above, which includes computer software and may include associated media, printed materials, and "online" or electronic documentation ("SOFTWARE PRODUCT"). By installing, copying, or otherwise using the SOFTWARE PRODUCT, you agree to be bound by the terms of this EULA. If you do not agree to the terms of this EULA, do not install or use the SOFTWARE PRODUCT.
 

src/licensedcode/data/licenses/broadcom-linking-unmodified.LICENSE

@@ -0,0 +1,10 @@
+As a special exception, the copyright holders of this software give you
+permission to link this software with independent modules, and to copy
+and distribute the resulting executable under terms of your choice,
+provided that you also meet, for each linked independent module, the
+terms and conditions of the license of that module.
+
+An independent module is a module which is not derived from this
+software.  The special exception does not apply to any modifications of
+the software.
+

src/licensedcode/data/licenses/broadcom-linking-unmodified.yml

@@ -0,0 +1,6 @@
+key: broadcom-linking-unmodified
+short_name: Broadcom Linking Exception if unmodified
+name: Broadcom Linking Exception if unmodified
+category: Copyleft Limited
+owner: Broadcom
+notes: The exception does not apply when modified

src/licensedcode/data/licenses/bsl-1.0.LICENSE

@@ -12,7 +12,7 @@ Change License:  Version 2 or later of the GNU General Public License as publish
 
 For information about  alternative licensing arrangements for the Software, please visit:  https://mariadb.com/products/mariadb-enterprise
 
-_____________________________________________
+ 
 
 You are granted limited license to the Software under this Business Source License.  Please read this Business Source License carefully, particularly the Use Limitation set forth above.  
 

src/licensedcode/data/licenses/cnri-jython.LICENSE

@@ -5,7 +5,7 @@
 
 3. In the event Licensee prepares a derivative work that is based on or incorporates the Software or any part thereof, and wants to make the derivative work available to the public as provided herein, then Licensee hereby agrees to indicate in any such work, in a prominently visible way, the nature of the modifications made to CNRI's Software.
	
 
-4. Licensee may not use CNRI trademarks or trade name, including JPython or CNRI, in a trademark sense to endorse or promote products or services of Licensee, or any third party. Licensee may use the mark JPython in connection with Licensee's derivative versions that are based on or incorporate the Software, but only in the form “JPython-based ___________________,” or equivalent.
 
+4. Licensee may not use CNRI trademarks or trade name, including JPython or CNRI, in a trademark sense to endorse or promote products or services of Licensee, or any third party. Licensee may use the mark JPython in connection with Licensee's derivative versions that are based on or incorporate the Software, but only in the form “JPython-based  ,” or equivalent.
 
 
 5. CNRI is making the Software available to Licensee on an “AS IS” basis. CNRI MAKES NO REPRESENTATIONS OR WARRANTIES, EXPRESS OR IMPLIED. BY WAY OF EXAMPLE, BUT NOT LIMITATION, CNRI MAKES NO AND DISCLAIMS ANY REPRESENTATION OR WARRANTY OF MERCHANTABILITY OR FITNESS FOR ANY PARTICULAR PURPOSE OR THAT THE USE OF THE SOFTWARE WILL NOT INFRINGE ANY THIRD PARTY RIGHTS.
 
 

src/licensedcode/data/licenses/cpal-1.0.LICENSE

@@ -90,20 +90,20 @@ The term “External Deployment” means the use, distribution, or communication
 
 
 EXHIBIT A. Common Public Attribution License Version 1.0.
-“The contents of this file are subject to the Common Public Attribution License Version 1.0 (the “License”); you may not use this file except in compliance with the License. You may obtain a copy of the License at _____________. The License is based on the Mozilla Public License Version 1.1 but Sections 14 and 15 have been added to cover use of software over a computer network and provide for limited attribution for the Original Developer. In addition, Exhibit A has been modified to be consistent with Exhibit B.
+“The contents of this file are subject to the Common Public Attribution License Version 1.0 (the “License”); you may not use this file except in compliance with the License. You may obtain a copy of the License at  . The License is based on the Mozilla Public License Version 1.1 but Sections 14 and 15 have been added to cover use of software over a computer network and provide for limited attribution for the Original Developer. In addition, Exhibit A has been modified to be consistent with Exhibit B.
 Software distributed under the License is distributed on an “AS IS” basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License for the specific language governing rights and limitations under the License.
-The Original Code is______________________.
-The Original Developer is not the Initial Developer and is __________. If left blank, the Original Developer is the Initial Developer.
-The Initial Developer of the Original Code is ____________. All portions of the code written by ___________ are Copyright (c) _____. All Rights Reserved.
-Contributor ______________________.
-Alternatively, the contents of this file may be used under the terms of the _____ license (the [___] License), in which case the provisions of [______] License are applicable instead of those above.
-If you wish to allow use of your version of this file only under the terms of the [____] License and not to allow others to use your version of this file under the CPAL, indicate your decision by deleting the provisions above and replace them with the notice and other provisions required by the [___] License. If you do not delete the provisions above, a recipient may use your version of this file under either the CPAL or the [___] License.”
+The Original Code is .
+The Original Developer is not the Initial Developer and is  . If left blank, the Original Developer is the Initial Developer.
+The Initial Developer of the Original Code is  . All portions of the code written by   are Copyright (c)  . All Rights Reserved.
+Contributor  .
+Alternatively, the contents of this file may be used under the terms of the   license (the [   ] License), in which case the provisions of [ ] License are applicable instead of those above.
+If you wish to allow use of your version of this file only under the terms of the [ ] License and not to allow others to use your version of this file under the CPAL, indicate your decision by deleting the provisions above and replace them with the notice and other provisions required by the [   ] License. If you do not delete the provisions above, a recipient may use your version of this file under either the CPAL or the [   ] License.”
 [NOTE: The text of this Exhibit A may differ slightly from the text of the notices in the Source Code files of the Original Code. You should use the text of this Exhibit A rather than the text found in the Original Code Source Code for Your Modifications.]
 
 
 EXHIBIT B. Attribution Information
-Attribution Copyright Notice: _______________________
-Attribution Phrase (not exceeding 10 words): _______________________
-Attribution URL: _______________________
+Attribution Copyright Notice:  
+Attribution Phrase (not exceeding 10 words):  
+Attribution URL:  
 Graphic Image as provided in the Covered Code, if any.
 Display of Attribution Information is [required/not required] in Larger Works which are defined in the CPAL as a work which combines Covered Code or portions thereof with code not governed by the terms of the CPAL.

src/licensedcode/data/licenses/cua-opl-1.0.LICENSE

@@ -443,24 +443,24 @@ basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See the
 License for the specific language governing rights and limitations
 under the License.
 
-The Original Code is ______________________________________.
+The Original Code is  .
 
-The Initial Developer of the Original Code is ________________________.
-Portions created by ______________________ are Copyright (C) ______
-_______________________. All Rights Reserved.
+The Initial Developer of the Original Code is  .
+Portions created by   are Copyright (C)  
+ . All Rights Reserved.
 
-Contributor(s): ______________________________________.
+Contributor(s):  .
 
 Alternatively, the contents of this file may be used under the terms
-of the _____ license (the "[___] License"), in which case the
-provisions of [______] License are applicable instead of those
+of the   license (the "[   ] License"), in which case the
+provisions of [ ] License are applicable instead of those
 above. If you wish to allow use of your version of this file only
-under the terms of the [____] License and not to allow others to use
+under the terms of the [ ] License and not to allow others to use
 your version of this file under the CUAPL, indicate your decision by
 deleting the provisions above and replace them with the notice and
-other provisions required by the [___] License. If you do not delete
+other provisions required by the [   ] License. If you do not delete
 the provisions above, a recipient may use your version of this file
-under either the CUAPL or the [___] License."
+under either the CUAPL or the [   ] License."
 
 [NOTE: The text of this Exhibit A may differ slightly from the text of
 the notices in the Source Code files of the Original Code. You should