model: spacy: ner: Add spacy models

Author: 0dust

.ci/deps.sh

@@ -172,6 +172,11 @@ if [ "x${PLUGIN}" == "xmodel/tensorflow_hub" ]; then
   python -m pip install -U -e "./model/tensorflow"
 fi
 
+if [[ "x${PLUGIN}" == "xmodel/spacy" ]]; then
+  conda install -y -c conda-forge spacy
+  python -m spacy download en_core_web_sm
+fi
+
 if [[ "x${PLUGIN}" == "xoperations/deploy" ]]; then
   python -m pip install -U -e "./feature/git"
 fi

.github/workflows/testing.yml

@@ -59,7 +59,7 @@ jobs:
       fail-fast: false
       max-parallel: 40
       matrix:
-        plugin: [., examples/shouldi, model/daal4py, model/tensorflow, model/tensorflow_hub, model/transformers, model/scratch, model/scikit, model/vowpalWabbit, model/autosklearn, operations/binsec, operations/deploy, operations/image, operations/nlp, source/mysql, feature/git, feature/auth, service/http, configloader/yaml, configloader/image]
+        plugin: [., examples/shouldi, model/daal4py, model/tensorflow, model/tensorflow_hub, model/transformers, model/scratch, model/scikit, model/vowpalWabbit, model/autosklearn, model/spacy, operations/binsec, operations/deploy, operations/image, operations/nlp, source/mysql, feature/git, feature/auth, service/http, configloader/yaml, configloader/image]
         python-version: [3.7, 3.8]
 
     steps:

CHANGELOG.md

@@ -6,6 +6,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [Unreleased]
 ### Added
+- Spacy model for NER
 - Added ability to rename outputs using GetSingle
 - Tutorial for using NLP operations with models
 - Operations plugin for NLP wrapping spacy and scikit functions

dffml/plugins.py

@@ -34,6 +34,7 @@ def inpath(binary):
     ("model", "transformers"),
     ("model", "vowpalWabbit"),
     ("model", "autosklearn"),
+    ("model", "spacy"),
 ]
 
 # Models which currently don't support Python 3.8

model/spacy/.coveragerc

@@ -0,0 +1,13 @@
+[run]
+source =
+    dffml_model_spacy
+    tests
+branch = True
+
+[report]
+exclude_lines =
+    no cov
+    no qa
+    noqa
+    pragma: no cover
+    if __name__ == .__main__.:

model/spacy/.gitignore

@@ -0,0 +1,20 @@
+*.log
+*.pyc
+.cache/
+.coverage
+.idea/
+.vscode/
+*.egg-info/
+build/
+dist/
+docs/build/
+venv/
+wheelhouse/
+*.egss
+.mypy_cache/
+*.swp
+.venv/
+.eggs/
+*.modeldir
+*.db
+htmlcov/

model/spacy/LICENSE

@@ -0,0 +1,21 @@
+Copyright (c) 2019 Intel
+
+MIT License
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

model/spacy/MANIFEST.in

@@ -0,0 +1,3 @@
+include README.md
+include LICENSE
+include setup_common.py

model/spacy/README.md

@@ -0,0 +1,12 @@
+# DFFML spacy Models
+
+## About
+
+dffml_model_spacy models.
+
+## Usage
+
+## License
+
+model_name Models are distributed under the terms of the
+[MIT License](LICENSE).