model: transformers: Add NER models

Signed-off-by: John Andersen <[email protected]>

Author: 0dust

.github/workflows/testing.yml

@@ -51,7 +51,7 @@ jobs:
       fail-fast: false
       max-parallel: 40
       matrix:
-        plugin: [., examples/shouldi, model/tensorflow, model/tensorflow_hub, model/scratch, model/scikit, source/mysql, feature/git, feature/auth, service/http, configloader/yaml]
+        plugin: [., examples/shouldi, model/tensorflow, model/tensorflow_hub, model/transformers, model/scratch, model/scikit, source/mysql, feature/git, feature/auth, service/http, configloader/yaml]
         python-version: [3.7]
 
     steps:

CHANGELOG.md

@@ -6,6 +6,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [Unreleased]
 ### Added
+- Hugging Face Transformers tensorflow based NER models.
 - PNG ConfigLoader for reading images as arrays to predict using MNIST trained models
 - Docstrings and doctestable examples to `record.py`.
 - Inputs can be validated using operations

dffml/service/dev.py

@@ -50,6 +50,7 @@
     ("model", "scratch"),
     ("model", "scikit"),
     ("model", "tensorflow_hub"),
+    ("model", "transformers"),
     ("examples", "shouldi"),
     ("feature", "git"),
     ("feature", "auth"),

model/transformers/.coveragerc

@@ -0,0 +1,13 @@
+[run]
+source =
+    dffml_model_transformer
+    tests
+branch = True
+
+[report]
+exclude_lines =
+    no cov
+    no qa
+    noqa
+    pragma: no cover
+    if __name__ == .__main__.:

model/transformers/.gitignore

@@ -0,0 +1,20 @@
+*.log
+*.pyc
+.cache/
+.coverage
+.idea/
+.vscode/
+*.egg-info/
+build/
+dist/
+docs/build/
+venv/
+wheelhouse/
+*.egss
+.mypy_cache/
+*.swp
+.venv/
+.eggs/
+*.modeldir
+*.db
+htmlcov/

model/transformers/LICENSE

@@ -0,0 +1,21 @@
+Copyright (c) 2019 Intel
+
+MIT License
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

model/transformers/MANIFEST.in

@@ -0,0 +1,3 @@
+include README.md
+include LICENSE
+include setup_common.py

model/transformers/README.md

@@ -0,0 +1,19 @@
+# DFFML Transformers Models
+
+
+## About
+
+Currently, dffml_model_transformers supports HuggingFace Transformers tensorflow based NER models. 
+
+## Documentation
+
+Documentation is hosted at https://intel.github.io/dffml/plugins/dffml_model.html#dffml-model-transformers
+
+## Acknowledgement
+
+dffml_model_transformers derives code from ![examples available on Transformers git repo.](https://github.com/huggingface/transformers/tree/master/examples), ![License](https://github.com/huggingface/transformers/blob/master/LICENSE)
+
+## License
+
+Transformers Models are distributed under the terms of the
+[MIT License](LICENSE).