docs: usage: mnist: Complete MNIST tutorial

- Added png configloader to reading .mnistpng files
  - Using the .mnistpng extension for now because
    we're making the images into 28x28 flat arrays,
    which is specific to the mnist example.
- Added images to examples/MNIST for predicting
- Documentation for MNIST prediction
- CSVSource can now replace data in columns if that
  data is a filename with the data in that file loaded by
  the configloader given by the files extention.

Signed-off-by: John Andersen <[email protected]>

Author: sakshamarora1

.gitignore

@@ -24,3 +24,4 @@ docs/plugins/service/http
 *pip-wheel-metadata/
 *.gz
 doctest/
+*.mnistpng

CHANGELOG.md

@@ -6,6 +6,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [Unreleased]
 ### Added
+- PNG ConfigLoader for reading images as arrays to predict using MNIST trained models
 - Docstrings and doctestable examples to `record.py`.
 - Inputs can be validated using operations
   - `validate` parameter in `Input` takes `Operation.instance_name`

configloader/png/.coveragerc

@@ -0,0 +1,13 @@
+[run]
+source =
+    dffml_config_png
+    tests
+branch = True
+
+[report]
+exclude_lines =
+    no cov
+    no qa
+    noqa
+    pragma: no cover
+    if __name__ == .__main__.:

configloader/png/.gitignore

@@ -0,0 +1,20 @@
+*.log
+*.pyc
+.cache/
+.coverage
+.idea/
+.vscode/
+*.egg-info/
+build/
+dist/
+docs/build/
+venv/
+wheelhouse/
+*.egss
+.mypy_cache/
+*.swp
+.venv/
+.eggs/
+*.modeldir
+*.db
+htmlcov/

configloader/png/LICENSE

@@ -0,0 +1,21 @@
+Copyright (c) 2019 Intel, Saksham
+
+MIT License
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

configloader/png/MANIFEST.in

@@ -0,0 +1,3 @@
+include README.md
+include LICENSE
+include setup_common.py

configloader/png/README.md

@@ -0,0 +1,10 @@
+# DFFML Config for png
+
+## About
+
+Allows for reading of 'PNG' images for [MNIST dataset](../../docs/usage/mnist.rst)
+
+## License
+
+png Models are distributed under the terms of the
+[MIT License](LICENSE).

configloader/png/dffml_config_png/__init__.py

@@ -0,0 +1,40 @@
+"""
+Description of what this config does
+"""
+from typing import Dict
+from io import BytesIO
+from PIL import Image
+import numpy as np
+
+from dffml.util.entrypoint import entrypoint
+from dffml.util.cli.arg import Arg
+from dffml.base import BaseConfig
+from dffml.configloader.configloader import (
+    BaseConfigLoaderContext,
+    BaseConfigLoader,
+)
+
+
+class PNGConfigLoaderContext(BaseConfigLoaderContext):
+    async def loadb(self, resource: bytes) -> list:
+        image_array = np.array(Image.open(BytesIO(resource)).convert("L"))
+        img_pil = Image.fromarray(image_array)
+        img_28x28 = np.array(img_pil.resize((28, 28), Image.ANTIALIAS))
+        img_784 = tuple(img_28x28.flatten())
+        return img_784
+
+    async def dumpb(self, resource: Dict) -> bytes:
+        raise NotImplementedError
+
+
+@entrypoint("mnistpng")
+class PNGConfigLoader(BaseConfigLoader):
+    CONTEXT = PNGConfigLoaderContext
+
+    @classmethod
+    def args(cls, args, *above) -> Dict[str, Arg]:
+        return args
+
+    @classmethod
+    def config(cls, config, *above) -> BaseConfig:
+        return BaseConfig()

configloader/png/dffml_config_png/configloader.py

@@ -0,0 +1 @@
+VERSION = "0.0.1"