Merge pull request #22 from nightfallai/add-documentation

add documentation

Author: Dan Hertz

CODEOWNERS

@@ -0,0 +1,3 @@
+# These owners will be the default owners for everything in
+# the repo.
+* @evanfuller @austinphilp @smkent @dhertz

CONTRIBUTING.md

@@ -0,0 +1,60 @@
+# Contributing
+
+Thank you for considering contributing to the Nightfall Python SDK.
+
+There are many ways to contribute, such as:
+* Writing code samples
+* Suggesting documentation improvements
+* Submitting bug reports and feature requests
+* Writing code to improve the library itself.
+
+Please, don't use the issue tracker for personal support questions. Feel free to reach out to `[email protected]`
+to address those issues.
+
+## Responsibilities
+* Ensure cross-platform compatibility for every change that's accepted. Windows, Mac, Debian & Ubuntu Linux, etc.
+* Ensure backwards compatibility with Python 3.7+.
+* Create issues for each major change and enhancement that you wish to make.
+* Discuss proposed changes transparently and collect community feedback.
+* Avoid introducing new external dependencies whenever possible. When absolutely required, validate the software
+licenses used by these dependencies (e.g. avoid unintentional copyleft requirements).
+
+## How to report a bug
+
+### Security Disclosures
+If you find a security vulnerability, do NOT open an issue. Email `[email protected]` instead.
+
+In order to determine whether you are dealing with a security issue, ask yourself the following questions:
+* Can I access something that's not mine, or something I shouldn't have access to?
+* Can I disable something for other people?
+* Is there a potential vulnerability stemming from a library dependency?
+
+If you answered yes to any of the above questions, then you're probably dealing with a security issue.
+Note that even if you answer "no" to all questions, you may still be dealing with a security issue, so if you're
+unsure, just email us at `[email protected]`.
+
+### Creating an Issue
+When filing an issue, make sure to answer these questions:
+1. What version of Java are you using?
+2. What operating system and processor architecture are you using?
+3. How did you discover the issue?
+4. Is the issue reproducible? What are the steps to reproduce?
+5. What did you expect to see?
+6. What did you see instead?
+
+
+## Suggesting a New Feature
+
+If you find yourself wishing for a feature that doesn't exist in this SDK, you are probably not alone.
+There are bound to be others out there with similar needs. Open an issue on our issues list on GitHub which
+describes the feature you would like to see, why you need it, and how it should work.
+
+## Code Review
+
+The core team looks at open pull requests on a regular basis. In order for your pull request to be merged, it
+must meet the following requirements:
+* It must pass the checkstyle linter; this should be run automatically when you run `mvn package`.
+* It must add unit tests to cover any new functionality.
+* It must get approval from one of the code owners.
+
+If a pull request remains idle for more than two weeks, we may close it.

README.md

@@ -1,44 +1,122 @@
 # Nightfall Python SDK
 
-This is a python SDK for working with the Nightfall API.
+**Embed Nightfall scanning and detection functionality into Python applications**
 
 [![PyPI version](https://badge.fury.io/py/nightfall.svg)](https://badge.fury.io/py/nightfall)
 
+##  Features
 
-## Installation 
+This SDK provides Python functions for interacting with the Nightfall API. It allows you to add functionality to your
+applications to scan plain text and files in order to detect different categories of information. You can leverage any
+of the detectors in Nightfall's pre-built library, or you may programmatically define your own custom detectors.
 
-This module requires Python 3.7 or higher.
+Additionally, this library provides convenience features such as encapsulating the steps to chunk and upload files.
+
+To obtain an API Key, login to the [Nightfall dashboard](https://app.nightfall.ai/) and click the section
+titled "Manage API Keys".
+
+See our [developer documentation](https://docs.nightfall.ai/docs/entities-and-terms-to-know) for more details about
+integrating with the Nightfall API.
+
+## Dependencies
+
+The Nightfall Python SDK requires Python 3.7 or later.
+
+For a full list of external dependencies please consult `setup.py`.
+
+
+## Installation
 
 ```
 pip install nightfall
 ```
 
-## Quickstart 
+## Usage
 
-Make a new [API Token](https://app.nightfall.ai/api/) in Nightfall and store the value as an environment variable.
 
-```python
-import os
+### Scanning Plain Text
 
+Nightfall provides pre-built detector types, covering data types ranging from PII to PHI to credentials. The following
+snippet shows an example of how to scan using pre-built detectors.
+
+####  Sample Code
+
+```python
 from nightfall import Confidence, DetectionRule, Detector, Nightfall
 
-nightfall = Nightfall(os.getenv('NIGHTFALL_API_KEY'))
+# By default, the client reads the API key from the environment variable NIGHTFALL_API_KEY
+nightfall = Nightfall()
+
+# A rule contains a set of detectors to scan with
+detection_rule = DetectionRule([
+    Detector(min_confidence=Confidence.LIKELY, nightfall_detector="CREDIT_CARD_NUMBER"),
+    Detector(min_confidence=Confidence.POSSIBLE, nightfall_detector="US_SOCIAL_SECURITY_NUMBER"),
+])
 
 findings, _ = nightfall.scan_text(
-        ["4916-6734-7572-5015 is my credit card number"],
-        [DetectionRule(
-            [Detector(min_confidence=Confidence.LIKELY,
-                     nightfall_detector="CREDIT_CARD_NUMBER")])])
+        ["hello world", "my SSN is 678-99-8212", "4242-4242-4242-4242"],
+        [detection_rule]
+)
+
 print(findings)
 ```
+### Scanning Files
+
+Scanning common file types like PDF's or office documents typically requires cumbersome text
+extraction methods like OCR.
+
+Rather than implementing this functionality yourself, the Nightfall API allows you to upload the
+original files, and then we'll handle the heavy lifting.
+
+The file upload process is implemented as a series of requests to upload the file in chunks. The library
+provides a single method that wraps the steps required to upload your file. Please refer to the
+[API Reference](https://docs.nightfall.ai/reference) for more details.
+
+The file is uploaded synchronously, but as files can be arbitrarily large, the scan itself is conducted asynchronously.
+The results from the scan are delivered by webhook; for more information about setting up a webhook server, refer to
+[the docs](https://docs.nightfall.ai/docs/creating-a-webhook-server).
+
+#### Sample Code
+
+```python
+from nightfall import Confidence, DetectionRule, Detector, Nightfall
+
+# By default, the client reads the API key from the environment variable NIGHTFALL_API_KEY
+nightfall = Nightfall()
+
+# A rule contains a set of detectors to scan with
+detection_rule = DetectionRule([
+    Detector(min_confidence=Confidence.LIKELY, nightfall_detector="CREDIT_CARD_NUMBER"),
+    Detector(min_confidence=Confidence.POSSIBLE, nightfall_detector="US_SOCIAL_SECURITY_NUMBER"),
+])
+
+
+# Upload the file and start the scan.
+# These are conducted asynchronously, so provide a webhook route to an HTTPS server to send results to.
+id, message = nightfall.scan_file(
+    "./super-secret-credit-cards.pdf",
+    "https://my-service.com/nightfall/listener",
+    detection_rules=[detection_rule]
+)
+print("started scan", id, message)
+```
 
-For more information on the details of this library, please refer to 
-the [API Documentation](https://docs.nightfall.ai/).
 ## Contributing
 
-Please create an issue with a description of your problem, or open a pull request with the fix. 
+Contributions are welcome! Open a pull request to fix a bug, or open an issue to discuss a new feature
+or change. Please adhere to the linting criteria expected by flake8, and be sure to add unit tests for
+any new functionality you add.
+
+Refer to `CONTRIBUTING.md` for the full details.
+
+## License
 
-## Development 
+This code is licensed under the terms of the MIT License. See [here](https://opensource.org/licenses/MIT)
+for more information.
+
+Please create an issue with a description of your problem, or open a pull request with the fix.
+
+## Development
 
 ### Installing Development Dependencies
 
@@ -58,19 +136,15 @@ Unit and Integration tests can be found in the `tests/` directory. You can run t
 
 You can view the code coverage report by running `coverage html` and `python3 -m http.server --directory htmlcov` after running the unit tests.
 
-### Creating a Release 
+### Creating a Release
 
-Releases are automatically published to PyPI using GitHub Actions. Creating a release in GitHub will trigger a new build that will publish the latest version of this library to [PyPI](https://pypi.org/project/nightfall/). 
+Releases are automatically published to PyPI using GitHub Actions. Creating a release in GitHub will trigger a new build that will publish the latest version of this library to [PyPI](https://pypi.org/project/nightfall/).
 
-The steps to do this are: 
+The steps to do this are:
 
 1. Add what changed to the CHANGELOG file
 2. Update the version in `setup.py`
-3. Commit changes and push to the main branch. 
-4. Create a new release in the GitHub UI. 
-5. Observe the release action succeed and see the latest version of this library on PyPI. 
-## License 
-
-MIT
-
+3. Commit changes and push to the main branch.
+4. Create a new release in the GitHub UI.
+5. Observe the release action succeed and see the latest version of this library on PyPI.
 

nightfall/api.py

@@ -45,7 +45,7 @@ def __init__(self, key: Optional[str] = None, signing_secret: Optional[str] = No
 
         self._headers = {
             "Content-Type": "application/json",
-            "User-Agent": "nightfall-python-sdk/1.0.0",
+            "User-Agent": "nightfall-python-sdk/1.1.0",
             'Authorization': f'Bearer {self.key}',
         }
         self.signing_secret = signing_secret

nightfall/detection_rules.py

@@ -124,6 +124,7 @@ def as_dict(self):
             "charsToIgnore": self.chars_to_ignore
         }
 
+
 @dataclass
 class RedactionConfig:
     """An object that configures how any detected findings should be redacted when returned to the client. When this

tests/test_api.py

@@ -1,6 +1,4 @@
-import datetime
 import os
-import time
 
 from freezegun import freeze_time
 import pytest