Provide project documentation (#28)

xen0l · web-flow · commit 15dd3148903d · 2021-01-06T02:00:11.000+01:00
diff --git a/NOTES.md b/NOTES.md
@@ -7,7 +7,7 @@
 ## Security Hub finding updates
 
 Security Hub findings can be updated. However, there are some limitations on which attributes can be updated 
-and one should be aware of them. The list can be found [here](https://docs.aws.amazon.com/securityhub/latest/userguide/finding-update-batchupdatefindings.html#batchupdatefindings-fields)
+and one should be aware of them. The list can be found [here](https://docs.aws.amazon.com/securityhub/latest/userguide/finding-update-batchupdatefindings.html#batchupdatefindings-fields).
 
 ## Findings discovered while working with Security Hub
 
diff --git a/README.md b/README.md
@@ -3,4 +3,4 @@
 [![codecov](https://codecov.io/gh/xen0l/python-asff/branch/master/graph/badge.svg?token=GEFB001RIX)](https://codecov.io/gh/xen0l/python-asff)
 [![Documentation Status](https://readthedocs.org/projects/python-asff/badge/?version=latest)](https://python-asff.readthedocs.io/en/latest/?badge=latest)
 
-Python library to work with Amazon Security Finding Format (ASFF)
+Python library to work with Amazon Security Finding Format (ASFF)
diff --git a/docs/changelog.md b/docs/changelog.md
diff --git a/docs/conf.py b/docs/conf.py
@@ -12,11 +12,10 @@
 #
 import os
 import sys
+from recommonmark.transform import AutoStructify
 
 sys.path.insert(0, os.path.abspath("../"))
 
-import sphinx_rtd_theme
-
 
 # -- Project information -----------------------------------------------------
 
@@ -34,9 +33,10 @@
 # extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
 # ones.
 extensions = [
+    "sphinx.ext.inheritance_diagram",
+    "sphinx.ext.napoleon",
     "sphinx.ext.autodoc",
     "autoapi.extension",
-    "sphinx.ext.inheritance_diagram",
     "recommonmark",
 ]
 
@@ -76,15 +76,19 @@
     "show-inheritance",
 ]
 autoapi_member_order = "groupwise"
-# autoapi_add_toctree_entry = True
+
+# Uncomment, if you are working on documentation and want faster rebuilds.
+# autoapi_generate_api_docs = False
+# autoapi_add_toctree_entry = False
+# autoapi_keep_files = True
 
 html_title = "python-asff"
 
 html_theme_options = {
     # Set the name of the project to appear in the navigation.
     "nav_title": "python-asff",
     # Set you GA account ID to enable tracking
-    "google_analytics_account": "UA-XXXXX",
+    # "google_analytics_account": "UA-XXXXX",
     # Specify a base_url used to generate sitemap.xml. If not
     # specified, then no sitemap will be built.
     #'base_url': 'https://project.github.io/project',
@@ -109,3 +113,17 @@
 html_sidebars = {
     "**": ["logo-text.html", "globaltoc.html", "localtoc.html", "searchbox.html"]
 }
+
+
+def setup(app):
+    app.add_config_value(
+        "recommonmark_config",
+        {
+            "auto_toc_tree_section": "Contents",
+            "enable_math": False,
+            "enable_inline_math": False,
+            "enable_eval_rst": True,
+        },
+        True,
+    )
+    app.add_transform(AutoStructify)
diff --git a/docs/developer/changelog.md b/docs/developer/changelog.md
@@ -0,0 +1 @@
+../../CHANGELOG.md
diff --git a/docs/developer/contributing.md b/docs/developer/contributing.md
@@ -0,0 +1,25 @@
+# Contributing
+
+Contributions are welcome in the following categories:
+
+* integration into open source security tools
+* reporting bugs  
+* documentation
+* new functionality & code
+
+
+## Reporting bugs
+
+If you found a bug, you should report it in the project [issue tracker](https://github.com/xen0l/python-asff/issues). 
+
+## Documentation
+
+All documentation contributions should be written in Markdown. The project uses Sphinx as a documentation generator, 
+but this is because it provides the best support for generating documentation from docstrings. mkdocs alternatives did 
+not provide an output of a similar quality (new project idea?).
+
+## Pull requests 
+
+All code contributions have to be made as a pull request as CI checks will run automatically.
+Every new code added should be accompanied by a test as I would like to maintain 100% test coverage, so detecting regressions is simple 
+as well as adding new functionality.
diff --git a/docs/developer/index.rst b/docs/developer/index.rst
@@ -0,0 +1,12 @@
+Develper page
+=============
+
+.. toctree::
+    :maxdepth: 1
+
+    contributing
+    testing
+    maintenance
+    notes
+    changelog
+    license
diff --git a/docs/developer/license.md b/docs/developer/license.md
diff --git a/docs/developer/maintenance.md b/docs/developer/maintenance.md
@@ -0,0 +1,9 @@
+# Maintenance
+
+The repository contains some scripts in _tools_ directory, They are essential for project maintenance.
+This page documents them.
+
+* **class_header.template.py** - header template for code generated from ASFF schema 
+* **generate_class.py** - script for reading ASFF schema and generating classses in asff/generated.py
+* **download_securityhub_event_files.py** - helper script for downloading Security Hub event samples used for testing
+* **update_schema.py** - script to check for ASFF schema updates (used by Github Actions) 
diff --git a/docs/developer/notes.md b/docs/developer/notes.md
@@ -0,0 +1 @@
+../../NOTES.md
diff --git a/docs/developer/testing.md b/docs/developer/testing.md
@@ -0,0 +1,15 @@
+# Testing
+
+## Local testing
+
+When adding new or changing existing functionality, it is important to ensure that no regressions were introduced.
+python-asff comes with a test suite that checks functionality.
+
+Just run this from the root of the repository:
+
+```bash
+./tests/run.py
+```
+
+These tests will be run automatically during the Github Actions build. 
+However, if you want to see if it will pass the tests, you can just run that quick command on your machine.
diff --git a/docs/examples.md b/docs/examples.md
@@ -0,0 +1,19 @@
+# Examples
+
+Code examples of python-asff and how to use it. These snippets are usable "as is".
+
+## Create a new finding 
+
+```eval_rst
+.. literalinclude:: ../examples/new_finding.py
+    :language: python
+    :linenos:
+```
+
+## Send a finding to Security Hub
+
+```eval_rst
+.. literalinclude:: ../examples/send_finding_to_hub.py
+    :language: python
+    :linenos:
+```
diff --git a/docs/index.rst b/docs/index.rst
@@ -1,11 +1,18 @@
 Welcome to python-asff documentation!
 =====================================
 
+python-asff is a library to work with Amazon Security Finding Format (ASFF). It aims to provide simple interface
+for integrations, minimal number of external dependencies and 100% schema correctness.
+
+
 .. toctree::
     :maxdepth: 1
 
-    changelog
-    license
+    overview
+    examples
+    user-guide/index
+    developer/index
+    autoapi/index
 
 Index
 =====
diff --git a/docs/overview.md b/docs/overview.md
@@ -0,0 +1,28 @@
+# Overview
+
+## Motivation
+
+I run security operations in multi-account AWS organization setup. There are various AWS & open source security tools used for handling security events in AWS. 
+However, I found it very beneficial to centralize all findings in [AWS Security Hub](https://aws.amazon.com/security-hub/) as it provides complete overview of all
+security events in the AWS organization. Apart from that, it also normalizes all security events into a common format: [Amazon Security Finding Format (ASFF)](https://docs.aws.amazon.com/securityhub/latest/userguide/securityhub-findings-format.html).
+
+The benefit of this is that regardless of vendor or tool, all the findings share the same format. However, there are some downsides:
+
+* the format is quite extensive (which is DEFINITELY good for verbose information, but can cause headaches what and how much data should be provided)
+* there are no easy to use libraries with proper validation and simple interface
+* lack of tooling to build ASFF findings regardless of the programming language
+
+*python-asff* tries to rectify this.
+
+## Main features
+
+python-asff main features are:
+
+* **schema correctness** - once Python class has been created, it will be fully validated, so user can be certain that class is always 
+  suitable for ingestion into AWS Security Hub.
+* **simplicity** - the library aims to provide extremely simple interface, so people can ingest finding easily and focus on what matters - fixing them. 
+  Moreover, it also provides helpers that allow you to fetch information about resources easily.
+* **minimal dependencies** - for as wider adoption as possible, we commit to keep the number of external dependencies very small. 
+  Only dependency to stay is [pydantic](https://github.com/samuelcolvin/pydantic), which provides underlying functionality. 
+* **CLI tooling** - provide CLI tooling that will allow non-Python projects to create ASFF findings from any security tool suitable 
+  for ingestion into Security Hub easily.
diff --git a/docs/user-guide/index.rst b/docs/user-guide/index.rst
@@ -0,0 +1,7 @@
+User guide
+==========
+
+.. toctree::
+    :maxdepth: 1
+
+    installation
diff --git a/docs/user-guide/installation.md b/docs/user-guide/installation.md
@@ -0,0 +1,15 @@
+# Installation
+
+## Python support
+
+Required minimal version of Python is 3.6+ and project is actively tested on all supported by versions.
+Python 3 versions prior to 3.6 can also work with some changes, but the code uses language features from Python 3.6+. 
+
+## PyPI
+
+python-asff is available via PyPI:
+
+```bash
+pip install python-asff
+```
+
