Suggestions (#3)

* Rename python-aws-ssm to aws-ssm

Adding Python to a Python package is like a snake
eating itself.

* README suggestions

People may not get involved if it's not obvious how to run
tests.

* Fix retrieval of SSM keys that do not exist

Using the retrieved SSM parameters means excluding non-existant
SSM keys. So we need to start from the provided SSM key names.

* Add custom SSM client example

* Improve docs and typing

* Update package name in README

* Improve type to reflect nested dictionaries

* Update README with development instructions

* Update License and authors

* Rename the package :( and small fixes

Author: mpaddle

Makefile

@@ -1,4 +1,3 @@
-
 .PHONY: setup
 setup: ## Create a virtual environment for the project and installs all dependencies
 	poetry install
@@ -17,7 +16,7 @@ lint: ## Check the .py files with Mypy, Flake8 and Black
 
 .PHONY: tests
 tests: ## Run tests with pytest and create the coverage report
-	poetry run pytest --cov=./ --cov-report=xml
+	PYTHONPATH=${PYTHONPATH}:. poetry run pytest --cov=./ --cov-report=xml
 
 .SILENT: help
 help: ## Shows all available commands

README.md

@@ -12,17 +12,24 @@ Perfect use case for this package is when secure parameters for an application a
 [AWS Parameter Store](https://docs.aws.amazon.com/systems-manager/latest/userguide/systems-manager-parameter-store.html)
 using a path hierarchy. During application startup you can use this package to fetch them and use them in your application.
 
+## Warning
+
+The SSM service is rate-limited by default. We strongly suggest using
+retrieving SSM keys by path, e.g. via `ParameterStore.get_parameters_by_path()`.
+This requires grouping keys by a useful path but reduces the chance of
+your own services being rate-limited in turn.
+
 ## Install
 ```bash
-pip install python-aws-ssm
+pip install aws-ssm
 ```
 
 ## Examples
 
 #### Basic Usage
 
 ```python
-from python_aws_ssm.parameters import ParameterStore
+from aws_ssm.parameters import ParameterStore
 
 # Assuming you have the parameters in the following format:
 # my-service/dev/param-1  -> with value `a`
@@ -38,7 +45,7 @@ value = parameters.get("param-1")
 #### Recursive and nested options
 
 ```python
-from python_aws_ssm.parameters import ParameterStore
+from aws_ssm.parameters import ParameterStore
 
 # Assuming you have the parameters in the following format:
 # my-service/dev/param-1  -> with value `a`
@@ -56,7 +63,7 @@ dev_parameters = parameters.get("dev")
 #### Get parameters by name
 
 ```python
-from python_aws_ssm.parameters import ParameterStore
+from aws_ssm.parameters import ParameterStore
 
 # Assuming you have the parameters in the following format:
 # my-service/dev/param-1  -> with value `a`
@@ -70,3 +77,52 @@ parameters = parameter_store.get_parameters(
 dev_parameters = parameters.get("/common/dev/param-2")
 # value should be `b`
 ```
+
+#### With custom client
+
+```python
+from aws_ssm.parameters import ParameterStore
+import boto3
+
+# Initialise an SSM client to specify the source of the credentials.
+# e.g. locally a profile would be more likely; an AWS Lambda would most
+# likely not override the credentials source.
+ssm_client = boto3.Session(profile_name='dev').client('ssm')
+parameter_store = ParameterStore(ssm_client)
+
+parameters = parameter_store.get_parameters(["/service/path/"])
+```
+
+## Development
+
+If you are missing any features or have found a bug, please open a PR or a new Github issue.
+
+
+#### Setup
+This project uses Poetry to manage the dependencies and the virtual environment.
+Follow the instructions from Poetry website (https://poetry.eustace.io/docs/#installation) to configure your local environment.
+
+After completing the Poetry setup, the virtual environment can be created running:
+```shell
+make setup
+```
+
+#### Tests
+Tests are run by Pytest
+```shell
+make test
+```
+
+#### Code style
+- Mypy is used for type annotations (https://github.com/python/mypy)
+- Black formatter (https://github.com/psf/black) is used to keep the coding style consistent.
+- Isort (https://github.com/timothycrosley/isort) is used to sort the imports.
+To format the codebase just run:
+```shell
+make format
+```
+and to check it before pushing:
+```shell
+make lint
+```
+

pyproject.toml

@@ -2,9 +2,9 @@
 name = "python-aws-ssm"
 version = "0.1.0"
 description = "Python package that interfaces with AWS System Manager"
-authors = ["Luca Valentini <[email protected]>"]
+authors = ["Luca Valentini <[email protected]>", "Maarten Jacobs <[email protected]>"]
 # New attributes
-license = "Apache License 2.0"
+license = "Apache-2.0"
 readme = "README.md"
 homepage = "https://github.com/PaddleHQ/python-aws-ssm"
 repository = "https://github.com/PaddleHQ/python-aws-ssm"

python_aws_ssm/parameters.py

@@ -19,30 +19,41 @@ def get_parameters(self, ssm_key_names: List[str]) -> Dict[str, Optional[str]]:
         returned in the result dict.
         """
 
-        parameters = self.client.get_parameters(
+        retrieved_parameters = self.client.get_parameters(
             Names=ssm_key_names, WithDecryption=True
         ).get("Parameters")
-        return {
-            parameter.get("Name"): parameter.get("Value")
-            for parameter in parameters
-            if parameter.get("Name") in ssm_key_names
+
+        # Initialise the result so that missing keys have a None value.
+        filled_parameters: Dict[str, Optional[str]] = {
+            parameter_name: None for parameter_name in ssm_key_names
         }
 
+        # Merge the retrieved parameters in.
+        for retrieved in retrieved_parameters:
+            if retrieved.get("Name") in ssm_key_names:
+                filled_parameters[retrieved.get("Name")] = retrieved.get("Value")
+
+        return filled_parameters
+
     def get_parameters_by_path(
         self,
         ssm_base_path: str,
         with_decryption: bool = True,
         recursive: bool = False,
         nested: bool = False,
-    ) -> Dict[str, Optional[str]]:
+    ) -> Dict[str, Union[Dict, Optional[str]]]:
         """
         Retrieve all the keys under a certain path on SSM.
-        * Wnen recursive is set to False, SSM doesn't parameters under a nested path.
+        * When recursive is set to False, SSM doesn't return keys under a nested path.
             e.g.: /{ssm_base_path}/foo/bar will not return 'bar' nor '/foo/bar'.
         * When recursive and nested are set to True, a nested dictionary is returned.
             e.g.: /{ssm_base_path}/foo/bar will return {"foo": {"bar": "value"}}
         * When nested is set to False, the full subpath is returned as key.
             e.g.: /{ssm_base_path}/foo/bar will return {"foo/bar": "value"}}}
+
+        :return If nested=False, a dictionary of string to optional string value.
+         If nested=True, a dictionary of string to potentially nested dictionaries with
+         optional string values.
         """
 
         parameters = self.client.get_parameters_by_path(
@@ -55,21 +66,29 @@ def get_parameters_by_path(
         }
 
         return (
-            self._parse_parameters(parameters) if recursive and nested else parameters
+            # Non-nested is the default behaviour (hence `else parameters`).
+            self._parse_parameters(parameters)
+            if recursive and nested
+            else parameters
         )
 
     @staticmethod
     def _parse_parameters(
         parameters: Dict[str, Optional[str]]
-    ) -> Dict[Union[Dict, str], Optional[Union[Dict, str]]]:
-        parsed_dict: Dict[Union[Dict, str], Optional[Union[Dict, str]]] = {}
+    ) -> Dict[str, Union[Dict, Optional[str]]]:
+        parsed_dict: Dict[str, Union[Dict, Optional[str]]] = {}
         for key, value in parameters.items():
             nested_dict = ParameterStore._tree_dict(key.split("/"), value)
             parsed_dict = ParameterStore._deep_merge(parsed_dict, nested_dict)
         return parsed_dict
 
     @staticmethod
     def _tree_dict(key_list: List[Any], value: Optional[Any]) -> Dict[Any, Any]:
+        """
+        Build a nested dictionary path from a list of keys and a value.
+        For example:
+            _tree_dict(["foo", "bar", "koo"], 42) ==> {"foo": {"bar": {"koo": 42}}}
+        """
         tree_dict: Dict[Any, Any] = {key_list[-1]: value}
         for key in reversed(key_list[:-1]):
             tree_dict = {key: tree_dict}

tests/test_python_aws_ssm.py

@@ -30,7 +30,11 @@ def test_get_parameters_keys_are_mapped(self):
         )
 
         self.assertEqual(
-            {"foo_ssm_key_1": "foo_ssm_value_1", "foo_ssm_key_3": "foo_ssm_value_3"},
+            {
+                "foo_ssm_key_1": "foo_ssm_value_1",
+                "foo_ssm_key_2": None,
+                "foo_ssm_key_3": "foo_ssm_value_3",
+            },
             secrets,
         )