7 detect instance variables in constructor (#14)

* upgraded to python 3.8 to have get_source_segment, WIP of constructor's code parsing

* fixed documentation link to poetry, closes #13

* handle non/typed and tuple assignment (to improve)

* properly handle type aliasing

* handle composition relationships

* refactored inspection and AST parsing code, bump to version 0.5.0

* added code-coverage plugin for unit tests

* WIP added UmlAttribute staticity. Some attributes are detected twice

* skipped subscriptable assignment and fixed attribute naming

* updated documentation (inspection and AST parsing)

* code clean-up

* added test case with multiple assignments

* removed obsolete test case

* fixed unit tests + code refactoring

* added unit tests on astvisitors

* fixed variable name

* fixes after @jjherphan code review

* code review of @jjerphan and @doyou89 (thanks!)

Author: lucsorel

.gitignore

@@ -2,6 +2,7 @@
 __pycache__/
 .venv/
 *.egg-info/
-*.puml
 dist/
 .vscode/
+# data generated by pytest-cov
+.coverage

.python-version

@@ -1 +1 @@
-3.7.6
+3.8.6

LICENSE

@@ -1,6 +1,6 @@
 MIT License
 
-Copyright (c) 2020 Luc Sorel-Giffo
+Copyright (c) 2020-2021 Luc Sorel-Giffo
 
 Permission is hereby granted, free of charge, to any person obtaining a copy
 of this software and associated documentation files (the "Software"), to deal

README.md

@@ -9,44 +9,46 @@
   <h1>Python to PlantUML</h1>
 </div>
 
-Generate Plantuml diagrams to document your python code
+Generate PlantUML class diagrams to document your Python application.
 
 # How it works
 
-## Features
-
-From a given path corresponding to a folder containing python code, `py2puml` loads each file as a module and generate a class diagram with the [PlantUML](https://plantuml.com/en/class-diagram) using:
+`py2puml` produces a class diagram [PlantUML script](https://plantuml.com/en/class-diagram) representing classes properties (static and instance attributes) and their relations (composition and inheritance relationships).
 
-* inspection to detect the classes to document (see the [inspect](https://docs.python.org/3/library/inspect.html) module)
-* annotations (the python type hinting syntax) to detect the attributes and their types (see the [typing](https://docs.python.org/3/library/typing.html) module)
-* fields for classes derived from namedtuples
-* composition and inheritance relationships are drawn only between the domain classes (this is designed on purpose, for documentation sake)
+`py2puml` internally uses code [inspection](https://docs.python.org/3/library/inspect.html) (also called *reflexion* in other programming languages) and [abstract tree parsing](https://docs.python.org/3/library/ast.html) to retrieve relevant information.
 
-## Current limitations
+## Features
 
-* type hinting is optional when writing Python code and discarded when it is executed, as mentionned in the [typing official documentation](https://docs.python.org/3/library/typing.html). The quality of the diagram output by `py2puml` depends on the reliability with which the type annotations were written
+From a given path corresponding to a folder containing Python code, `py2puml` processes each file as a module and generates a [PlantUML script](https://plantuml.com/en/class-diagram) of its classe-like definitions using:
 
-> The Python runtime does not enforce function and variable type annotations. They can be used by third party tools such as type checkers, IDEs, linters, etc.
+* **[inspection](https://docs.python.org/3/library/inspect.html)** and [type annotations](https://docs.python.org/3/library/typing.html) to detect:
+  * static class attributes and [dataclass](https://docs.python.org/3/library/dataclasses.html) fields
+  * fields of [namedtuples](https://docs.python.org/3/library/collections.html#collections.namedtuple)
+  * members of [enumerations](https://docs.python.org/3/library/enum.html)
+  * composition and inheritance relationships (between your domain classes only, for documentation sake)
 
-* complex type hints with more than one level of genericity are not properly handled for the moment: `List[MyClass]` or `Dict[str, MyClass]` are handled properly, `Dict[str, List[MyClass]]` is not. If your domain classes (also called business objects or DTOs) have attributes with complex type hints, it may be a code smell indicating that you should write a class which would better represent the business logic. But I may improve this part of the library as well 😀
+* parsing [abstract syntax trees](https://docs.python.org/3/library/ast.html#ast.NodeVisitor) to detect the instance attributes defined in `__init__` constructors
 
-* `py2puml` outputs diagrams in PlantUML syntax, which can be saved in text files along your python code and versioned with them. To generate image files, use the PlantUML runtime or a docker image (see [think/plantuml](https://hub.docker.com/r/think/plantuml))
+`py2puml` outputs diagrams in PlantUML syntax, which can be:
+* versioned along your code with a unit-test ensuring its consistency (see the [test_py2puml.py's test_py2puml_model_on_py2uml_domain](tests/py2puml/test_py2puml.py) example)
+* generated and hosted along other code documentation (better option: generated documentation should not be versioned with the codebase)
 
-* `py2puml` uses features of python 3 (generators for example) and thus won't work with python 2 runtimes. It relies on native python modules and uses no 3rd-party library, except [pytest](https://docs.pytest.org/en/latest/) as a development dependency for running the unit-tests
+To generate image files, use the PlantUML runtime, a docker image of the runtime (see [think/plantuml](https://hub.docker.com/r/think/plantuml)) or of a server (see the CLI documentation below)
 
-If you like tools around PlantUML, you may also be interested in this [lucsorel/plantuml-file-loader](https://github.com/lucsorel/plantuml-file-loader) project: A webpack loader which converts PlantUML files into images during the webpack processing (useful to [include PlantUML diagrams in your slides](https://github.com/lucsorel/markdown-image-loader/blob/master/README.md#web-based-slideshows) with RevealJS or RemarkJS).
+If you like tools related with PlantUML, you may also be interested in this [lucsorel/plantuml-file-loader](https://github.com/lucsorel/plantuml-file-loader) project:
+a webpack loader which converts PlantUML files into images during the webpack processing (useful to [include PlantUML diagrams in your slides](https://github.com/lucsorel/markdown-image-loader/blob/master/README.md#web-based-slideshows) with RevealJS or RemarkJS).
 
 # Install
 
-Install from PyPI:
+Install from [PyPI](https://pypi.org/project/py2puml/):
 
 * with `pip`:
 
 ```sh
 pip install py2puml
 ```
 
-* with [poetry](https://pipenv.readthedocs.io/en/latest/):
+* with [poetry](https://python-poetry.org/docs/):
 
 ```sh
 poetry add py2puml
@@ -64,25 +66,27 @@ pipenv install py2puml
 
 Once `py2puml` is installed at the system level, an eponymous command is available in your environment shell.
 
-For example, to create the diagram of the classes used by `py2puml`, one can use:
+For example, to create the diagram of the classes used by `py2puml`, run:
+
 ```sh
 py2puml py2puml/domain py2puml.domain
 ```
 
-This will output the following PlantUML script:
+This outputs the following PlantUML script:
 
 ```plantuml
 @startuml
 class py2puml.domain.umlclass.UmlAttribute {
   name: str
   type: str
+  static: bool
 }
 class py2puml.domain.umlclass.UmlClass {
   attributes: List[UmlAttribute]
 }
 class py2puml.domain.umlitem.UmlItem {
   name: str
-  fqdn: str
+  fqn: str
 }
 class py2puml.domain.umlenum.Member {
   name: str
@@ -92,12 +96,12 @@ class py2puml.domain.umlenum.UmlEnum {
   members: List[Member]
 }
 enum py2puml.domain.umlrelation.RelType {
-  COMPOSITION: *
-  INHERITANCE: <|
+  COMPOSITION: * {static}
+  INHERITANCE: <| {static}
 }
 class py2puml.domain.umlrelation.UmlRelation {
-  source_fqdn: str
-  target_fqdn: str
+  source_fqn: str
+  target_fqn: str
   type: RelType
 }
 py2puml.domain.umlclass.UmlClass *-- py2puml.domain.umlclass.UmlAttribute
@@ -110,7 +114,7 @@ py2puml.domain.umlrelation.UmlRelation *-- py2puml.domain.umlrelation.RelType
 
 Using PlantUML, this script renders this diagram:
 
-![py2puml UML Diagram](https://www.plantuml.com/plantuml/png/ZP91IyGm48Nl-HKvBsmF7iiUTbaA1jnMQZs9I7OxIY19Qp8H5jV_xZIse5GsFULrQBvvCozRZz9XC9gTjFIUz-URdhwojZDIsOnah6UFHkyGdJe61Fx9EBVIGCuzEj9uxaVzbSRi1n4HSWBwdDyfZq-_cpnVOIa4Cw04dJCph--jJPa16qns07C4Dxl_8NM0HG1oKD0P2IR2fa5-qCC8mu__t7UW9QhEPZNeXhON6VlgS5yzY4PKPSvNL13bRL6BPbVkYvnlBdC_SnvvgaSTcRuBxWGlSIbJMjAz0SRItm17BzGc6TzglLxqL5WYlCs5GAbkBB5_CdCzuoKk4Y6pPJkFNj9niotObkhi6m00)
+![py2puml UML Diagram](http://www.plantuml.com/plantuml/png/ZPB1Qy8m5CRl-IlUMR277Oi7HOGLfXrTTnfZfFLj59AqIru7elxlUuqQ1ewcftmcvlTzUL-NZgIbNYjHA-aST8U7Zdyb-rRBnYGi_NxogjMAo3PLJmX70M2anXGSMTPqw89c7ZLr2bNRAd6EKzU3y4HvuxiKdXf7RtyztqTO3Q4UK1clTza-lusN8_VHz3hPegxGtbt_aQh7IG0EiE7L4xI7tTvnGGyl6FxuptsBYeVMcgH0LV8iFMETRv_pbwpCybqACpXU1dlcasptk2coShLRRr9OdC9HI3ZYm2cBg_OkhkrjZHzXIW0axHTIs0drNhEnIRJDsNm-wKCIaIuN9mR5t4Ia3muptlca5ECcOjh4VPPu_MA9Pi_xlm00)
 
 For a full overview of the CLI, run:
 
@@ -128,14 +132,13 @@ Pipe the result of the CLI with a PlantUML server for instantaneous documentatio
 
 ```sh
 # runs a local PlantUML server from a docker container:
-docker run -d -p 1234:8080 --name plantumlserver plantuml/plantuml-server:jetty 
+docker run -d --rm -p 1234:8080 --name plantumlserver plantuml/plantuml-server:jetty 
 
 py2puml py2puml/domain py2puml.domain | curl -X POST --data-binary @- http://localhost:1234/svg/ --output - | display
 
-# stops the container when you don't need it anymore, restarts it later, removes it
+# stops the container when you don't need it anymore, restarts it later
 docker stop plantumlserver
 docker start plantumlserver
-docker rm plantumlserver
 ```
 
 ## Python API
@@ -154,6 +157,7 @@ print(''.join(py2puml('py2puml/domain', 'py2puml.domain')))
 with open('py2puml/domain.puml', 'w') as puml_file:
     puml_file.writelines(py2puml('py2puml/domain', 'py2puml.domain'))
 ```
+
 * running it (`python3 -m py2puml.example`) will output the previous PlantUML diagram in the terminal and write it in a file.
 
 
@@ -167,13 +171,20 @@ poetry run python -m pytest -v
 python3 -m pytest -v
 ```
 
+Code coverage (with missed [branch statements](https://pytest-cov.readthedocs.io/en/latest/config.html?highlight=--cov-branch)):
+
+```sh
+poetry run python -m pytest -v --cov=py2puml --cov-branch --cov-report term-missing --cov-fail-under 90
+```
+
 # Changelog
 
+* `0.5.0`: handle instance attributes in class constructors, add code coverage of unit tests
 * `0.4.0`: add a simple CLI
 * `0.3.1`: inspect sub-folders recursively
-* `0.3.0`: handle classes derived from namedtuples (attribute types are `any`)
-* `0.2.0`: handle inheritance relationships and enums. Unit tested
-* `0.1.3`: first release, handle all module of a folder and compositions of domain classes
+* `0.3.0`: handle classes derived from namedtuples (attribute types are `Any`)
+* `0.2.0`: handle inheritance relationships and enums
+* `0.1.3`: first release, handle all modules of a folder and compositions of domain classes
 
 # Licence
 
@@ -187,6 +198,24 @@ Unless stated otherwise all works are licensed under the [MIT license](http://sp
 
 Pull-requests are welcome and will be processed on a best-effort basis.
 
+# Current limitations
+
+* regarding **inspection**
+
+  * type hinting is optional when writing Python code and discarded when it is executed, as mentionned in the [typing official documentation](https://docs.python.org/3/library/typing.html). The quality of the diagram output by `py2puml` depends on the reliability with which the type annotations were written
+
+  > The Python runtime does not enforce function and variable type annotations. They can be used by third party tools such as type checkers, IDEs, linters, etc.
+
+  * complex type hints with more than one level of genericity are not properly handled for the moment: `List[MyClass]` or `Dict[str, MyClass]` are handled properly, `Dict[str, List[MyClass]]` is not.
+  If your domain classes (also called business objects or DTOs) have attributes with complex type hints, it may be a code smell indicating that you should write a class which would better represent the business logic.
+  But I may improve this part of the library as well 😀
+
+* regarding the detection of instance attributes with **AST parsing**:
+  * only constructors are visited, attributes assigned in other functions won't be documented
+  * attribute types are inferred from type annotations:
+    * of the attribute itself
+    * of the variable assigned to the attribute: a signature parameter or a locale variable
+    * to avoid side-effects, no code is executed nor interpreted
 
 # Alternatives