Merge pull request #84 from CITCOM-project/tidying

Odd jobs to clean up package

Author: bobturneruk

README.md

@@ -22,28 +22,30 @@ The causal testing framework has three core components:
 For more information on each of these steps, follow the link to their respective documentation.
 
 ## Causal Inference Terminology
+
 Here are some explanations for the causal inference terminology used above.
 
 - Causal inference (CI) is a family of statistical techniques designed to quantify and establish **causal** relationships in data. In contrast to purely statistical techniques that are driven by associations in data, CI incorporates knowledge about the data-generating mechanisms behind relationships in data to derive causal conclusions.
 - One of the key advantages of CI is that it is possible to answer causal questions using **observational data**. That is, data which has been passively observed rather than collected from an experiment and, therefore, may contain all kinds of bias. In a testing context, we would like to leverage this advantage to test causal relationships in software without having to run costly experiments.
 - There are many forms of CI techniques with slightly different aims, but in this framework we focus on graphical CI techniques that use directed acyclic graphs to obtain causal estimates. These approaches used a causal DAG to explain the causal relationships that exist in data and, based on the structure of this graph, design statistical experiments capable of estimating the causal effect of a particular intervention or action, such as taking a drug or changing the value of an input variable.
 
-# Installation
+## Installation
 
 To use the causal testing framework, clone the repository, `cd` into the root directory, and run `pip install -e .`. More detailled installation instructions can be found in the [online documentation](https://causal-testing-framework.readthedocs.io/en/latest/installation.html).
 
-
-# Usage
+## Usage
 
 There are currently two methods of using the Causal Testing Framework, through the [JSON Front End](https://causal-testing-framework.readthedocs.io/en/latest/json_front_end.html) or directly as described below.
 
 The causal testing framework is made up of three main components: Specification, Testing, and Data Collection. The first step is to specify the (part of the) system under test as a modelling `Scenario`. Modelling scenarios specify the observable variables and any constraints which exist between them. We currently support three types of variable:
-  - `Input` variables are input parameters to the system.
-  - `Output` variables are outputs from the system.
-  - `Meta` variables are not directly observable but are relevant to system testing, e.g. a model may take a `location` parameter and expand this out into `average_age` and `household_size` variables "under the hood". These parameters can be made explicit by instantiating them as metavariables.
+
+- `Input` variables are input parameters to the system.
+- `Output` variables are outputs from the system.
+- `Meta` variables are not directly observable but are relevant to system testing, e.g. a model may take a `location` parameter and expand this out into `average_age` and `household_size` variables "under the hood". These parameters can be made explicit by instantiating them as metavariables.
 
 To instantiate a scenario, simply provide a set of variables and an optional set of constraints, e.g.
-```
+
+```{python}
 from causal_testing.specification.variable import Input, Output, Meta
 from causal_testing.specification.scenario import Scenario
 
@@ -57,7 +59,8 @@ modelling_scenario = Scenario({x, y, z}, {x > z, z < 3})  # Define a scenario wi
 Note that scenario constraints are primarily intended to help specify the region of the input space under test in a manner consistent with the Category Partition Method. It is not intended to serve as a test oracle. Use constraints sparingly and with caution to avoid introducing data selection bias. We use Z3 to handle constraints. For help with this, check out [their documentation](https://ericpony.github.io/z3py-tutorial/guide-examples.htm).
 
 Having fully specified the modelling scenario, we are now ready to test. Causal tests are, essentially [metamorphic tests](https://en.wikipedia.org/wiki/Metamorphic_testing) which are executed using statistical causal inference. A causal test expresses the change in a given output that we expect to see when we change a particular input in a particular way, e.g.
-```
+
+```{python}
 from causal_testing.testing.causal_test_case import CausalTestCase
 from causal_testing.testing.causal_test_outcome import Positive
 
@@ -72,15 +75,15 @@ causal_test_case = CausalTestCase(
 Before we can run our test case, we first need data. There are two ways to acquire this: 1. run the model with the specific input configurations we're interested in, 2. use data from previous model runs. For a small number of specific tests where accuracy is critical, the first approach will yield the best results. To do this, you need to instantiate the `ExperimentalDataCollector` class.
 
 Where there are many test cases using pre-existing data is likely to be faster. If the program's behaviour can be estimated statistically, the results should still be reliable as long as there is enough data for the estimator to work as intended. This will vary depending on the program and the estimator. To use this method, simply instantiate the `ObservationalDataCollector` class with the modelling scenario and a path to the CSV file containing the runtime data, e.g.
-```
+
+```{python}
 data_csv_path = 'results/data.csv'
 data_collector = ObservationalDataCollector(modelling_scenario, data_csv_path)
 ```
 
-
 The actual running of the tests is done using the `CausalTestEngine` class. This is still a work in progress and may change in the future to improve ease of use, but currently proceeds as follows.
 
-```
+```{python}
 causal_test_engine = CausalTestEngine(causal_test_case, causal_specification, data_collector)  # Instantiate the causal test engine
 minimal_adjustment_set = causal_test_engine.load_data(data_csv_path, index_col=0)  # Calculate the adjustment set
 treatment_vars = list(causal_test_case.treatment_input_configuration)
@@ -89,14 +92,14 @@ minimal_adjustment_set = minimal_adjustment_set - set([v.name for v in treatment
 
 Whether using fresh or pre-existing data, a key aspect of causal inference is estimation. To actually execute a test, we need an estimator. We currently support two estimators: linear regression and causal forest. These can simply be instantiated as per the [documentation](https://causal-testing-framework.readthedocs.io/en/latest/autoapi/causal_testing/testing/estimators/index.html).
 
-```
+```{python}
 from causal_testing.testing.estimators import LinearRegressionEstimator
 estimation_model = LinearRegressionEstimator("x",), 0, 1, minimal_adjustment_set, ("y",), causal_test_engine.scenario_execution_data_df)
 ```
 
 We can now execute the test using the estimation model. This returns a causal test result, from which we can extract various information. Here, we simply assert that the observed result is (on average) what we expect to see.
 
-```
+```{python}
 causal_test_result = causal_test_engine.execute_test(estimation_model)
 test_passes = causal_test_case.expected_causal_effect.apply(causal_test_result)
 assert test_passes, "Expected to see a positive change in y."

causal_testing/pyflakes.sh

@@ -1,35 +1,41 @@
 from setuptools import setup, find_packages
 
 requirements = [
-    "numpy~=1.20",
-    "pandas~=1.3.4",
-    "setuptools~=58.5.3",
-    "networkx~=2.6.3",
-    "pygraphviz~=1.7",
-    "scikit-learn~=1.0.1",
-    "matplotlib~=3.5.0",
     "econml~=0.12.0",
-    "statsmodels~=0.13.1",
-    "z3-solver~=4.8.13.0",
-    "lhsmdu",
-    "tabulate",
-    "scipy~=1.7.2",
-    "fitter~=1.4"
+    "fitter~=1.4.0",
+    "lhsmdu~=1.1",
+    "networkx~=2.6.3",
+    "numpy~=1.22.4",
+    "pandas~=1.3.5",
+    "scikit_learn~=1.1.2",
+    "scipy~=1.7.3",
+    "statsmodels~=0.13.2",
+    "tabulate~=0.8.10",
+    "z3_solver~=4.8.13.0",
 ]
 
 # Additional dependencies for development
-dev_requirements = [
-        "autopep8",
-        "isort",
-        "pytest",
-        "pylint",
-        "black"
-]
+dev_requirements = ["autopep8", "isort", "pytest", "pylint", "black"]
+
+readme = open("README.md", encoding = "UTF-8").read()
 
 setup(
     name="causal_testing_framework",
     version="0.0.1",
+    description="A framework for causal testing using causal directed acyclic graphs.",
+    long_description=readme,
+    long_description_content_type="text/markdown",
+    author="The CITCOM team",
+    url="https://github.com/CITCOM-project/CausalTestingFramework",
+    project_urls={
+        "Bug Tracker": "https://github.com/CITCOM-project/CausalTestingFramework/issues",
+        "Documentation": "https://causal-testing-framework.readthedocs.io/",
+        "Source": "https://github.com/CITCOM-project/CausalTestingFramework",
+    },
+    python_requires=">=3.9",
     install_requires=requirements,
     extras_require={"dev": dev_requirements},
-    packages=find_packages()
+    packages=find_packages(),
+    license="MIT",
+    keywords="causal inference, verification",
 )