Skip to content

Commit ee98a2a

Browse files
committed
Updated docs
1 parent 5b78c16 commit ee98a2a

File tree

5 files changed

+18
-56
lines changed

5 files changed

+18
-56
lines changed

docs/source/description.rst

Lines changed: 0 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -34,18 +34,4 @@ The Causal Testing Framework consists of 3 main components: 1) Causal Specificat
3434
test should pass or fail based on the results. In the simplest case, this takes the form of an assertion which
3535
compares the point estimate to the expected causal effect specified in the causal test case.
3636

37-
38-
39-
#.
40-
:doc:`Data Collection <../modules/data_collector>`\ : Data for the system-under-test can be collected in two
41-
ways: experimentally or observationally. The former involves executing the system-under-test under controlled
42-
conditions which, by design, isolate the causal effect of interest (accurate but expensive), while the latter
43-
involves collecting suitable previous execution data and utilising our causal knowledge to draw causal inferences (
44-
potentially less accurate but efficient). To collect experimental data, the user must implement a single method which
45-
runs the system-under-test with a given input configuration. On the other hand, when dealing with observational data,
46-
we automatically check whether the data is suitable for the identified estimand in two steps. First, confirm whether
47-
the data contains a column for each variable in the causal DAG. Second, we check
48-
for `positivity violations <https://www.youtube.com/watch?v=4xc8VkrF98w>`_. If there are positivity violations, we can
49-
provide instructions for an execution that will fill the gap (future work).
50-
5137
For more information on each of these steps, follow the link to their respective documentation.

docs/source/index.rst

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -106,7 +106,6 @@ system-under-test that is expected to cause a change to some output(s).
106106
:maxdepth: 1
107107
:caption: Module Descriptions
108108

109-
/modules/data_collector
110109
/modules/causal_specification
111110
/modules/causal_tests
112111

docs/source/modules/data_collector.rst

Lines changed: 0 additions & 28 deletions
This file was deleted.

docs/source/usage.rst

Lines changed: 13 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -65,19 +65,23 @@ the given output and input and the desired effect. This information is the minim
6565
6666
Before we can run our test case, we first need data. There are two ways to acquire this: 1. run the model with the
6767
specific input configurations we're interested in, 2. use data from previous model runs. For a small number of specific
68-
tests where accuracy is critical, the first approach will yield the best results. To do this, you need to instantiate
69-
the ``ExperimentalDataCollector`` class.
68+
tests where accuracy is critical, the first approach will yield the best results. To do this, you can use the
69+
`ExperimentalEstimator` class. This will run the system directly and calculate the causal effect estimate from this.
7070

71-
Where there are many test cases using pre-existing data is likely to be faster. If the program's behaviour can be
71+
Where there are many test cases, using pre-existing data is likely to be faster. If the program's behaviour can be
7272
estimated statistically, the results should still be reliable as long as there is enough data for the estimator to work
7373
as intended. This will vary depending on the program and the estimator. To use this method, simply instantiate
74-
the ``ObservationalDataCollector`` class with the modelling scenario and a path to the CSV file containing the runtime
75-
data, e.g.
74+
one of the other estimator classes with a Pandas dataframe containing the runtime data, e.g.
7675

7776
.. code-block:: python
78-
79-
obs_df = pd.read_csv('results/data.csv')
80-
data_collector = ObservationalDataCollector(modelling_scenario, obs_df)
77+
estimator = LinearRegressionEstimator(
78+
treatment_variable,
79+
treatment_value,
80+
control_value,
81+
minimal_adjustment_set,
82+
outcome_variable,
83+
df=pd.read_csv(observational_data_path),
84+
)
8185
8286
8387
Whether using fresh or pre-existing data, a key aspect of causal inference is estimation. To actually execute a test, we
@@ -99,7 +103,7 @@ various information. Here, we simply assert that the observed result is (on aver
99103

100104
.. code-block:: python
101105
102-
causal_test_result = causal_test_case.execute_test(estimation_model, data_collector)
106+
causal_test_result = causal_test_case.execute_test(estimation_model)
103107
test_passes = causal_test_case.expected_causal_effect.apply(causal_test_result)
104108
assert test_passes, "Expected to see a positive change in y."
105109

examples/covasim_/vaccinating_elderly/README.md

Lines changed: 5 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -13,15 +13,16 @@ four test cases: one focusing on each of the four previously mentioned outputs.
1313

1414
Further details are provided in Section 5.3 (Prioritising the elderly for vaccination) of the paper.
1515

16-
**Note**: this version of the CTF utilises the observational data collector in order to separate the software execution
17-
and testing. Older versions of this framework simulate the data using the custom experimental data collector and the
18-
`covasim` package (version 3.0.7) as outlined below.
16+
>[!NOTE]
17+
>This version of the CTF uses observational data to separate the software execution and testing.
18+
Older versions of this framework simulate the data using a custom experimental data collector and the `covasim`
19+
package (version 3.0.7) as outlined below.
1920

2021
## How to run
2122
To run this case study:
2223
1. Ensure all project dependencies are installed by running `pip install .` from the top
2324
level of this directory (instructions are provided in the project README).
24-
2. Additionally, in order to run Covasim, install version 3.0.7 by running `pip install covasim==3.0.7`.
25+
2. If necessary, install version 3.0.7 by running `pip install covasim==3.0.7`.
2526
3. Change directory to `causal_testing/examples/covasim_/vaccinating_elderly`.
2627
4. Run the command `python example_vaccine.py`.
2728

0 commit comments

Comments
 (0)