1
1
Background
2
2
=====================================
3
3
4
- The Causal Testing Framework consists of 3 main components: 1) Causal Specification, 2) Causal Test Case and 3) Data Collection .
4
+ The Causal Testing Framework consists the following two components: 1) Causal Specification and 2) Causal Test Case.
5
5
6
6
#.
7
7
:doc: `Causal Specification <../modules/causal_specification >`\ : To apply graphical CI
8
- techniques for testing, we need a causal DAG which depicts causal relationships amongst inputs and outputs. To
8
+ techniques for testing, we need a causal DAG, which depicts causal relationships amongst inputs and outputs. To
9
9
collect this information, users must create a *causal specification *. This comprises a set of scenarios which place
10
10
constraints over input variables that capture the use-case of interest, a causal DAG corresponding to this scenario,
11
11
and a series of high-level functional requirements that the user wishes to test. In causal testing, these
@@ -14,7 +14,7 @@ The Causal Testing Framework consists of 3 main components: 1) Causal Specificat
14
14
15
15
16
16
#.
17
- :doc: `Causal Tests <../modules/causal_tests >`\ : With a causal specification in hand, we can now go about designing
17
+ :doc: `Causal Tests <../modules/causal_tests >`\ : With a causal specification in hand, we can now design
18
18
a series of test cases that interrogate the causal relationships of interest in the scenario-under-test. Informally,
19
19
a causal test case is a triple ``(M, X, Delta, Y) ``, where ``M `` is the modelling scenario, ``X `` is an input configuration,
20
20
``Delta `` is an intervention which should be applied to ``X ``, and ``Y `` is the expected *causal effect * of that intervention on
@@ -24,14 +24,13 @@ The Causal Testing Framework consists of 3 main components: 1) Causal Specificat
24
24
25
25
a. Using the causal DAG, identify an estimand for the effect of the intervention on the output of interest. That is,
26
26
a statistical procedure capable of estimating the causal effect of the intervention on the output.
27
- #. Collect the data to which the statistical procedure will be applied (see Data collection below).
28
- #. Apply a statistical model (e.g. linear regression or causal forest) to the data to obtain a point estimate for
27
+ #. Apply a statistical model (e.g. linear regression or logistic regression) to the data to obtain a point estimate for
29
28
the causal effect. Depending on the estimator used, confidence intervals may also be obtained at a specified
30
- confidence level e.g. 0.05 corresponds to 95% confidence intervals (optional).
29
+ significance level, e.g. 0.05 corresponds to 95% confidence intervals (optional).
31
30
#. Return the casual test result including a point estimate and 95% confidence intervals, usually quantifying the
32
31
average treatment effect (ATE).
33
32
#. Implement and apply a test oracle to the causal test result - that is, a procedure that determines whether the
34
33
test should pass or fail based on the results. In the simplest case, this takes the form of an assertion which
35
34
compares the point estimate to the expected causal effect specified in the causal test case.
36
35
37
- For more information on each of these steps, follow the link to their respective documentation.
36
+ For more information on each of these steps, follow the links above to their respective documentation.
0 commit comments