Skip to content

Commit 85a1da3

Browse files
committed
Fixed citations
1 parent 772d5c9 commit 85a1da3

File tree

1 file changed

+8
-6
lines changed

1 file changed

+8
-6
lines changed

paper/paper.md

Lines changed: 8 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -50,9 +50,9 @@ Causal Inference works by using domain knowledge to identify and mitigate for bi
5050
The Causal Testing Framework is a software testing framework that uses Causal Inference techniques to establish causal effects between software variables from pre-existing runtime data rather than having to collect bespoke, highly curated datasets especially for testing.
5151

5252
# Statement of need
53-
Metamorphic Testing @[chen1998metamorphic] is a popular technique for testing computational models (and other traditionally "hard to test" software).
53+
Metamorphic Testing [@chen1998metamorphic] is a popular technique for testing computational models (and other traditionally "hard to test" software).
5454
Test goals are expressed as _metamorphic relations_ that specify how changing an input in a particular way should affect the software output.
55-
Nondeterministic software can be tested using Statistical Metamorphic Testing @[guderlei2007smt], which uses statistical tests over multiple executions of the software to determine whether the specified metamorphic relations hold.
55+
Nondeterministic software can be tested using Statistical Metamorphic Testing [@guderlei2007smt], which uses statistical tests over multiple executions of the software to determine whether the specified metamorphic relations hold.
5656
However, this requires the software to be executed repeatedly for each set of parameters of interest, so is computationally expensive, and is constrained to testing properties over software inputs that can be directly and precisely controlled.
5757
Statistical Metamorphic Testing cannot be used to test properties that relate internal variables or outputs to each other, since these cannot be controlled a priori.
5858

@@ -61,19 +61,19 @@ The Causal Testing Framework is written in python but is language agnostic in te
6161
All that is required is a set of properties to be validated, a causal model, and a set of software runtime data.
6262

6363
# Causal Testing
64-
Causal Testing @[clark2023testing] has four main steps, outlined in \ref{fig:schematic}.
64+
Causal Testing [@clark2023testing] has four main steps, outlined in \ref{fig:schematic}.
6565
Firstly, the user supplies a causal model, which takes the form of a directed acyclic graph (DAG) in which an edge $X \to Y$ represents variable $X$ having a direct causal effect on variable $Y$.
6666
Secondly, the user supplies a set of causal properties to be tested.
67-
Such properties can be generated from the causal DAG @[clark2023metamorphic]: for each $X \to Y$ edge, a test to validate the presence of a causal effect is generated, and for each missing edge, a test to validate independence is generated.
67+
Such properties can be generated from the causal DAG [@clark2023metamorphic]: for each $X \to Y$ edge, a test to validate the presence of a causal effect is generated, and for each missing edge, a test to validate independence is generated.
6868
The user may also refine tests to validate the nature of a particular relationship.
6969
Next, the user supplies a set of runtime data in the form of a table with each column representing a variable and rows containing the value of each variable for a particular run of the software.
7070
Finally, the Causal Testing Framework automatically validates the supplied causal properties by using the supplied causal DAG and data to calculate a causal effect estimate, and validating this against the expected causal relationship.
7171

72-
![Causal Testing workflow.\label{fig:schematic}](images/schematic.png)
72+
![Causal Testing workflow.\label{fig:schematic}](../images/schematic.png)
7373

7474
## Test Adequacy
7575
Because the properties being tested are completely separate from the data used to validate them, traditional coverage-based metrics are not appropriate here.
76-
The Causal Testing Framework instead evaluates the adequacy of a particular dataset by calculating a statistical metric @[foster2024adequacy] based on the stability of the causal effect estimate, with numbers closer to zero representing more adequate data.
76+
The Causal Testing Framework instead evaluates the adequacy of a particular dataset by calculating a statistical metric [@foster2024adequacy] based on the stability of the causal effect estimate, with numbers closer to zero representing more adequate data.
7777

7878
## Missing Variables
7979
Causal Testing works by using the supplied causal DAG to identify those variables which need to be statistically controlled for to remove their biassing effect on the causal estimate.
@@ -87,3 +87,5 @@ Traditional Causal Inference cannot handle this, however the Causal Testing Fram
8787

8888
# Acknowledgements
8989
This work was supported by the EPSRC CITCoM grant EP/T030526/1.
90+
91+
# References

0 commit comments

Comments
 (0)