|
1 | | -""" |
2 | | -Script to automate verification of tstrait against known statistical |
3 | | -results and benchmark programs such as AlphaSimR and simplePHENOTYPES. |
4 | | -
|
5 | | -We have conducted the following tests: |
6 | | -
|
7 | | -1. Exact tests |
8 | | -We simulated effect sizes and phenotypes by using AlphaSimR, simplePHENOTYPES |
9 | | -and the simulation framework described in ARG-Needle paper, and used the |
10 | | -simulated effect sizes in tstrait to simulate phenotypes, while setting the |
11 | | -environmental noise to be zero in all simulations. We then tested if the |
12 | | -simulated phenotypes in tstrait exactly match the simulated phenotypes |
13 | | -of external programs. |
14 | | -
|
15 | | -This test aims to examine whether tstrait can correctly use the genetic information |
16 | | -of individuals to accurately compute the genetic values. We have validated the |
17 | | -tstrait's output for single trait simulation and pleiotropic trait simulation. |
18 | | -These tests are implemented in `ExactTest` class. |
19 | | -
|
20 | | -2. Comparison tests |
21 | | -We simulated phenotypes in AlphaSimR, simplePHENOTYPES and the simulation |
22 | | -framework described in ARG-Needle paper by using the same parameters as the |
23 | | -tstrait simulation. We have simulated traits for a single individual in the |
24 | | -tree sequence multiple times and examined if their phenotype distributions match |
25 | | -by using a QQ-plot. |
26 | | -
|
27 | | -This test serves as an end to end testing of tstrait with environmental noise |
28 | | -simulation and tries to examine if the statistical properties of the |
29 | | -simulated traits matches the output of different simulation packages. |
30 | | -We have examined the tstrait output for different values of |
31 | | -heritability and the alpha parameter that is used in the frequency dependence |
32 | | -architecture. These tests are implemented in ComparisonTest. |
33 | | -
|
34 | | -3. Statistical tests |
35 | | -We have examined the statistical properties of tstrait's simulation output. |
36 | | -The tests in `EffectSizeDistribution` examine the statistical properties of |
37 | | -simulated effect sizes and the tests in `EnvironmentalNoise` examine the |
38 | | -simulated environmental noise. |
39 | | -
|
40 | | -NOTE: The properties of tstrait's simulation algorithm (such as whether it |
41 | | -can correctly detect mutations in a tree sequence) are validated in unit tests. |
42 | | -
|
43 | | -THe differences between each simulators are highlighted as the following: |
44 | | -1. simplePHENOTYPES |
45 | | -- Effect sizes can only be simulated from geometric series, so a normal |
46 | | -distribution must be specified if we want to simulate traits where effect |
47 | | -sizes are drawn from a normal distribution |
48 | | -- Ancestral state is set as a causal state in simplePHENOTYPES |
49 | | -
|
50 | | -2. AlphaSimR |
51 | | -- Genetic values are normalized in the simulation process |
52 | | -
|
53 | | -3. Simulation framework in ARG-Needle paper |
54 | | -- We assume that all sites are causal |
55 | | -
|
56 | | -These codes are largely adapted from msprime/verification.py. Please |
57 | | -see its documentation for usage. |
58 | | -
|
59 | | -""" |
| 1 | +# Script to automate verification of tstrait against known statistical |
| 2 | +# results and benchmark programs such as AlphaSimR and simplePHENOTYPES. |
| 3 | + |
| 4 | +# We have conducted the following tests: |
| 5 | + |
| 6 | +# 1. Exact tests |
| 7 | +# We simulated effect sizes and phenotypes by using AlphaSimR, simplePHENOTYPES |
| 8 | +# and the simulation framework described in ARG-Needle paper, and used the |
| 9 | +# simulated effect sizes in tstrait to simulate phenotypes, while setting the |
| 10 | +# environmental noise to be zero in all simulations. We then tested if the |
| 11 | +# simulated phenotypes in tstrait exactly match the simulated phenotypes |
| 12 | +# of external programs. |
| 13 | + |
| 14 | +# This test aims to examine whether tstrait can correctly use the genetic information |
| 15 | +# of individuals to accurately compute the genetic values. We have validated the |
| 16 | +# tstrait's output for single trait simulation and pleiotropic trait simulation. |
| 17 | +# These tests are implemented in `ExactTest` class. |
| 18 | + |
| 19 | +# 2. Comparison tests |
| 20 | +# We simulated phenotypes in AlphaSimR, simplePHENOTYPES and the simulation |
| 21 | +# framework described in ARG-Needle paper by using the same parameters as the |
| 22 | +# tstrait simulation. We have simulated traits for a single individual in the |
| 23 | +# tree sequence multiple times and examined if their phenotype distributions match |
| 24 | +# by using a QQ-plot. |
| 25 | + |
| 26 | +# This test serves as an end to end testing of tstrait with environmental noise |
| 27 | +# simulation and tries to examine if the statistical properties of the |
| 28 | +# simulated traits matches the output of different simulation packages. |
| 29 | +# We have examined the tstrait output for different values of |
| 30 | +# heritability and the alpha parameter that is used in the frequency dependence |
| 31 | +# architecture. These tests are implemented in ComparisonTest. |
| 32 | + |
| 33 | +# 3. Statistical tests |
| 34 | +# We have examined the statistical properties of tstrait's simulation output. |
| 35 | +# The tests in `EffectSizeDistribution` examine the statistical properties of |
| 36 | +# simulated effect sizes and the tests in `EnvironmentalNoise` examine the |
| 37 | +# simulated environmental noise. |
| 38 | + |
| 39 | +# NOTE: The properties of tstrait's simulation algorithm (such as whether it |
| 40 | +# can correctly detect mutations in a tree sequence) are validated in unit tests. |
| 41 | + |
| 42 | +# THe differences between each simulators are highlighted as the following: |
| 43 | +# 1. simplePHENOTYPES |
| 44 | +# - Effect sizes can only be simulated from geometric series, so a normal |
| 45 | +# distribution must be specified if we want to simulate traits where effect |
| 46 | +# sizes are drawn from a normal distribution |
| 47 | +# - Ancestral state is set as a causal state in simplePHENOTYPES |
| 48 | + |
| 49 | +# 2. AlphaSimR |
| 50 | +# - Genetic values are normalized in the simulation process |
| 51 | + |
| 52 | +# 3. Simulation framework in ARG-Needle paper |
| 53 | +# - We assume that all sites are causal |
| 54 | + |
| 55 | +# These codes are largely adapted from msprime/verification.py. Please |
| 56 | +# see its documentation for usage. |
| 57 | + |
60 | 58 | import argparse |
61 | 59 | import concurrent.futures |
62 | 60 | import inspect |
|
0 commit comments