You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: specs/abtesting-v3/common/schemas/Variant.yml
+26-22Lines changed: 26 additions & 22 deletions
Original file line number
Diff line number
Diff line change
@@ -6,29 +6,29 @@ variants:
6
6
The first variant is your _control_ index, typically your production index.
7
7
All of the additional variants are indexes with changed settings that you want to test against the control.
8
8
items:
9
-
$ref: "#/variant"
9
+
$ref: '#/variant'
10
10
11
11
variant:
12
12
type: object
13
13
additionalProperties: false
14
14
properties:
15
15
description:
16
-
$ref: "../parameters.yml#/description"
16
+
$ref: '../parameters.yml#/description'
17
17
estimatedSampleSize:
18
18
type: integer
19
19
description: |
20
20
Estimated number of searches required to achieve the desired statistical significance.
21
21
22
-
The A/B test configuration must include a `mininmumDetectableEffect` setting for this number to be included in the response.
22
+
The A/B test configuration must include a `minimumDetectableEffect` setting for this number to be included in the response.
23
23
example: 0
24
24
index:
25
-
$ref: "../parameters.yml#/index"
25
+
$ref: '../parameters.yml#/index'
26
26
trafficPercentage:
27
-
$ref: "../parameters.yml#/trafficPercentage"
27
+
$ref: '../parameters.yml#/trafficPercentage'
28
28
metrics:
29
-
$ref: "#/metrics"
29
+
$ref: '#/metrics'
30
30
metadata:
31
-
$ref: "#/variantMetadata"
31
+
$ref: '#/variantMetadata'
32
32
33
33
required:
34
34
- index
@@ -38,9 +38,9 @@ variant:
38
38
39
39
metrics:
40
40
type: array
41
-
description: All ABTest metrics that were defined during test creation
41
+
description: All ABTest metrics that were defined during test creation.
42
42
items:
43
-
$ref: "#/metric"
43
+
$ref: '#/metric'
44
44
45
45
metric:
46
46
type: object
@@ -56,24 +56,28 @@ metric:
56
56
valueCIHigh:
57
57
type: number
58
58
format: double
59
+
description: |
60
+
The upper bound of the 95% confidence interval for the metric value. The confidence interval is calculated using
61
+
either the relative ratio or relative difference between the metric values for the control and the variant.
62
+
Relative ratio is used for metrics that are ratios (e.g., click-through rate, conversion rate),
63
+
while relative difference is used for continuous metrics (e.g., revenue).
59
64
valueCILow:
60
65
type: number
61
66
format: double
67
+
description: |
68
+
The lower bound of the 95% confidence interval for the metric value. The confidence interval is calculated using
69
+
either the relative ratio or relative difference between the metric values for the control and the variant.
70
+
Relative ratio is used for metrics that are ratios (e.g., click-through rate, conversion rate),
71
+
while relative difference is used for continuous metrics (e.g., revenue).
62
72
pValue:
63
73
type: number
64
74
format: double
65
75
description: PValue for the first variant (control) will always be 0. For the other variants, pValue is calculated for the current variant based on the control.
66
-
pValueCIHigh:
67
-
type: number
68
-
format: double
69
-
pValueCILow:
70
-
type: number
71
-
format: double
72
76
dimension:
73
77
type: string
74
-
description: Dimension defined during test creation
78
+
description: Dimension defined during test creation.
0 commit comments