Skip to content

Commit 5beb7b1

Browse files
Add filters and errorCorrection to AB test config, remove legacy outliers and empty search
1 parent 4cebf3f commit 5beb7b1

File tree

1 file changed

+42
-21
lines changed

1 file changed

+42
-21
lines changed

specs/abtesting-v3/common/schemas/ABTest.yml

Lines changed: 42 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -58,29 +58,12 @@ ABTestConfiguration:
5858
type: object
5959
description: A/B test configuration.
6060
properties:
61-
outliers:
62-
$ref: "#/Outliers"
63-
emptySearch:
64-
$ref: "#/EmptySearch"
6561
minimumDetectableEffect:
6662
$ref: "#/MinimumDetectableEffect"
67-
68-
Outliers:
69-
type: object
70-
description: Configuration for handling outliers.
71-
properties:
72-
exclude:
73-
type: boolean
74-
description: Whether to exclude outliers when calculating A/B test results.
75-
default: true
76-
77-
EmptySearch:
78-
type: object
79-
description: Configuration for handling empty searches.
80-
properties:
81-
exclude:
82-
type: boolean
83-
description: Whether to exclude empty searches when calculating A/B test results.
63+
filters:
64+
$ref: "#/MetricsFilters"
65+
errorCorrection:
66+
$ref: "#/ErrorCorrectionType"
8467

8568
MinimumDetectableEffect:
8669
type: object
@@ -113,3 +96,41 @@ MigratedABTestId:
11396
type: integer
11497
description: Unique migrated A/B test identifier.
11598
example: 224
99+
100+
MetricsFilter:
101+
type: object
102+
additionalProperties: false
103+
description: |
104+
Boolean filter applied to the A/B test population. Each filter targets a boolean metric
105+
and decides whether to include (true) or exclude (false) matching records.
106+
properties:
107+
domain:
108+
type: string
109+
description: Metric domain (for example `abtesting`, `personalization`).
110+
example: abtesting
111+
name:
112+
type: string
113+
description: Public metric name.
114+
example: isOutlier
115+
trackEffects:
116+
type: boolean
117+
description: Whether the experiment should record the effects of this filter.
118+
includes:
119+
type: boolean
120+
description: If true, keep items that match the filter; if false, exclude them.
121+
required:
122+
- domain
123+
- name
124+
125+
MetricsFilters:
126+
type: array
127+
description: List of metric filters applied to the test population.
128+
items:
129+
$ref: "#/MetricsFilter"
130+
131+
ErrorCorrectionType:
132+
type: string
133+
description: Multiple-testing correction method applied when evaluating metric significance.
134+
enum:
135+
- bonferroni
136+
- benjamini-hochberg

0 commit comments

Comments
 (0)