|
| 1 | +--- |
| 2 | +id: jetstream |
| 3 | +title: Jetstream overview |
| 4 | +--- |
| 5 | + |
| 6 | +**[Jetstream]** is an analysis framework for experiments. |
| 7 | +Jetstream aggregates and summarizes product telemetry, |
| 8 | +using the experiment definitions in Experimenter, |
| 9 | +to produce the datasets that drive the results dashboards. |
| 10 | + |
| 11 | +Most investigation owners will not interact with Jetstream directly. |
| 12 | +Jetstream computes a default set of statistics for every experiment. |
| 13 | +Investigation owners can add additional metrics to a results dashboard by choosing |
| 14 | +**outcomes** <!-- TODO: linkme --> |
| 15 | +in Experimenter while designing an experiment. |
| 16 | + |
| 17 | +Data scientists can extend Jetstream with new outcomes by contributing |
| 18 | +**outcome definitions** <!-- TODO: linkme --> |
| 19 | +to the [jetstream-config] repository. |
| 20 | + |
| 21 | +Data scientists can also ask Jetstream to evaluate custom metrics for a particular experiment |
| 22 | +by contributing |
| 23 | +**experiment configurations** <!-- TODO: linkme --> |
| 24 | +to jetstream-config. |
| 25 | + |
| 26 | +## Analysis paradigm |
| 27 | + |
| 28 | +Experiments are analyzed using the concept of analysis windows. Analysis |
| 29 | +windows describe an interval marked from each client’s day of |
| 30 | +enrollment. The “day 0” analysis window aggregates data from the days |
| 31 | +that each client enrolled in the experiment. Because the intervals are |
| 32 | +demarcated from enrollment, they are not calendar dates; for some |
| 33 | +clients in an experiment, day 0 could be a Tuesday, and for others a |
| 34 | +Saturday. |
| 35 | + |
| 36 | +The week 0 analysis window aggregates data from each client’s days 0 |
| 37 | +through 6, the week 1 window aggregates data from days 7 through 13, and |
| 38 | +so on. |
| 39 | + |
| 40 | +Clients are given a fixed amount of time, specified in Experimenter and |
| 41 | +often a week long, to enroll. Final day 0 results are available for |
| 42 | +reporting at the end of the enrollment period, after the last eligible |
| 43 | +client has enrolled, and week 0 results are available a week after the |
| 44 | +enrollment period closes. Results for each window are published as soon |
| 45 | +as complete data is available for all enrolled clients. |
| 46 | + |
| 47 | +The "overall" window, published after the experiment has ended, is a |
| 48 | +window beginning on each client’s day 0 that spans the longest period |
| 49 | +for which all clients have complete data. |
| 50 | + |
| 51 | +## Datasets |
| 52 | + |
| 53 | +The datasets that back the Experimenter results dashboards |
| 54 | +are available in BigQuery. |
| 55 | +[Technical documentation][jetstream-dtmo] |
| 56 | +is available in the Mozilla data docs. |
| 57 | + |
| 58 | +[jetstream]: https://github.com/mozilla/jetstream |
| 59 | +[jetstream-config]: https://github.com/mozilla/jetstream-config |
| 60 | +[jetstream-dtmo]: https://docs.telemetry.mozilla.org/datasets/jetstream.html |
0 commit comments