Skip to content
Open
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 22 additions & 2 deletions docs/feature-flagging/concepts/flag-allocations.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,27 @@ In Eppo, allocations are stacked in a vertical waterfall as seen below:

Here, there are two allocations:

- The first allocation is a feature gate that targets internal users, which are identified with by their email domain. If a user's email matches the criteria, they are show `Variant 1`. If not, they are moved along to the next allocation (i.e down the waterfall).
- The second allocation is an experiment allocation that targets users in North America using an iPhone. 50% of users who match this criteria see `Variant 1` and the remaining 50% of users see `Variant 2`. If users do no match, they are moved along. Since there are no more allocations to evaluate, all remaining users see the default value, which in this example is `Variant 1`.
- The first allocation is a feature gate that targets internal users, which are identified with by their email domain. If a user's email matches the criteria, they are show `Variant 1`. If not, they are moved along to the next allocation (i.e down the waterfall).
- The second allocation is an experiment allocation that targets users in North America using an iPhone. 50% of users who match this criteria see `Variant 1` and the remaining 50% of users see `Variant 2`. If users do no match, they are moved along. Since there are no more allocations to evaluate, all remaining users see the default value, which in this example is `Variant 1`.

Note that it is possible to reduce an allocation's traffic exposure to less than 100%. In this case, the allocation's rules are evaluated and then subsequently traffic exposure check is computed. Subjects that pass the rules test but fall outside the traffic exposure continue down the waterfall to the next rule.

## Traffic exposure adjustments

When you adjust the traffic exposure of an allocation:

- **Increasing traffic exposure**: When you increase traffic exposure, existing users who were already assigned to the allocation will maintain their variant assignments when they return (i.e. have sticky assignments). This means that users who have previously been bucketed for this allocation will continue to receive the same experience. Only new users (who are being evaluated for the first time) and users who were not previously bucketed for this allocation will be subject to the updated traffic rules.

- **Decreasing traffic exposure**: When you decrease traffic exposure, existing users who were already assigned to a bucket will be re-evaluated. This means that users who have previously been bucketed for a specific variant will potentially receive a different experience.

- **Experiment completion**: This is particularly useful for experiments with a measurement window. For example, if you've reached your sample size goals, you can set traffic exposure to 0% to stop new assignments while allowing already-assigned users to complete their measurement window.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this section misrepresents how traffic exposure adjustments work. For instance, changing the traffic allocation to 0% will pass everyone through to the next assignment/allocation (the default variant in most cases), even if they had previously been enrolled.

Also, it's important to state that users will only receive the same variant if they remain in the targeted audience.

For example, imagine you have a flag where free users are randomly split between control and treatment, but paid users "fall through" to the default variant (control). In this case once a user goes from free -> paid, they will stop seeing the treatment. Eppo's SDK does not store any history of who has already been assigned.

Assignments are "idempotent", but not "sticky". That is, for the same inputs to the SDK (user ID, user context), the SDK will return the same value. But if a user's context changes and they are no longer in the targeted audience, Eppo's SDK will not "remember" that they were in treatment earlier.

The SDK does guarantee that if a user was in treatment when traffic exposure was at 5%, that they'll still be in treatment once we ramp to 20% exposure. This is because the hashing for traffic allocation and variant weighting is independent. I think that's what this section is trying to say, but we should be more clear about the limitations


## Overriding allocations with feature gates

The waterfall structure of allocations allows for powerful override capabilities:

- **Error recovery**: If you discover an issue with an experiment, you can add a feature gate above it in the waterfall targeting 100% of users to a specific variant (e.g. control).

- **Priority of allocations**: Since allocations are evaluated from top to bottom, a feature gate placed above experiment allocations will take precedence, overriding any experiment assignments for both new and returning users.

- **Emergency toggles**: This structure provides a quick way to implement emergency toggles to revert all users to a safe experience when needed.