You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/sdks/faqs/latency.md
+71-31Lines changed: 71 additions & 31 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -4,22 +4,26 @@ sidebar_position: 6
4
4
5
5
# Latency
6
6
7
-
Eppo’s feature flagging architecture enables faster delivery of client-side web experiments. By leveraging a "dumb server, smart client" approach and utilizing the power of the Fastly CDN, Eppo offers low-latency feature flag evaluations and efficient updates.
7
+
## Introduction
8
8
9
-
### Architecture Overview
9
+
Eppo’s feature flags leverage a "dumb server, smart client" approach. This, paired with using Fastly's Global CDN, allows Eppo to provide low-latency SDK initialization and near instantaneous evaluation of feature flag and experiment variations.
10
10
11
-
The Eppo feature flagging architecture is designed as a JSON delivery service. The server maintains a file containing feature flags and their corresponding assignment rules. The client downloads this file and determines which feature flags apply to its group.
11
+
The system is designed as a JSON delivery service. Our CDN maintains a file containing feature flags and their corresponding assignment rules. The SDK client then downloads this file and determines what variant to apply for a specific subject (e.g., user).
12
12
13
-
Our architecture follows the "smart client" principle, offloading the evaluation work to the client-side SDKs. This approach requires initial development effort to implement new SDKs and update existing ones when new targeting rules are introduced. However, it allows us to tap into the resources of the Fastly CDN, benefiting from its global infrastructure and distributed caching capabilities.
13
+
Offloading the evaluation work to the SDK means that once the SDK is initialized all evaluation happens locally, typically in under 1 ms. Further, if user context changes mid-session, there is no need to reach out to Eppo's servers to understand their new targeting eligibility. All of the targeting happens locally, so Eppo's SDK will always ensure users see the right experiment given their unique targeting attributes.
14
14
15
-
### Latency Considerations
15
+
This page walks through some of the latency considerations associated with building a global feature flagging system, describes options for how to reach your internal SLAs, and presents performance benchmarks.
16
16
17
-
When it comes to feature-flagging services, latency is a crucial factor for client experiences. We distinguish between two types of latency:
17
+
## Latency Considerations
18
18
19
-
1.**Evaluation Latency**: The time it takes for a client to determine which flag value applies to it.
20
-
2.**Update Latency**: The time it takes for updated rules to reach the client.
19
+
When it comes to feature-flagging services, latency is a crucial factor for end user experiences. We distinguish between two types of latency:
21
20
22
-
We have made architectural tradeoffs to optimize these latency factors. Instead of a "smart server" architecture that requires frequent server polling, Eppo prioritizes fast evaluations by accepting relatively slower updates.
21
+
1.**Evaluation Latency**: The time it takes to determine which flag value applies for a specific subject (user).
22
+
2.**Update Latency**: The time it takes for updated rules to reach end users.
23
+
24
+
Eppo's "smart client" approach allows us to give very impressive evaluation latencies (typically under 1ms), while still providing update latencies that satisfy internal SLAs for disabling problematic features.
25
+
26
+
This is in contrast to a server-side evaluation which requires frequent network requests each time a flag is evaluated, or a user's context changes.
23
27
24
28
### Leveraging the Fastly CDN
25
29
@@ -29,32 +33,68 @@ The diagram below illustrates our architecture, where requests enter from the ri
29
33
30
34
31
35
32
-
This architectural choice allows us to achieve impressive latency figures. Let's explore the uncached and cached latency numbers to understand the client experience better.
33
-
34
-
### Uncached Latency
35
-
36
-
The following map displays uncached latency figures obtained using a generic latency testing tool. Clients can download the feature-flag file in less than a second from most locations worldwide.
The cached latency numbers provide a more representative view of typical client experiences. These numbers correspond to requests made within a couple of minutes after the initial request.
Most regions, including the US, Europe, South Korea, and Australia, experience latencies under 100ms. Even in locations further away, latencies are generally below half a second. To put it into perspective, the ping time between New York and London is approximately 72ms. From the client's perspective, it appears as though Eppo's servers are distributed globally, even though they are physically located in the corn fields and cow pastures of Iowa.
36
+
### Latency benchmark
37
+
38
+
This architectural choice allows us to achieve impressive latency figures. To understand initialization times around the world, we ran a benchmark with ~80 active feature flag using the open source tool Grafana k6. Two users (i.e., threads) executed 500 requests in a row. This is repeated for each region in GCP.
39
+
40
+
The table below shows the measured percentiles for each region, reported in milliseconds.
41
+
42
+
:::note
43
+
These figures measure latency from the VMs used for the test, so they are data center to data center. These figures will not be representative of what an end-user would experience in the mentioned locations for each provider - those figures would be higher (due to having to traverse the open internet/low bandwidth connections).
Most regions experience latencies under 20ms. Even in locations further away, latencies are well below half a second. To put this into perspective, the ping time between New York and London is approximately 72ms. From the client's perspective, it appears as though Eppo's servers are distributed globally, even though they are physically located in the corn fields and cow pastures of Iowa.
47
90
48
91
### Update Latency
49
92
50
-
Update latency refers to the time required for updated feature-flagging rules (JSON file) to reach the clients.
51
-
52
-
For client side SDKs (Android, iOS, React, etc), the rules are updated each time the Eppo SDK is initialized, which should happen only one time per application lifecycle. If the SDK is initialized more than once during an application’s lifecycle, an exception is thrown.
93
+
Update latency refers to the time required for updated feature-flagging rules (JSON file) to reach the clients. Most of Eppo's SDKs have built-in polling that is configurable at initialization. This makes it easy to set a desired update latency: simply set the polling cadence to reach your internal SLA for changes to go live.
53
94
54
-
For server side SDKs (Java, Node.js, Ruby, etc), the rules are first updated when the Eppo SDK is initialized, then again every 30 seconds for the duration of the application’s lifecycle.
95
+
Some SDKs also offer a method to manually trigger a reload of configurations, allowing for a lot of flexibility in how to handle update latency.
55
96
56
-
###Conclusion
97
+
## Conclusion
57
98
58
-
Eppo's feature-flag architecture offers a faster and cost-effective way to deliver client-side web experiments. By prioritizing client-side evaluation and leveraging the Fastly CDN, we achieve sub-100ms evaluation latencies while maintaining efficient update processes.
99
+
Eppo's feature-flagging architecture is optimized for evaluation latency. Once the SDK is initialized (typically in under 20ms), all evaluation of flags happen effectively immediately. Eppo also provides flexibility in how to handle update latency, ensuring changes made in Eppo's UI reach end users within a specified time window.
59
100
60
-
For any further questions or assistance, please refer to the Eppo Feature Flagging Service documentation or reach out to our support team.
0 commit comments