Skip to content

Commit 0915866

Browse files
committed
Add an use case to motivation
1 parent a15faf5 commit 0915866

File tree

1 file changed

+15
-7
lines changed

1 file changed

+15
-7
lines changed

toc/rfc/rfc-draft-hash-based-routing.md

Lines changed: 15 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -26,8 +26,16 @@ allows these load balancing algorithms to be configured on the application route
2626

2727
However, these existing algorithms are not ideal for scenarios that require routing based on specific identifiers.
2828

29-
An example scenario: users from different tenants send requests to application instances that establish connections to
30-
tenant-specific databases.
29+
One use case is optimizing resource management of complex in-memory caches. While 12-factor apps are stateless and can
30+
retrieve necessary information from backing services, it is often useful to cache data and reduce latency. When a cache
31+
is limited in size (e.g., Least Recently Used), exposing each app instance to all users may lead to thrashing and lower
32+
cache efficiency. By "pinning" users to a particular instance, the cache can remain effective. In the event of an
33+
instance exchange (up or downscaling, evacuation, rolling update), another instance can still provide a response and
34+
fill its cache without interruption for the user. For most users, subsequent requests can be processed at lower latency
35+
by utilizing a warm and effective cache.
36+
37+
Another use case: users from different tenants send requests to application instances that establish connections to
38+
tenant-specific databases.
3139

3240
![](rfc-draft-hash-based-routing/problem.drawio.png)
3341

@@ -42,13 +50,13 @@ derived from request identifiers, such as headers.
4250
## Proposal
4351

4452
We propose introducing hash-based routing as a load balancing algorithm for use on a per-route basis to address the
45-
issues described in the earlier scenario.
53+
issues described in the earlier use cases.
4654

4755
The approach leverages an HTTP header, which is associated with each incoming request and contains the specific
4856
identifier. This one is used to compute a hash value, which will serve as the basis for routing decisions.
4957

50-
In the previously mentioned scenario, the tenant ID acts as the identifier included in the header and serves as the
51-
basis for hash calculation. This hash value determines the appropriate application instance for each request, ensuring
58+
In the previously mentioned use cases, the specific identifier included in the header can serve as the basis for hash
59+
calculation. This hash value determines the appropriate application instance for each request, ensuring
5260
that all requests with this identifier are consistently routed to the same instance or might be routed to another
5361
instance when the instance is saturated. Consequently, the load balancing algorithm effectively directs requests for a
5462
single tenant to a particular application instance, so that instance can minimize database connection overhead and
@@ -122,12 +130,12 @@ A possible presentation of deterministic handling can be a ring like:
122130

123131
- Gorouter MUST be extended to take a specific identifier via the request header
124132
- Gorouter MUST implement hash calculation, based on the provided header
125-
- Gorouter SHOULD store the mapping between computed hash values and application instances locally to avoid
133+
- Gorouter MAY store the mapping between computed hash values and application instances locally to avoid
126134
expensive recalculations for each incoming request
127135
- Gorouters SHOULD NOT implement a distributed shared cache
128136
- Gorouter MUST assess the current number of in-flight requests across all application instances mapped to a
129137
particular route to consider overload situations
130-
- Gorouter MUST update its local hash table following the registration or deregistration of an endpoint, ensuring
138+
- Gorouter MAY update its local hash table following the registration or deregistration of an endpoint, ensuring
131139
minimal rehashing
132140
- Gorouter SHOULD NOT incur any performance hit when 0 apps use hash routing.
133141

0 commit comments

Comments
 (0)