@@ -26,8 +26,16 @@ allows these load balancing algorithms to be configured on the application route
26
26
27
27
However, these existing algorithms are not ideal for scenarios that require routing based on specific identifiers.
28
28
29
- An example scenario: users from different tenants send requests to application instances that establish connections to
30
- tenant-specific databases.
29
+ One use case is optimizing resource management of complex in-memory caches. While 12-factor apps are stateless and can
30
+ retrieve necessary information from backing services, it is often useful to cache data and reduce latency. When a cache
31
+ is limited in size (e.g., Least Recently Used), exposing each app instance to all users may lead to thrashing and lower
32
+ cache efficiency. By "pinning" users to a particular instance, the cache can remain effective. In the event of an
33
+ instance exchange (up or downscaling, evacuation, rolling update), another instance can still provide a response and
34
+ fill its cache without interruption for the user. For most users, subsequent requests can be processed at lower latency
35
+ by utilizing a warm and effective cache.
36
+
37
+ Another use case: users from different tenants send requests to application instances that establish connections to
38
+ tenant-specific databases.
31
39
32
40
![ ] ( rfc-draft-hash-based-routing/problem.drawio.png )
33
41
@@ -42,13 +50,13 @@ derived from request identifiers, such as headers.
42
50
## Proposal
43
51
44
52
We propose introducing hash-based routing as a load balancing algorithm for use on a per-route basis to address the
45
- issues described in the earlier scenario .
53
+ issues described in the earlier use cases .
46
54
47
55
The approach leverages an HTTP header, which is associated with each incoming request and contains the specific
48
56
identifier. This one is used to compute a hash value, which will serve as the basis for routing decisions.
49
57
50
- In the previously mentioned scenario , the tenant ID acts as the identifier included in the header and serves as the
51
- basis for hash calculation. This hash value determines the appropriate application instance for each request, ensuring
58
+ In the previously mentioned use cases , the specific identifier included in the header can serve as the basis for hash
59
+ calculation. This hash value determines the appropriate application instance for each request, ensuring
52
60
that all requests with this identifier are consistently routed to the same instance or might be routed to another
53
61
instance when the instance is saturated. Consequently, the load balancing algorithm effectively directs requests for a
54
62
single tenant to a particular application instance, so that instance can minimize database connection overhead and
@@ -122,12 +130,12 @@ A possible presentation of deterministic handling can be a ring like:
122
130
123
131
- Gorouter MUST be extended to take a specific identifier via the request header
124
132
- Gorouter MUST implement hash calculation, based on the provided header
125
- - Gorouter SHOULD store the mapping between computed hash values and application instances locally to avoid
133
+ - Gorouter MAY store the mapping between computed hash values and application instances locally to avoid
126
134
expensive recalculations for each incoming request
127
135
- Gorouters SHOULD NOT implement a distributed shared cache
128
136
- Gorouter MUST assess the current number of in-flight requests across all application instances mapped to a
129
137
particular route to consider overload situations
130
- - Gorouter MUST update its local hash table following the registration or deregistration of an endpoint, ensuring
138
+ - Gorouter MAY update its local hash table following the registration or deregistration of an endpoint, ensuring
131
139
minimal rehashing
132
140
- Gorouter SHOULD NOT incur any performance hit when 0 apps use hash routing.
133
141
0 commit comments