|
2 | 2 | title: 'Replica-aware routing' |
3 | 3 | slug: /manage/replica-aware-routing |
4 | 4 | description: 'How to use Replica-aware routing to increase cache re-use' |
5 | | -keywords: ['cloud', 'sticky endpoints', 'sticky', 'endpoints', 'sticky routing', 'routing', 'replica aware routing'] |
| 5 | +keywords: ['cloud', 'sticky endpoints', 'sticky', 'endpoints', 'sticky routing', 'routing', 'replica aware routing', 'session_id'] |
6 | 6 | doc_type: 'guide' |
| 7 | +unlisted: true |
7 | 8 | --- |
8 | 9 |
|
9 | 10 | import PrivatePreviewBadge from '@theme/badges/PrivatePreviewBadge'; |
10 | 11 |
|
11 | 12 | <PrivatePreviewBadge/> |
12 | 13 |
|
13 | | -Replica-aware routing (also known as sticky sessions, sticky routing, or session affinity) utilizes [Envoy proxy's ring hash load balancing](https://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/upstream/load_balancing/load_balancers#ring-hash). The main purpose of replica-aware routing is to increase the chance of cache reuse. It doesn't guarantee isolation. |
| 14 | +Replica-aware routing (also known as sticky sessions, sticky routing, or session affinity) increases the chance of cache reuse by consistently routing related requests to the same ClickHouse replica. It is best-effort and does not guarantee isolation. |
14 | 15 |
|
15 | | -When enabling replica-aware routing for a service, we allow a wildcard subdomain on top of the service hostname. For a service with the host name `abcxyz123.us-west-2.aws.clickhouse.cloud`, you can use any hostname which matches `*.sticky.abcxyz123.us-west-2.aws.clickhouse.cloud` to visit the service: |
| 16 | +:::warning Protocol support — HTTP/HTTPS only |
| 17 | +Replica-aware routing is applied at the proxy layer over the **HTTP/HTTPS interface**, using the `session_id` query parameter (see below). It is **not available over the native protocol** (native port, e.g. the `clickhouse-go` driver in its default native mode). If you need sticky routing from a native-protocol client, connect that workload over the HTTP interface instead — for `clickhouse-go` (v2), set `Protocol: clickhouse.HTTP` on the connection. |
| 18 | +::: |
16 | 19 |
|
17 | | -|Example hostnames| |
18 | | -|---| |
19 | | -|`aaa.sticky.abcxyz123.us-west-2.aws.clickhouse.cloud`| |
20 | | -|`000.sticky.abcxyz123.us-west-2.aws.clickhouse.cloud`| |
21 | | -|`clickhouse-is-the-best.sticky.abcxyz123.us-west-2.aws.clickhouse.cloud`| |
| 20 | +## HTTP-based routing (session_id) {#http-based-routing} |
22 | 21 |
|
23 | | -When Envoy receives a hostname that matches such a pattern, it will compute the routing hash based on the hostname and find the corresponding ClickHouse server on the hash ring based on the computed hash. Assuming that there is no ongoing change to the service (e.g. server restarts, scale out/in), Envoy will always choose the same ClickHouse server to connect to. |
| 22 | +To pin a workload to a replica, set the `session_id` query parameter on the HTTPS interface. The proxy parses the request and uses consistent hashing on the `session_id` to select a replica, so all requests sharing the same `session_id` are routed to the same server — until the cluster topology changes. |
24 | 23 |
|
25 | | -Note the original hostname will still use `LEAST_CONNECTION` load balancing, which is the default routing algorithm. |
| 24 | +```bash |
| 25 | +echo 'SELECT hostName()' | curl \ |
| 26 | + -H 'X-ClickHouse-User: default' -H 'X-ClickHouse-Key: <password>' \ |
| 27 | + 'https://<host>:8443/?session_id=my-workload-1' -d @- |
| 28 | +``` |
26 | 29 |
|
27 | | -## Limitations of Replica-aware routing {#limitations-of-replica-aware-routing} |
| 30 | +Every request carrying `session_id=my-workload-1` lands on the same replica. A different `session_id` value hashes independently and may land on the same or a different replica — the mapping is consistent, but you do not choose *which* replica a given value maps to. |
| 31 | + |
| 32 | +## Subdomain-based routing (deprecated) {#subdomain-based-routing-deprecated} |
| 33 | + |
| 34 | +:::danger Deprecated |
| 35 | +The subdomain-based mechanism below is **deprecated** and is no longer enabled on new services. It does not scale (each sticky endpoint requires its own TLS certificate). Use the [HTTP-based `session_id` method](#http-based-routing) instead. |
| 36 | +::: |
| 37 | + |
| 38 | +Previously, enabling replica-aware routing allowed a wildcard subdomain on top of the service hostname. For a service with the host name `abcxyz123.us-west-2.aws.clickhouse.cloud`, any hostname matching `*.sticky.abcxyz123.us-west-2.aws.clickhouse.cloud` (e.g. `aaa.sticky.abcxyz123.us-west-2.aws.clickhouse.cloud`) was hashed by Envoy to a consistent replica. The original hostname continued to use `LEAST_CONNECTION` load balancing, the default routing algorithm. |
| 39 | + |
| 40 | +## Limitations of replica-aware routing {#limitations-of-replica-aware-routing} |
28 | 41 |
|
29 | 42 | ### Replica-aware routing doesn't guarantee isolation {#replica-aware-routing-does-not-guarantee-isolation} |
30 | 43 |
|
31 | | -Any disruption to the service, e.g. server pod restarts (due to any reason like a version upgrade, crash, vertical scaling up, etc.), server scaled out / in, will cause a disruption to the routing hash ring. This will cause connections with the same hostname to land on a different server pod. |
| 44 | +Any disruption to the service — server pod restarts (version upgrade, crash, vertical scaling), or scale out / in — changes the routing hash ring. This causes requests sharing the same `session_id` to land on a different server pod. |
32 | 45 |
|
33 | 46 | ### Replica-aware routing doesn't work out of the box with private link {#replica-aware-routing-does-not-work-out-of-the-box-with-private-link} |
34 | 47 |
|
35 | | -Customers need to manually add a DNS entry to make name resolution work for the new hostname pattern. It is possible that this can cause imbalance in the server load if customers use it incorrectly. |
| 48 | +When using the deprecated subdomain method, customers need to manually add a DNS entry to make name resolution work for the new hostname pattern. It is possible that this can cause imbalance in the server load if customers use it incorrectly. |
| 49 | + |
| 50 | +### Replica-aware routing requires the HTTP protocol {#replica-aware-routing-requires-http} |
| 51 | + |
| 52 | +Sticky routing is keyed on the `session_id` query parameter, which only exists on the HTTP/HTTPS interface. The native binary protocol carries no such parameter for the proxy to hash on, so replica-aware routing is not available over the native protocol. Native-protocol clients must move the relevant workload to the HTTP interface to use this feature. |
36 | 53 |
|
37 | 54 | ## Configuring replica-aware routing {#configuring-replica-aware-routing} |
38 | 55 |
|
39 | | -To enable Replica-aware routing, please contact [our support team](https://clickhouse.com/support/program). |
| 56 | +To enable replica-aware routing, please contact [our support team](https://clickhouse.com/support/program). |
0 commit comments