Skip to content

Commit 4a7c5c4

Browse files
aa1exaa1ex
authored
Feat/multi filter chain support (#207)
* feat: support multiple filter chains in TCP proxy listeners * test: add Kafka-style multi-broker SNI routing e2e test * test: add unit tests for multiple filter chains support * feat: add Kafka Broker Filter support - Delegate cluster extraction to unified implementation - Add unit and e2e tests for multi-filter chains - Use envoy contrib image for Kafka filter support * fix: extract "true" string to constant (goconst) * docs: add Kafka Broker Filter documentation * fix: correct protobuf serialization and optimize cluster extraction - Fix json vs protojson: use anypb.UnmarshalNew + protojson.Marshal for *anypb.Any types instead of direct json.Unmarshal on wire bytes - Remove unused pool allocations in extractClustersFromFilter - Remove intermediate slice copies, keep only final copy - Add edge case unit tests for empty filters, nil config, weighted clusters * feat: add debug logging for unknown filter types * fix: use UseProtoNames for protojson to fix field name matching * fix: handle false positives in generic cluster extraction * helm new package
1 parent acadfb4 commit 4a7c5c4

38 files changed

+2870
-330
lines changed

‎docs/kafka-broker-filter.md‎

Lines changed: 169 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,169 @@
1+
# Kafka Broker Filter Support
2+
3+
This document describes how to proxy Kafka traffic through Envoy using the Kafka Broker Filter with automatic broker address rewriting.
4+
5+
## Overview
6+
7+
The Envoy Kafka Broker Filter enables transparent Kafka proxying by intercepting Kafka protocol messages and rewriting broker addresses in metadata responses. This allows external clients to connect to Kafka through Envoy without modifying Kafka's `advertised.listeners` configuration.
8+
9+
## Requirements
10+
11+
| Requirement | Value | Notes |
12+
|-------------|-------|-------|
13+
| **Kafka version** | **≤ 3.8.0** | Kafka 4.0+ is NOT supported |
14+
| **Envoy image** | `envoyproxy/envoy-contrib` | Standard `envoy` image does not include the filter |
15+
| **Protocol** | Plaintext | TLS not supported with address rewriting |
16+
17+
> **Important**: The Kafka Broker Filter is **experimental** and under active development. Configuration structures may change. Thorough testing is recommended before production use.
18+
19+
## Architecture
20+
21+
Each Kafka broker requires a dedicated port on Envoy:
22+
23+
```
24+
Client ──► Envoy :19092 ──► kafka-broker-0:9092
25+
──► Envoy :19093 ──► kafka-broker-1:9092
26+
──► Envoy :19094 ──► kafka-broker-2:9092
27+
```
28+
29+
The filter rewrites broker addresses in Kafka protocol responses (Metadata, FindCoordinator, etc.) so clients receive Envoy addresses instead of internal broker addresses.
30+
31+
## Configuration
32+
33+
### Listener
34+
35+
Create a Listener for each broker with the Kafka Broker Filter **before** TCP Proxy:
36+
37+
```yaml
38+
apiVersion: envoy.kaasops.io/v1alpha1
39+
kind: Listener
40+
metadata:
41+
name: kafka-broker-0-listener
42+
spec:
43+
name: kafka-broker-0-listener
44+
address:
45+
socket_address:
46+
address: 0.0.0.0
47+
port_value: 19092
48+
filter_chains:
49+
- filters:
50+
- name: envoy.filters.network.kafka_broker
51+
typed_config:
52+
"@type": type.googleapis.com/envoy.extensions.filters.network.kafka_broker.v3.KafkaBroker
53+
stat_prefix: kafka-broker-0
54+
id_based_broker_address_rewrite_spec:
55+
rules:
56+
- id: 0
57+
host: envoy.example.com
58+
port: 19092
59+
- id: 1
60+
host: envoy.example.com
61+
port: 19093
62+
- id: 2
63+
host: envoy.example.com
64+
port: 19094
65+
- name: envoy.filters.network.tcp_proxy
66+
typed_config:
67+
"@type": type.googleapis.com/envoy.extensions.filters.network.tcp_proxy.v3.TcpProxy
68+
stat_prefix: kafka-broker-0
69+
cluster: kafka-broker-0
70+
```
71+
72+
**Key points**:
73+
- `id_based_broker_address_rewrite_spec.rules` must contain ALL brokers in the cluster
74+
- `id` corresponds to Kafka's `broker.id` or `node.id`
75+
- `host` and `port` are the external Envoy addresses that clients will use
76+
- The same rules should be replicated in all broker listeners
77+
78+
### Cluster
79+
80+
```yaml
81+
apiVersion: envoy.kaasops.io/v1alpha1
82+
kind: Cluster
83+
metadata:
84+
name: kafka-broker-0
85+
spec:
86+
name: kafka-broker-0
87+
connect_timeout: 5s
88+
type: STRICT_DNS
89+
load_assignment:
90+
cluster_name: kafka-broker-0
91+
endpoints:
92+
- lb_endpoints:
93+
- endpoint:
94+
address:
95+
socket_address:
96+
address: kafka-broker-0.internal
97+
port_value: 9092
98+
```
99+
100+
### VirtualService
101+
102+
```yaml
103+
apiVersion: envoy.kaasops.io/v1alpha1
104+
kind: VirtualService
105+
metadata:
106+
name: kafka-broker-0
107+
annotations:
108+
envoy.kaasops.io/node-id: <envoy-node-id>
109+
spec:
110+
listener:
111+
name: kafka-broker-0-listener
112+
```
113+
114+
## Verification
115+
116+
### Check address rewriting
117+
118+
```bash
119+
kafka-broker-api-versions.sh --bootstrap-server <envoy>:19092
120+
```
121+
122+
**Expected output** (Envoy addresses):
123+
```
124+
envoy.example.com:19092 (id: 0 rack: null) -> ...
125+
envoy.example.com:19093 (id: 1 rack: null) -> ...
126+
envoy.example.com:19094 (id: 2 rack: null) -> ...
127+
```
128+
129+
**Problem** (internal addresses visible):
130+
```
131+
kafka-broker-0.internal:9092 (id: 0 rack: null) -> ...
132+
```
133+
134+
### Check metrics
135+
136+
```bash
137+
curl -s http://<envoy>:19000/stats | grep "kafka.*unknown"
138+
```
139+
140+
If `request.unknown > 0`, the Kafka version is incompatible.
141+
142+
## Troubleshooting
143+
144+
| Symptom | Cause | Solution |
145+
|---------|-------|----------|
146+
| `Bootstrap broker disconnected` | Kafka 4.0+ or standard Envoy image | Use Kafka ≤3.8.0, `envoy-contrib` image |
147+
| `request.unknown > 0` | Incompatible Kafka version | Downgrade to Kafka 3.8.0 |
148+
| Internal addresses in metadata | Wrong broker ID in rules | Verify broker IDs match |
149+
| Filter not applied | Filter order | `kafka_broker` must be before `tcp_proxy` |
150+
| Consumer TimeoutException | Slow FindCoordinator via proxy | Increase client timeout |
151+
152+
## Limitations
153+
154+
1. **Kafka 4.0+** — not supported due to protocol changes
155+
2. **TLS** — not supported with address rewriting
156+
3. **Filter status** — experimental in Envoy
157+
4. **Scaling** — adding brokers requires updating rules in ALL listeners
158+
159+
## Alternative: SNI-based Routing
160+
161+
For TLS support or lower latency requirements, consider SNI-based routing instead. This approach:
162+
- Supports TLS
163+
- Has minimal latency overhead (L4 proxy only)
164+
- Requires configuring `advertised.listeners` on Kafka brokers
165+
166+
## References
167+
168+
- [Envoy Kafka Broker Filter Documentation](https://www.envoyproxy.io/docs/envoy/latest/configuration/listeners/network_filters/kafka_broker_filter)
169+
- [Proxying Kafka with Envoy without changing advertised.listeners](https://adam-kotwasinski.medium.com/proxying-kafka-with-envoy-without-changing-advertised-listeners-by-using-rewrite-rules-387792cce066)

‎helm/charts/envoy-xds-controller/Chart.yaml‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -15,13 +15,13 @@ type: application
1515
# This is the chart version. This version number should be incremented each time you make changes
1616
# to the chart and its templates, including the app version.
1717
# Versions are expected to follow Semantic Versioning (https://semver.org/)
18-
version: "0.83.0"
18+
version: "0.84.0"
1919

2020
# This is the version number of the application being deployed. This version number should be
2121
# incremented each time you make changes to the application. Versions are not expected to
2222
# follow Semantic Versioning. They should reflect the version the application is using.
2323
# It is recommended to use it with quotes.
24-
appVersion: "v0.15.0"
24+
appVersion: "v0.16.0"
2525

2626
home: https://github.com/kaasops/envoy-xds-controller
2727
sources:

0 commit comments

Comments
 (0)