Skip to content

Commit ef2e3fb

Browse files
agrawrohagrawroh
authored
geoip: add a new Network GeoIP filter (#42564)
## Description Today we have an [HTTP GeoLocation filter](https://www.envoyproxy.io/docs/envoy/latest/configuration/http/http_filters/geoip_filter) which could use providers like MaxMind to fetch the Geo details of the incoming traffic using the Client IP. We have mixed traffic where we also receive a lot of Postgres, Kafka and other protocols in addition to HTTP. We need a holistic approach to fetch the Geo details for all the incoming traffic and log it in access logs. This PR adds a new Network Geolocation filter which could use the same providers that HTTP use and save the Geo details associated with the incoming traffic using Filter State. Fix #42493 --- **Commit Message:** geoip: add a new Network GeoIP filter **Additional Description:** Adds a new Network Geolocation filter which could use the same providers that HTTP use and save the Geo details associated with the incoming traffic using Filter State. **Risk Level:** Low **Testing:** Added Unit & Integration Tests **Docs Changes:** Added **Release Notes:** Added --------- Signed-off-by: Rohit Agrawal <rohit.agrawal@databricks.com>
1 parent ef29fc1 commit ef2e3fb

File tree

29 files changed

+1572
-103
lines changed

29 files changed

+1572
-103
lines changed

‎CODEOWNERS‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -382,6 +382,8 @@ extensions/upstreams/tcp @ggreenway @mattklein123
382382
/*/extensions/health_check/event_sinks/file @botengyao @yanavlasov
383383
# IP Geolocation
384384
/*/extensions/filters/http/geoip @nezdolik @ravenblackx
385+
# IP Geolocation Network Filter
386+
/*/extensions/filters/network/geoip @agrawroh @nezdolik @ravenblackx @wbpcode
385387
/*/extensions/geoip_providers/common @nezdolik @ravenblackx
386388
# Maxmind geolocation provider
387389
/*/extensions/geoip_providers/maxmind @nezdolik @ravenblackx

‎api/BUILD‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -251,6 +251,7 @@ proto_library(
251251
"//envoy/extensions/filters/network/generic_proxy/matcher/v3:pkg",
252252
"//envoy/extensions/filters/network/generic_proxy/router/v3:pkg",
253253
"//envoy/extensions/filters/network/generic_proxy/v3:pkg",
254+
"//envoy/extensions/filters/network/geoip/v3:pkg",
254255
"//envoy/extensions/filters/network/http_connection_manager/v3:pkg",
255256
"//envoy/extensions/filters/network/local_ratelimit/v3:pkg",
256257
"//envoy/extensions/filters/network/mongo_proxy/v3:pkg",

‎api/envoy/extensions/filters/network/geoip/v3/BUILD‎

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
# DO NOT EDIT. This file is generated by tools/proto_format/proto_sync.py.
2+
3+
load("@envoy_api//bazel:api_build_system.bzl", "api_proto_package")
4+
5+
licenses(["notice"]) # Apache 2
6+
7+
api_proto_package(
8+
deps = [
9+
"//envoy/config/core/v3:pkg",
10+
"@com_github_cncf_xds//udpa/annotations:pkg",
11+
"@com_github_cncf_xds//xds/annotations/v3:pkg",
12+
],
13+
)

‎api/envoy/extensions/filters/network/geoip/v3/geoip.proto‎

Lines changed: 40 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,40 @@
1+
syntax = "proto3";
2+
3+
package envoy.extensions.filters.network.geoip.v3;
4+
5+
import "envoy/config/core/v3/extension.proto";
6+
7+
import "xds/annotations/v3/status.proto";
8+
9+
import "udpa/annotations/status.proto";
10+
import "validate/validate.proto";
11+
12+
option java_package = "io.envoyproxy.envoy.extensions.filters.network.geoip.v3";
13+
option java_outer_classname = "GeoipProto";
14+
option java_multiple_files = true;
15+
option go_package = "github.com/envoyproxy/go-control-plane/envoy/extensions/filters/network/geoip/v3;geoipv3";
16+
option (udpa.annotations.file_status).package_version_status = ACTIVE;
17+
option (xds.annotations.v3.file_status).work_in_progress = true;
18+
19+
// [#protodoc-title: Geoip]
20+
// Geoip :ref:`configuration overview <config_network_filters_geoip>`.
21+
// [#extension: envoy.filters.network.geoip]
22+
23+
// The network geolocation filter performs IP geolocation lookups on incoming connections
24+
// and stores the results in the connection's filter state under the well-known key
25+
// ``envoy.geoip``. The stored data is a ``GeoipInfo`` object that supports
26+
// serialization for access logging and field-level access.
27+
//
28+
// See :ref:`well known filter state <well_known_filter_state>` for details on accessing
29+
// the geolocation data.
30+
message Geoip {
31+
// The prefix to use when emitting statistics. This is useful when there are multiple
32+
// listeners configured with geoip filters, allowing stats to be grouped per listener.
33+
// For example, with ``stat_prefix: "listener_1."``, stats would be emitted as
34+
// ``listener_1.geoip.total``.
35+
string stat_prefix = 1;
36+
37+
// Geoip driver specific configuration which depends on the driver being instantiated.
38+
// [#extension-category: envoy.geoip_providers]
39+
config.core.v3.TypedExtensionConfig provider = 2 [(validate.rules).message = {required: true}];
40+
}

‎api/envoy/extensions/geoip_providers/common/v3/common.proto‎

Lines changed: 71 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -17,9 +17,13 @@ option (udpa.annotations.file_status).package_version_status = ACTIVE;
1717
// Common configuration shared across geolocation providers.
1818

1919
message CommonGeoipProviderConfig {
20-
// The set of geolocation headers to add to the request. If any of the configured headers is present
21-
// in the incoming request, it will be overridden by the :ref:`GeoIP filter <config_http_filters_geoip>`.
20+
// The set of geolocation headers to add to request. If any of the configured headers is present
21+
// in the incoming request, it will be overridden by the :ref:`HTTP GeoIP filter <config_http_filters_geoip>`.
2222
// [#next-free-field: 13]
23+
//
24+
// .. attention::
25+
// This field is deprecated in favor of :ref:`geo_field_keys
26+
// <envoy_v3_api_field_extensions.geoip_providers.common.v3.CommonGeoipProviderConfig.geo_field_keys>`.
2327
message GeolocationHeadersToAdd {
2428
// If set, the header will be used to populate the country ISO code associated with the IP address.
2529
string country = 1
@@ -80,6 +84,69 @@ message CommonGeoipProviderConfig {
8084
[(validate.rules).string = {well_known_regex: HTTP_HEADER_NAME ignore_empty: true}];
8185
}
8286

83-
// Configuration for geolocation headers to add to the request.
84-
GeolocationHeadersToAdd geo_headers_to_add = 1 [(validate.rules).message = {required: true}];
87+
// The set of geolocation field keys to use for storing lookup results.
88+
// These keys define how the geolocation lookup results will be stored. The actual storage
89+
// mechanism depends on the filter using the provider:
90+
//
91+
// - The :ref:`HTTP GeoIP filter <config_http_filters_geoip>` stores results as HTTP request headers.
92+
// - The :ref:`Network GeoIP filter <config_network_filters_geoip>` stores results in the
93+
// connection's filter state under the well-known key ``envoy.geoip``.
94+
//
95+
// [#next-free-field: 12]
96+
message GeolocationFieldKeys {
97+
// If set, the key will be used to populate the country ISO code associated with the IP address.
98+
string country = 1;
99+
100+
// If set, the key will be used to populate the city associated with the IP address.
101+
string city = 2;
102+
103+
// If set, the key will be used to populate the region ISO code associated with the IP address.
104+
// The least specific subdivision will be selected as the region value.
105+
string region = 3;
106+
107+
// If set, the key will be used to populate the ASN associated with the IP address.
108+
string asn = 4;
109+
110+
// If set, the IP address will be checked if it belongs to any type of anonymization network
111+
// (e.g., VPN, public proxy). The result will be stored with this key. Value will be set to
112+
// either ``true`` or ``false`` depending on the check result.
113+
string anon = 5;
114+
115+
// If set, the IP address will be checked if it belongs to a VPN and the result will be stored
116+
// with this key. Value will be set to either ``true`` or ``false`` depending on the check result.
117+
string anon_vpn = 6;
118+
119+
// If set, the IP address will be checked if it belongs to a hosting provider and the result
120+
// will be stored with this key. Value will be set to either ``true`` or ``false`` depending on
121+
// the check result.
122+
string anon_hosting = 7;
123+
124+
// If set, the IP address will be checked if it belongs to a TOR exit node and the result will
125+
// be stored with this key. Value will be set to either ``true`` or ``false`` depending on the
126+
// check result.
127+
string anon_tor = 8;
128+
129+
// If set, the IP address will be checked if it belongs to a public proxy and the result will
130+
// be stored with this key. Value will be set to either ``true`` or ``false`` depending on the
131+
// check result.
132+
string anon_proxy = 9;
133+
134+
// If set, the key will be used to populate the ISP associated with the IP address.
135+
string isp = 10;
136+
137+
// If set, the IP address will be checked if it belongs to the ISP named iCloud Private Relay
138+
// and the result will be stored with this key. Value will be set to either ``true`` or ``false``
139+
// depending on the check result.
140+
string apple_private_relay = 11;
141+
}
142+
143+
// Configuration for geolocation headers to add to HTTP requests.
144+
// This field is deprecated in favor of ``geo_field_keys``. If both are set, ``geo_field_keys``
145+
// takes precedence.
146+
GeolocationHeadersToAdd geo_headers_to_add = 1
147+
[deprecated = true, (envoy.annotations.deprecated_at_minor_version) = "3.0"];
148+
149+
// Configuration for geolocation field keys.
150+
// At least one of ``geo_headers_to_add`` or ``geo_field_keys`` must be set.
151+
GeolocationFieldKeys geo_field_keys = 3;
85152
}

‎api/versioning/BUILD‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -190,6 +190,7 @@ proto_library(
190190
"//envoy/extensions/filters/network/generic_proxy/matcher/v3:pkg",
191191
"//envoy/extensions/filters/network/generic_proxy/router/v3:pkg",
192192
"//envoy/extensions/filters/network/generic_proxy/v3:pkg",
193+
"//envoy/extensions/filters/network/geoip/v3:pkg",
193194
"//envoy/extensions/filters/network/http_connection_manager/v3:pkg",
194195
"//envoy/extensions/filters/network/local_ratelimit/v3:pkg",
195196
"//envoy/extensions/filters/network/mongo_proxy/v3:pkg",

‎changelogs/current.yaml‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -231,6 +231,11 @@ removed_config_or_runtime:
231231
Removed runtime guard ``envoy.reloadable_features.router_filter_resetall_on_local_reply`` and legacy code paths.
232232
233233
new_features:
234+
- area: network_filter
235+
change: |
236+
Added :ref:`geoip network filter <config_network_filters_geoip>` to perform geolocation lookups
237+
at the network layer and store results in filter state. This enables geolocation data to be
238+
used for access logging, routing, and other purposes without requiring HTTP traffic.
234239
- area: oauth2
235240
change: |
236241
Added support for configuring cookie path in the OAuth2 filter. The :ref:`path

‎docs/BUILD‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -23,6 +23,7 @@ filegroup(
2323
"root/configuration/http/http_filters/_include/checksum_filter.yaml",
2424
"root/configuration/http/http_filters/_include/dns-cache-circuit-breaker.yaml",
2525
"root/configuration/http/http_filters/_include/geoip-filter.yaml",
26+
"root/configuration/listeners/network_filters/_include/geoip-network-filter.yaml",
2627
"root/configuration/http/http_filters/_include/http-cache-configuration-fs.yaml",
2728
"root/configuration/http/http_filters/_include/http-cache-v2-configuration-fs.yaml",
2829
"root/configuration/other_features/_include/dlb.yaml",

‎docs/root/configuration/advanced/well_known_filter_state.rst‎

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -73,6 +73,23 @@ The following lists the filter state object keys used by the Envoy extensions to
7373
<envoy_v3_api_field_service.ratelimit.v3.RateLimitRequest.hits_addend>` override on a per-route basis.
7474
Accepts a number string as a constructor.
7575

76+
``envoy.geoip``
77+
:ref:`Network GeoIP filter <config_network_filters_geoip>` stores geolocation lookup results
78+
in this filter state object. The object contains fields for geographic data such as country,
79+
city, region, and ASN. Supports serialization for access logging and field-level access. Fields:
80+
81+
* ``country``: ISO country code;
82+
* ``city``: city name;
83+
* ``region``: ISO region code;
84+
* ``asn``: autonomous system number;
85+
* ``anon``: anonymization network check result (``true`` or ``false``);
86+
* ``anon_vpn``: VPN check result (``true`` or ``false``);
87+
* ``anon_hosting``: hosting provider check result (``true`` or ``false``);
88+
* ``anon_tor``: TOR exit node check result (``true`` or ``false``);
89+
* ``anon_proxy``: public proxy check result (``true`` or ``false``);
90+
* ``isp``: ISP name;
91+
* ``apple_private_relay``: iCloud Private Relay check result (``true`` or ``false``).
92+
7693
``envoy.network.network_namespace``
7794
Contains the value of the downstream connection's Linux network namespace if it differs from the default.
7895

‎docs/root/configuration/listeners/network_filters/_include/geoip-network-filter.yaml‎

Lines changed: 49 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,49 @@
1+
static_resources:
2+
listeners:
3+
- name: listener_0
4+
address:
5+
socket_address:
6+
address: 0.0.0.0
7+
port_value: 10000
8+
filter_chains:
9+
- filters:
10+
- name: envoy.filters.network.geoip
11+
typed_config:
12+
"@type": type.googleapis.com/envoy.extensions.filters.network.geoip.v3.Geoip
13+
provider:
14+
name: envoy.geoip_providers.maxmind
15+
typed_config:
16+
"@type": type.googleapis.com/envoy.extensions.geoip_providers.maxmind.v3.MaxMindConfig
17+
common_provider_config:
18+
geo_field_keys:
19+
country: "country"
20+
city: "city"
21+
region: "region"
22+
asn: "asn"
23+
city_db_path: "/etc/GeoLite2-City.mmdb"
24+
asn_db_path: "/etc/GeoLite2-ASN.mmdb"
25+
- name: envoy.filters.network.tcp_proxy
26+
typed_config:
27+
"@type": type.googleapis.com/envoy.extensions.filters.network.tcp_proxy.v3.TcpProxy
28+
stat_prefix: tcp
29+
cluster: cluster_0
30+
access_log:
31+
- name: envoy.access_loggers.file
32+
typed_config:
33+
"@type": type.googleapis.com/envoy.extensions.access_loggers.file.v3.FileAccessLog
34+
path: /dev/stdout
35+
log_format:
36+
text_format_source:
37+
inline_string: "[%START_TIME%] %DOWNSTREAM_REMOTE_ADDRESS% geo=%FILTER_STATE(envoy.geoip:PLAIN)%\n"
38+
clusters:
39+
- name: cluster_0
40+
type: STATIC
41+
load_assignment:
42+
cluster_name: cluster_0
43+
endpoints:
44+
- lb_endpoints:
45+
- endpoint:
46+
address:
47+
socket_address:
48+
address: 127.0.0.1
49+
port_value: 10001

0 commit comments

Comments
 (0)