Skip to content

Add page Block Services with ExternalIPs #52631

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
+100 −0
Open

Add page Block Services with ExternalIPs #52631

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
100 changes: 100 additions & 0 deletions content/en/docs/tasks/network/block-services-with-externalips.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,100 @@
---
reviewers:
- thockin
- danwinship
- aojea
min-kubernetes-server-version: v1.30
title: Block Services with ExternalIPs
content_type: task
---

<!-- overview -->

This document shares how to control how Services with ExternalIPs are managed within your cluster.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
This document shares how to control how Services with ExternalIPs are managed within your cluster.
This document explains a way to control how {{< glossary_tooltip text="Services" term_id="service" >}} with external IP address(es) are managed within your cluster.


An ExternalIP is a powerful tool that could be used for [malicious intent](https://www.cvedetails.com/cve/CVE-2020-8554/).
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
An ExternalIP is a powerful tool that could be used for [malicious intent](https://www.cvedetails.com/cve/CVE-2020-8554/).
The ability to [set an external IP address for a Service](/docs/concepts/services-networking/service/#external-ips) could be misused as a way for
an otherwise unprivileged user to intercept traffic associated with that IP address.
See
[CVE-2020-8554](https://www.cvedetails.com/cve/CVE-2020-8554/) for more details.

?

nit: also update https://k8s.io/docs/concepts/services-networking/service/#external-ips to link to this new task page.

Aside: if we fix #46623 (which will create a page for each known Kubernetes vulnerability), we can link to that page instead.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The problem is that it's not supposed to be a powerful tool. The power was accidental, and that's the problem.

Some background to incorporate:

  • The feature was originally intended to be an alternative to LoadBalancer Services for non-cloud platforms.
  • It was added in the very early days of Kubernetes when there was no real concept of untrusted users, and the API is designed in a way that makes it difficult to lock down when there are untrusted users.
  • Ordinary (non-admin) users generally don't have the ability to use the feature in the intended way (since this usually requires adding the externalIP to an interface on the node) but they do have the ability to use it to exploit the CVE!
  • The feature is enabled and unrestricted by default for backward-compatibility reasons, but we recommend people disable or restrict it in clusters that don't need it.


Any user who can create a Service with ExternalIPs could:

- intercept other users' outbound traffic to arbitrary IPs.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
- intercept other users' outbound traffic to arbitrary IPs.
- intercept other users' outbound traffic to arbitrary cluster-external IPs.

- could (non-deterministically) steal other users' inbound traffic to their own ExternalIPs.
Comment on lines +13 to +20
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: avoid using UpperCamelCase for externalIPs; in our style guide, we use otherwise unstyled UpperCamelCase for API kinds (eg StatefulSet, Service).

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need some (style-guide-approved) way of being explicit that we are referring to the externalIPs feature and not just the vague concept of "external IPs", which can easily be misinterpreted. (eg if this point was rewritten as "to their own external IPs", it would be easy to assume it meant cluster-external IPs as in the previous bullet point).

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use the field capitalization: externalIPs. You can also come up with new jargon if you write it in italics. Example from elsewhere: adding a taint to a Node lets…
However, I couldn't think of any jargon that would work.

For example:

A Kubernetes Service with externalIPs set lets you expose Pods running in your cluster to the outside network, without using a Gateway, an Ingress, or a load balancer integration. Using a Gateway or a Service of type: LoadBalancer is usually a better way to expose Pods and their listening ports. In fact, using externalIPs is rare in platforms built to use Kubernetes architecture.

To use .spec.externalIPs with a Service, either you as a cluster administrator need a way to configure additional network interfaces on the relevant node(s), or you need to use a controller that automates doing the same thing.

The problem with .spec.externalIPs is that, unfortunately, someone with access to create this kind of Service can abuse…

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

However, I couldn't think of any jargon that would work.

Yeah, we don't have any standard way of referring to this unambiguously other than the field name. (Probably in part because we don't refer to the feature very often, because of its problems.)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
- could (non-deterministically) steal other users' inbound traffic to their own ExternalIPs.
- (non-deterministically) steal other users' inbound traffic to their own ExternalIPs.

(You already have a "could" introducing the list.)


## {{% heading "prerequisites" %}}

{{< include "task-tutorial-prereqs.md" >}}

{{< version-check >}}

<!-- steps -->

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

somewhere in this doc you should also meantion that you can disable the feature entirely by enabling the DenyServiceExternalIPs admission controller via kube-apiserver's --enable-admission-plugins flag.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tabs might work well here.

## Kubernetes Service ExternalIP Policies
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
## Kubernetes Service ExternalIP Policies
## Service external IP address policies for Kubernetes


Cluster administrators can implement policies to control the creation and modification of Services with ExternalIPs within the cluster. This allows for centralized management of the allowed ExternalIPs used for Services and helps prevent unintended or conflicting configurations. Kubernetes provides mechanisms like Validating Admission Policies to enforce these rules.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Cluster administrators can implement policies to control the creation and modification of Services with ExternalIPs within the cluster. This allows for centralized management of the allowed ExternalIPs used for Services and helps prevent unintended or conflicting configurations. Kubernetes provides mechanisms like Validating Admission Policies to enforce these rules.
As a cluster administrator, you can implement policies to control the creation and modification of Services with external IP addresses within the cluster.
This allows for centralized management of the allowed external IP addresses that can be used for Services,
and helps prevent unintended or conflicting configurations.
Kubernetes provides mechanisms such as [ValidatingAdmissionPolicies](/docs/reference/access-authn-authz/validating-admission-policy/) that
you can use to enforce these rules.

nit: I would also move this paragraph to the introduction, since the page is specifically about solving this with ValidatingAdmissionPolicy.

Copy link
Contributor

@danwinship danwinship Oct 6, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would also move this paragraph to the introduction, since the page is specifically about solving this with ValidatingAdmissionPolicy.

(If you add a section mentioning the admission controller (which I'm pretty sure we don't document anywhere else other than the list-of-all-admission-controllers) then that wouldn't apply.)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In that case, reword it and move the reworded thing.


### Allowing only specific ExternalIPs within a certain IP range to be created
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
### Allowing only specific ExternalIPs within a certain IP range to be created
## Restrict Service external IP addresses to permitted address ranges


The following example allows an administrator to restrict the allowed IP address range(s) of any new or updated Service:

```yaml
---
apiVersion: admissionregistration.k8s.io/v1
kind: ValidatingAdmissionPolicy
metadata:
name: "allow-specific-externalips"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider adding a kubernetes.io/description annotation; I see that as a good practice.

spec:
failurePolicy: Fail
matchConstraints:
resourceRules:
- apiGroups: [""]
apiVersions: ["v1"]
operations: ["CREATE", "UPDATE"]
resources: ["services"]
variables:
- name: allowed
expression: "['192.0.2.0/24', '2001:db8::/64']"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Needs a comment: change this to your actual allowed IP address range.

validations:
- expression: |
!has(object.spec.externalIPs) ||
object.spec.externalIPs.all(ip, variables.allowed.exists(cidr, cidr(cidr).containsIP(ip)))
message: "All externalIPs must be within the allowed CIDR ranges."
---
apiVersion: admissionregistration.k8s.io/v1
kind: ValidatingAdmissionPolicyBinding
metadata:
name: "allow-specific-externalips-binding"
spec:
policyName: "allow-specific-externalips"
validationActions: [Deny, Audit]
```

### Restricting which users/groups may create/update Services with ExternalIPs
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
### Restricting which users/groups may create/update Services with ExternalIPs
## Restrict which users or groups may specify external IP addresses for Services


```yaml
---
apiVersion: admissionregistration.k8s.io/v1
kind: ValidatingAdmissionPolicy
metadata:
name: "allow-specific-users-to-manage-externalips"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider adding a kubernetes.io/description annotation; I see that as a good practice.

spec:
failurePolicy: Fail
matchConstraints:
resourceRules:
- apiGroups: [""]
apiVersions: ["v1"]
operations: ["CREATE", "UPDATE"]
resources: ["services"]
validations:
- expression: |
!has(object.spec.externalIPs) ||
request.userInfo.username == "myuser" ||
request.userInfo.groups.exists(g, g in ["system:masters", "net-admins"])
message: "Only user 'myuser' or members of groups 'system:masters' and 'net-admins' can assign externalIPs."
---
apiVersion: admissionregistration.k8s.io/v1
kind: ValidatingAdmissionPolicyBinding
metadata:
name: "allow-specific-users-binding"
spec:
policyName: "allow-specific-users-to-manage-externalips"
validationActions: [Deny, Audit]
```
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should get someone in sig-auth to review your VAPs and make sure they're correct too

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SIG Security are also fine folks to ask.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

that's probably what I meant 😬

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You need to be clear about what these policies do and don't protect against.

The first policy protects against "users can intercept traffic to arbitrary cluster-external IPs", but it doesn't prevent "users can intercept each others' ExternalIPs".

The second policy doesn't protect against either problem; it just assumes that you're restricting the feature to users who you trust to not (intentionally or accidentally) exploit the CVE.

Does CEL let you use maps in variables? It would be cool to have a policy that says "service account X is allowed to use these specific externalIPs, and service account Y is allowed to use these specific externalIPs (which are different from X's)". That would block both problems.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can use parameters in bindings, which gives you a way to do mappings I think (not tried it).