Skip to content

Commit 4309ba9

Browse files
robmryrobmry
committed
Describe migration from iptables to experimental nftables
Adds engine/network/firewall-nftables.md Signed-off-by: Rob Murray <[email protected]>
1 parent f7e847c commit 4309ba9

File tree

2 files changed

+286
-3
lines changed

2 files changed

+286
-3
lines changed

‎content/manuals/engine/network/firewall-nftables.md

Lines changed: 272 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,272 @@
1+
---
2+
title: Docker with nftables
3+
weight: 10
4+
description: How Docker works with nftables
5+
keywords: network, nftables, firewall
6+
---
7+
8+
> [!WARNING]
9+
>
10+
> Support for nftables is experimental, configuration options, behavior and
11+
> implementation may all change in future releases.
12+
> The rules for overlay networks have not yet been migrated from iptables.
13+
> So, nftables cannot be enabled when the daemon has Swarm enabled.
14+
15+
To use nftables instead of iptables, use Docker Engine option
16+
`--firewall-backend=nftables` on its command line, or `"firewall-backend": "nftables"`
17+
in its configuration file. You may also need to modify IP forwarding configuration
18+
on the host, and migrate rules from the iptables `DOCKER-USER` chain, see
19+
[migrating from iptables to nftables](#migrating-from-iptables-to-nftables).
20+
21+
For bridge networks, Docker creates nftables rules in the host's network
22+
namespace. For bridge and other network types, nftables rules for DNS are
23+
also created in the container's network namespace.
24+
25+
Creation of nftables rules can be disabled using daemon options `iptables`
26+
and `ip6tables`. _These options apply to both iptables and nftables._
27+
See [Prevent Docker from manipulating firewall rules](packet-filtering-firewalls.md#prevent-docker-from-manipulating-firewall-rules).
28+
However, this is not recommended for most users as it will likely break
29+
container networking.
30+
31+
## Docker's nftables tables
32+
33+
For bridge networks, Docker creates two tables, `ip docker-bridges` and
34+
`ip6 docker-bridges`.
35+
36+
Each table contains a number of [base chains](https://wiki.nftables.org/wiki-nftables/index.php/Configuring_chains#Adding_base_chains),
37+
and further chains are added for each bridge network. The moby project
38+
has some [internal documentation](https://github.com/moby/moby/blob/master/integration/network/bridge/nftablesdoc/index.md)
39+
describing its nftables, and how they depend on network and container
40+
configuration. But, the tables and their rules are likely to change between
41+
Docker Engine releases.
42+
43+
Do not modify Docker's tables directly as the modifications are likely to
44+
be lost, Docker expects to have full ownership of its tables.
45+
46+
> [!NOTE]
47+
>
48+
> Because iptables has a fixed set of chains, equivalent to nftables base
49+
> chains, all rules are included in those chains. The `DOCKER-USER` chain
50+
> is supplied as a way to insert rules into the `filter` table's `FORWARD`
51+
> chain, to run before Docker's rules.
52+
> In Docker's nftables implementation, there is no `DOCKER-USER` chain.
53+
> Instead, rules can be added in separate tables, with base chains that
54+
> have the same types and hook points as Docker's base chains. If necessary,
55+
> [base chain priority](https://wiki.nftables.org/wiki-nftables/index.php/Configuring_chains#Base_chain_priority)
56+
> can be used to tell nftables which order to call the chains in.
57+
> Docker uses well known [priority values](https://wiki.nftables.org/wiki-nftables/index.php/Netfilter_hooks#Priority_within_hook) for each of its base chains.
58+
59+
## Migrating from iptables to nftables
60+
61+
If the Docker daemon has been running with the iptables firewall backend,
62+
restarting it with the nftables backend will delete most of Docker's iptables
63+
chains and rules, and create nftables rules instead.
64+
65+
If IP forwarding is not enabled, Docker will report an error when creating
66+
a bridge network that needs it. Because of the default bridge, if IPv4
67+
forwarding is disabled, the error will be reported during daemon startup.
68+
See [IP forwarding](#ip-forwarding).
69+
70+
If you have rules in the `DOCKER-USER` chain, see [Migrating
71+
`DOCKER-USER`](#migrating-docker-user).
72+
73+
You may need to manually update the iptables `FORWARD` policy if it has
74+
been set to `DROP` by Docker with iptables, or as part of your host's
75+
firewall configuration. See [FORWARD policy in iptables](#forward-policy-in-iptables).
76+
77+
### IP forwarding
78+
79+
IP forwarding on the Docker host enables Docker functionality including port
80+
publishing, communication between bridge networks, and direct routing from
81+
outside the host to containers in bridge networks.
82+
83+
When running with iptables, depending on network and daemon configuration,
84+
Docker may enable IPv4 and IPv6 forwarding on the host.
85+
86+
With its nftables firewall backend enabled, Docker will not enable IP forwarding
87+
itself. It will report an error if forwarding is needed, but not already enabled.
88+
To disable Docker's check for IP forwarding, letting it start and create networks
89+
when it determines that forwarding is disabled, use Daemon option `--ip-forward=false`,
90+
or `"ip-forward": false` in its configuration file.
91+
92+
> [!WARNING]
93+
>
94+
> When enabling IP forwarding, make sure you have firewall rules to block
95+
> unwanted forwarding between non-Docker interfaces.
96+
97+
> [!NOTE]
98+
>
99+
> If you stop Docker to migrate to nftables, Docker may have already enabled
100+
> IP forwarding on your system. After a reboot, if no other service re-enables
101+
> forwarding, Docker will fail to start.
102+
103+
If Docker is in a VM that has a single network interface and no other
104+
software running, there is probably no unwanted forwarding to block.
105+
But, on a physical host with multiple network interfaces, forwarding
106+
between those interfaces should probably be blocked unless the host
107+
is acting as a router.
108+
109+
To enable IP forwarding on the host, set the following sysctls:
110+
111+
- `net.ipv4.ip_forward=1`
112+
- `net.ipv6.conf.all.forwarding=1`
113+
114+
If your host uses `systemd`, you may be able to use `systemd-sysctl`. For
115+
example, by editing `/etc/sysctl.d/99-sysctl.conf`.
116+
117+
If the host is running `firewalld`, you may be able to use it to block
118+
unwanted forwarding. Docker's bridges are in a firewalld zone called
119+
`docker`, it creates a forwarding policy called `docker-forwarding` that
120+
accepts forwarding from `ANY` zone to the `docker` zone.
121+
122+
As an example, to use nftables to block forwarding between interfaces `eth0`
123+
and `eth1`, you could use:
124+
125+
```console
126+
table inet no-ext-forwarding {
127+
chain no-ext-forwarding {
128+
type filter hook forward priority filter; policy accept;
129+
iifname "eth0" oifname "eth1" drop
130+
iifname "eth1" oifname "eth0" drop
131+
}
132+
}
133+
```
134+
135+
### FORWARD policy in iptables
136+
137+
An iptables chain with `FORWARD` policy `DROP` will drop packets that have
138+
been accepted by Docker's nftables rules, because the packet will be processed
139+
by the iptables chains as well as Docker's nftables chains.
140+
141+
Some features, including port publishing, will not work unless the `DROP`
142+
policy is removed, or additional iptables rules are added to the iptables
143+
`FORWARD` chain to accept Docker-related traffic.
144+
145+
When Docker is using iptables, and it enables IP forwarding on the host,
146+
it sets the default policy of the iptables `FORWARD` chain to `DROP`. So,
147+
if you stop Docker to migrate to nftables, it may have set a `DROP` that
148+
you need to remove. It will be removed anyway on reboot.
149+
150+
To keep using rules in `DOCKER-USER` that rely on the chain having policy
151+
`DROP`, you must add explicit `ACCEPT` rules for Docker related traffic.
152+
153+
To check the current iptables `FORWARD` policy, use:
154+
155+
```console
156+
$ iptables -L FORWARD
157+
Chain FORWARD (policy DROP)
158+
target prot opt source destination
159+
$ ip6tables -L FORWARD
160+
Chain FORWARD (policy ACCEPT)
161+
target prot opt source destination
162+
```
163+
164+
To set the iptables policies to `ACCEPT` for IPv4 and IPv6:
165+
166+
```console
167+
$ iptables -P FORWARD ACCEPT
168+
$ ip6tables -P FORWARD ACCEPT
169+
```
170+
171+
### Migrating `DOCKER-USER`
172+
173+
With firewall backend "iptables", rules added to the iptables `DOCKER-USER`
174+
are processed before Docker's rules in the filter table's `FORWARD` chain.
175+
176+
When starting the daemon with nftables after running with iptables, Docker
177+
will not remove the jump from the `FORWARD` chain to `DOCKER-USER`. So,
178+
rules created in `DOCKER-USER` will continue to run until the jump is
179+
removed or the host is rebooted.
180+
181+
When starting with nftables, the daemon will not add the jump. So, unless
182+
there is an existing jump, rules in `DOCKER-USER` will be ignored.
183+
184+
#### Migrating ACCEPT rules
185+
186+
Some rules in the `DOCKER-USER` chain will continue to work. For example, if a
187+
packet is dropped, it will be dropped before or after the nftables rules in
188+
Docker's `filter-FORWARD` chain. But other rules, particularly `ACCEPT` rules
189+
to override Docker's `DROP` rules, will not work.
190+
191+
In nftables, an "accept" rule is not final. It terminates processing
192+
for its base chain, but the accepted packet will still be processed by
193+
other base chains, which may drop it.
194+
195+
To override Docker's `drop` rule, you must use a firewall mark. Select a
196+
mark not already in use on your host, and use Docker Engine option
197+
`--bridge-accept-fwmark`.
198+
199+
For example, `--bridge-accept-fwmark=1` tells the daemon to accept any
200+
packet with an `fwmark` value of `1`. Optionally, you can supply a mask
201+
to match specific bits in the mark, `--bridge-accept-fwmark=0x1/0x3`.
202+
203+
Then, instead of accepting the packet in `DOCKER-USER`, add the firewall
204+
mark you have chosen and Docker will not drop it.
205+
206+
The firewall mark must be added before Docker's rules run. So if the mark
207+
is added in a chain with type `filter` and hook `forward`, it must have
208+
priority `filter - 1` or lower.
209+
210+
#### Replacing `DOCKER-USER` with an nftables table
211+
212+
Because nftables doesn't have pre-defined chains, to replace the `DOCKER-USER`
213+
chain you can create your own table and add chains and rules to it.
214+
215+
The `DOCKER-USER` chain has type `filter` and hook `forward`, so it can
216+
only have rules in the filter forward chain. The base chains in your
217+
table can have any `type` or `hook`. If your rules need to run before
218+
Docker's rules, give the base chains a lower `priority` number than
219+
Docker's chain. Or, a higher priority to make sure they run after Docker's
220+
rules.
221+
222+
Docker's base chains use the priority values defined at
223+
[priority values](https://wiki.nftables.org/wiki-nftables/index.php/Netfilter_hooks#Priority_within_hook)
224+
225+
#### Example: restricting external connections to containers
226+
227+
By default, any remote host can connect to ports published to the Docker
228+
host's external addresses.
229+
230+
To allow only a specific IP or network to access the containers, create a
231+
table with a base chain that has a drop rule. For example, the
232+
following table drops packets from all IP addresses except `192.0.2.2`:
233+
234+
```console
235+
table ip my-table {
236+
chain my-filter-forward {
237+
type filter hook forward priority filter; policy accept;
238+
iifname "ext_if" ip saddr != 192.0.2.2 counter drop
239+
}
240+
}
241+
```
242+
243+
You will need to change `ext_if` to your host's external interface name.
244+
245+
You could instead accept connections from a source subnet. The following
246+
table only accepts access from the subnet `192.0.2.0/24`:
247+
248+
```console
249+
table ip my-table {
250+
chain my-filter-forward {
251+
type filter hook forward priority filter; policy accept;
252+
iifname "ext_if" ip saddr != 192.0.2.0/24 counter drop
253+
}
254+
}
255+
```
256+
257+
If you are running other services on the host that use IP forwarding
258+
and need to be accessed by different external hosts, you will need more
259+
specific filters. For example, to match the default prefix `br-` of
260+
bridge devices belonging to Docker's user-defined bridge networks:
261+
262+
```console
263+
table ip my-table {
264+
chain my-filter-forward {
265+
type filter hook forward priority filter; policy accept;
266+
iifname "ext_if" oifname "br-*" ip saddr != 192.0.2.0/24 counter drop
267+
}
268+
}
269+
```
270+
271+
For more detailed information about nftables configuration and advanced usage,
272+
refer to the [nftables wiki](https://wiki.nftables.org/wiki-nftables/index.php/Main_Page).

‎content/manuals/engine/network/packet-filtering-firewalls.md

Lines changed: 14 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -25,10 +25,15 @@ implement functionality including port publishing, and NAT/masquerading.
2525
2626
## Firewall backend
2727

28-
Docker Engine creates its firewall rules using iptables,
29-
see [Docker with iptables](./firewall-iptables.md).
28+
By default, Docker Engine creates its firewall rules using iptables,
29+
see [Docker with iptables](./firewall-iptables.md). It also has
30+
support for nftables, see [Docker with nftables](./firewall-nftables.md).
3031

32+
For bridge networks, iptables and nftables have the same functionality.
3133

34+
Docker Engine option `firewall-backend` can be used to select whether
35+
iptables or nftables is used. See
36+
[daemon configuration](https://docs.docker.com/reference/cli/dockerd/).
3237

3338
## Docker on a router
3439

@@ -46,11 +51,17 @@ To stop Docker from setting the forwarding policy to "drop", include
4651
`"ip-forward-no-drop": true` in `/etc/docker/daemon.json`, or add option
4752
`--ip-forward-no-drop` to the `dockerd` command line.
4853

54+
> [!NOTE]
55+
>
56+
> With the experimental nftables backend, Docker does not enable IP forwarding
57+
> itself, and it will not create a default "drop" nftables policy. See
58+
> [Migrating from iptables to nftables](./firewall-nftables.md#migrating-from-iptables-to-nftables).
59+
4960
## Prevent Docker from manipulating firewall rules
5061

5162
Setting the `iptables` or `ip6tables` keys to `false` in
5263
[daemon configuration](https://docs.docker.com/reference/cli/dockerd/), will
53-
prevent Docker from creating most of its `iptables` rules. But,
64+
prevent Docker from creating most of its `iptables` or `nftables` rules. But,
5465
this option is not appropriate for most users, it is likely to break
5566
container networking for the Docker Engine.
5667

0 commit comments

Comments
 (0)