Skip to content

Commit fc431e9

Browse files
robmryrobmry
committed
Describe migration from iptables to experimental nftables
Adds engine/network/firewall-nftables.md Signed-off-by: Rob Murray <[email protected]>
1 parent 01b176d commit fc431e9

File tree

2 files changed

+287
-3
lines changed

2 files changed

+287
-3
lines changed

‎content/manuals/engine/network/firewall-nftables.md

Lines changed: 273 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,273 @@
1+
---
2+
title: Docker with nftables
3+
weight: 10
4+
description: How Docker works with nftables
5+
keywords: network, nftables, firewall
6+
---
7+
8+
> [!WARNING]
9+
>
10+
> Support for nftables is experimental, configuration options, behavior and
11+
> implementation may all change in future releases.
12+
> The rules for overlay networks have not yet been migrated from iptables.
13+
> So, nftables cannot be enabled when the daemon has Swarm enabled.
14+
15+
To use nftables instead of iptables, use Docker Engine option
16+
`--firewall-backend=nftables` on its command line, or `"firewall-backend": true`
17+
in its configuration file. You may also need to modify IP forwarding configuration
18+
on the host, and migrate rules from the iptables `DOCKER-USER` chain, see
19+
[migrating from iptables to nftables](#migrating-from-iptables-to-nftables).
20+
21+
For bridge networks, Docker creates nftables rules in the host's network
22+
namespace. For bridge and other network types, nftables rules for DNS are
23+
also created in the container's network namespace.
24+
25+
Creation of nftables rules can be disabled using daemon options `iptables`
26+
and `ip6tables`. _These options apply to both iptables and nftables._
27+
See [Prevent Docker from manipulating firewall rules](packet-filtering-firewalls.md#prevent-docker-from-manipulating-firewall-rules).
28+
But, this is not appropriate for most users, it is likely to break
29+
container networking.
30+
31+
## Docker's nftables tables
32+
33+
For bridge networks, Docker creates two tables, `ip docker-bridges` and
34+
`ip6 docker-bridges`.
35+
36+
Each table contains a number of [base chains](https://wiki.nftables.org/wiki-nftables/index.php/Configuring_chains#Adding_base_chains),
37+
and further chains are added for each bridge network. The moby project
38+
has some [internal documentation](https://github.com/moby/moby/blob/master/integration/network/bridge/nftablesdoc/index.md)
39+
describing its nftables, and how they depend on network and container
40+
configuration. But, the tables and their rules are likely to change between
41+
Docker Engine releases.
42+
43+
Do not modify Docker's tables directly as the modifications are likely to
44+
be lost, Docker expects to have full ownership of its tables.
45+
46+
> [!NOTE]
47+
>
48+
> Because iptables has a fixed set of chains, equivalent to nftables base
49+
> chains, all rules are included in those chains. The `DOCKER-USER` chain
50+
> is supplied as a way to insert rules into the `filter` table's `FORWARD`
51+
> chain, to run before Docker's rules.
52+
> In Docker's nftables implementation, there is no `DOCKER-USER` chain.
53+
> Instead, rules can be added in separate tables, with base chains that
54+
> have the same types and hook points as Docker's base chains. If necessary,
55+
> [base chain priority](https://wiki.nftables.org/wiki-nftables/index.php/Configuring_chains#Base_chain_priority)
56+
> can be used to tell nftables which order to call the chains in.
57+
> Docker uses well known [priority values](https://wiki.nftables.org/wiki-nftables/index.php/Netfilter_hooks#Priority_within_hook) for each of its base chains.
58+
59+
## Migrating from iptables to nftables
60+
61+
If the Docker daemon has been running with the iptables firewall backend,
62+
restarting it with the nftables backend will delete most of Docker's iptables
63+
chains and rules, and create nftables rules instead.
64+
65+
If IP forwarding is not enabled, Docker will report an error when creating
66+
a bridge network that needs it. Because of the default bridge, if IPv4
67+
forwarding is disabled, the error will be reported during daemon startup.
68+
See [IP forwarding](#ip-forwarding).
69+
70+
If you have rules in the `DOCKER-USER` chain, see [Migrating
71+
`DOCKER-USER`](#migrating-docker-user).
72+
73+
You may need to manually update the iptables `FORWARD` policy if it has
74+
been set to `DROP` by Docker with iptables, or as part of your host's
75+
firewall configuration. See [FORWARD policy in iptables](#forward-policy-in-iptables).
76+
77+
### IP forwarding
78+
79+
IP forwarding on the Docker host enables Docker functionality including port
80+
publishing, communication between bridge networks, and direct routing from
81+
outside the host to containers in bridge networks.
82+
83+
When running with iptables, depending on network and daemon configuration,
84+
Docker may enable IPv4 and IPv6 forwarding on the host.
85+
86+
With its nftables firewall backend enabled, Docker will not enable IP forwarding
87+
itself. It will report an error if forwarding is needed, but not already enabled.
88+
To disable Docker's check for IP forwarding, allowing it to start and
89+
create networks when it reports that forwarding is disabled, use Daemon option
90+
`--ip-forward=false`, or `"ip-forward": false` in its configuration file.
91+
92+
> [!WARNING]
93+
>
94+
> When enabling IP forwarding, make sure you have firewall rules to block
95+
> unwanted forwarding between non-Docker interfaces.
96+
97+
> [!NOTE]
98+
>
99+
> If you stop Docker to migrate to nftables, Docker may already have enabled
100+
> forwarding. If nothing else enables it when you reboot, Docker will
101+
> probably not start up.
102+
103+
If Docker is in a VM that has a single network interface and no other
104+
software running, there is probably no unwanted forwarding to block.
105+
But, on a physical host with multiple network interfaces, forwarding
106+
between those interfaces should probably be blocked unless the host
107+
is acting as a router.
108+
109+
To enable IP forwarding on the host, set the following sysctls:
110+
111+
- `net.ipv4.ip_forward=1`
112+
- `net.ipv6.conf.all.forwarding=1`
113+
114+
If your host uses `systemd`, you may be able to use `systemd-sysctl`. For
115+
example, by editing `/etc/sysctl.d/99-sysctl.conf`.
116+
117+
If the host is running `firewalld`, you may be able to use it to block
118+
unwanted forwarding. Docker's bridges are in a firewalld zone called
119+
`docker`, it creates a forwarding policy called `docker-forwarding` that
120+
allows forwarding from `ANY` zone to the `docker` zone.
121+
122+
As an example, to use nftables to block forwarding between interfaces `eth0`
123+
and `eth1`, you could use:
124+
125+
```console
126+
table inet no-ext-forwarding {
127+
chain no-ext-forwarding {
128+
type filter hook forward priority filter; policy accept;
129+
iifname "eth0" oifname "eth1" drop
130+
iifname "eth1" oifname "eth0" drop
131+
}
132+
}
133+
```
134+
135+
### FORWARD policy in iptables
136+
137+
An iptables chain with `FORWARD` policy `DROP` will drop packets that have
138+
been accepted by Docker's nftables rules, because the packet will be processed
139+
by the iptables chains as well as Docker's nftables chains.
140+
141+
Some features, including port publishing, will not work unless the `DROP`
142+
policy is removed, or additional iptables rules are added to the iptables
143+
`FORWARD` chain to accept Docker-related traffic.
144+
145+
When Docker is using iptables, and it enables IP forwarding on the host,
146+
it sets the default policy of the iptables `FORWARD` chain to `DROP`. So,
147+
if you stop Docker to migrate to nftables, it may have set a `DROP` that
148+
you need to remove. It will be removed anyway on reboot.
149+
150+
If you want to keep using rules in `DOCKER-USER` that rely on the chain
151+
having policy `DROP`, you will need to add explicit `ACCEPT` rules for
152+
Docker related traffic.
153+
154+
To check the current iptables `FORWARD` policy, use:
155+
156+
```console
157+
$ iptables -L FORWARD
158+
Chain FORWARD (policy DROP)
159+
target prot opt source destination
160+
$ ip6tables -L FORWARD
161+
Chain FORWARD (policy ACCEPT)
162+
target prot opt source destination
163+
```
164+
165+
To set the iptables policies to `ACCEPT` for IPv4 and IPv6:
166+
167+
```console
168+
$ iptables -P FORWARD ACCEPT
169+
$ ip6tables -P FORWARD ACCEPT
170+
```
171+
172+
### Migrating `DOCKER-USER`
173+
174+
With firewall backend "iptables", rules added to the iptables `DOCKER-USER`
175+
are processed before Docker's rules in the filter table's `FORWARD` chain.
176+
177+
When starting the daemon with nftables after running with iptables, Docker
178+
will not remove the jump from the `FORWARD` chain to `DOCKER-USER`. So,
179+
rules created in `DOCKER-USER` will continue to run until the jump is
180+
removed or the host is rebooted.
181+
182+
When starting with nftables, the daemon will not add the jump. So, unless
183+
there is an existing jump, rules in `DOCKER-USER` will be ignored.
184+
185+
#### Migrating ACCEPT rules
186+
187+
Some rules in the `DOCKER-USER` chain will continue to work. For example, if a
188+
packet is dropped, it will be dropped before or after the nftables rules in
189+
Docker's `filter-FORWARD` chain. But other rules, particularly `ACCEPT` rules
190+
to override Docker's `DROP` rules, will not work.
191+
192+
In nftables, an "accept" rule is not final. It terminates processing
193+
for its base chain, but the accepted packet will still be processed by
194+
other base chains, which may drop it.
195+
196+
To override Docker's `drop` rule, you must use a firewall mark. Select a
197+
mark not already in use on your host, and use Docker Engine option
198+
`--bridge-accept-fwmark`.
199+
200+
For example, `--bridge-accept-fwmark=1` tells the daemon to accept any
201+
packet with an `fwmark` value of `1`. Optionally, you can supply a mask
202+
to match specific bits in the mark, `--bridge-accept-fwmark=0x1/0x3`.
203+
204+
Then, instead of accepting the packet in `DOCKER-USER`, add the firewall
205+
mark you have chosen and Docker will not drop it.
206+
207+
The firewall mark must be added before Docker's rules run. So if the mark
208+
is added in a chain with type `filter` and hook `forward`, it must have
209+
priority `filter - 1` or lower.
210+
211+
#### Replacing `DOCKER-USER` with an nftables table
212+
213+
Because nftables doesn't have pre-defined chains, to replace the `DOCKER-USER`
214+
chain you can create your own table and add chains and rules to it.
215+
216+
The `DOCKER-USER` chain has type `filter` and hook `forward`, so it can
217+
only have rules in the filter forward chain. The base chains in your
218+
table can have any `type` or `hook`. If your rules need to run before
219+
Docker's rules, give the base chains a lower `priority` number than
220+
Docker's chain. Or, a higher priority to make sure they run after Docker's
221+
rules.
222+
223+
Docker's base chains use the priority values defined at
224+
[priority values](https://wiki.nftables.org/wiki-nftables/index.php/Netfilter_hooks#Priority_within_hook)
225+
226+
#### Example: restricting external connections to containers
227+
228+
By default, any remote host can connect to ports published to the Docker
229+
host's external addresses.
230+
231+
To allow only a specific IP or network to access the containers, create a
232+
table with a base chain that has a drop rule. For example, the
233+
following table drops packets from all IP addresses except `192.0.2.2`:
234+
235+
```console
236+
table ip my-table {
237+
chain my-filter-forward {
238+
type filter hook forward priority filter; policy accept;
239+
iifname "ext_if" ip saddr != 192.0.2.2 counter drop
240+
}
241+
}
242+
```
243+
244+
You will need to change `ext_if` to your host's external interface name.
245+
246+
You could instead allow connections from a source subnet. The following
247+
table only allows access from the subnet `192.0.2.0/24`:
248+
249+
```console
250+
table ip my-table {
251+
chain my-filter-forward {
252+
type filter hook forward priority filter; policy accept;
253+
iifname "ext_if" ip saddr != 192.0.2.2/24 counter drop
254+
}
255+
}
256+
```
257+
258+
If you are running other services on the host that use IP forwarding
259+
and need to be accessed by different external hosts, you will need more
260+
specific filters. For example, to match the default prefix `br-` of
261+
bridge devices belonging to Docker's user-defined bridge networks:
262+
263+
```console
264+
table ip my-table {
265+
chain my-filter-forward {
266+
type filter hook forward priority filter; policy accept;
267+
iifname "ext_if" oifname "br-*" ip saddr != 192.0.2.2/24 counter drop
268+
}
269+
}
270+
```
271+
272+
`nftables` is complicated. There is a lot more information at the
273+
[nftables wiki](https://wiki.nftables.org/wiki-nftables/index.php/Main_Page).

‎content/manuals/engine/network/packet-filtering-firewalls.md

Lines changed: 14 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -25,10 +25,15 @@ implement functionality including port publishing, and NAT/masquerading.
2525
2626
## Firewall backend
2727

28-
Docker Engine creates its firewall rules using iptables,
29-
see [Docker with iptables](./firewall-iptables.md).
28+
By default, Docker Engine creates its firewall rules using iptables,
29+
see [Docker with iptables](./firewall-iptables.md). It also has
30+
support for nftables, see [Docker with nftables](./firewall-nftables.md).
3031

32+
For bridge networks, iptables and nftables have the same functionality.
3133

34+
Docker Engine option `firewall-backend` can be used to select whether
35+
iptables or nftables is used. See
36+
[daemon configuration](https://docs.docker.com/reference/cli/dockerd/).
3237

3338
## Docker on a router
3439

@@ -46,11 +51,17 @@ To stop Docker from setting the forwarding policy to "drop", include
4651
`"ip-forward-no-drop": true` in `/etc/docker/daemon.json`, or add option
4752
`--ip-forward-no-drop` to the `dockerd` command line.
4853

54+
> [!NOTE]
55+
>
56+
> With the experimental nftables backend, Docker does not enable IP forwarding
57+
> itself, and it will not create a default "drop" nftables policy. See
58+
> [Migrating from iptables to nftables](./firewall-nftables.md#migrating-from-iptables-to-nftables).
59+
4960
## Prevent Docker from manipulating firewall rules
5061

5162
Setting the `iptables` or `ip6tables` keys to `false` in
5263
[daemon configuration](https://docs.docker.com/reference/cli/dockerd/), will
53-
prevent Docker from creating most of its `iptables` rules. But,
64+
prevent Docker from creating most of its `iptables` or `nftables` rules. But,
5465
this option is not appropriate for most users, it is likely to break
5566
container networking for the Docker Engine.
5667

0 commit comments

Comments
 (0)