Skip to content

Commit 1d08e82

Browse files
committed
net: remove loopback XDP socket
Removes the loopback XDP socket from the net tile. Also removes the ethtool-loopback configure stage. Loopback is done via a sock tile in the future.
1 parent 57ee3fd commit 1d08e82

File tree

22 files changed

+43
-399
lines changed

22 files changed

+43
-399
lines changed

book/api/cli.md

Lines changed: 0 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -70,7 +70,6 @@ following stages to each configure command:
7070
device.
7171
- `ethtool-gro` Disables generic receive offload (GRO) on the network
7272
device.
73-
- `ethtool-loopback` Disables UDP segmentation on the loopback device.
7473

7574
| Arguments | Description |
7675
|-------------------|-------------|
@@ -102,7 +101,6 @@ and configure the number of combined channels on the network device.
102101
| `root` | increase `/proc/sys/vm/nr_hugepages` and mount hugetlbfs filesystems. Only applies for the `hugetlbfs` stage |
103102
| `root` | increase network device channels with `ethtool --set-channels`. Only applies for the `ethtool-channels` stage |
104103
| `root` | disable network device generic-receive-offload (gro) with `ethtool --offload IFACE generic-receive-offload off`. Only applies for the `ethtool-gro` stage |
105-
| `root` | disable network device tx-udp-segmentation with `ethtool --offload lo tx-udp-segmentation off`. Only applies for the `ethtool-loopback` stage |
106104
| `CAP_SYS_ADMIN` | set kernel parameters in `/proc/sys`. Only applies for the `sysctl` stage |
107105

108106
:::

book/guide/initializing.md

Lines changed: 1 addition & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -11,8 +11,6 @@ so Firedancer can run correctly. It does the following:
1111
device.
1212
* **ethtool-gro** Disable generic-receive-offload (GRO) on the network
1313
device.
14-
* **ethtool-loopback** Disable tx-udp-segmentation on the loopback
15-
device.
1614

1715
The `hugetlbfs` configuration must be performed every time the system
1816
is rebooted, to remount the `hugetlbfs` filesystems, as do `sysctl`,
@@ -30,7 +28,7 @@ where `mode` is one of:
3028
- `fini` Unconfigure (reverse) the stage if it is reversible.
3129

3230
`stage` can be one or more of `hugetlbfs`, `sysctl`, `hyperthreads`,
33-
`ethtool-channels`, `ethtool-gro`, `ethtool-loopback`, and `snapshots`
31+
`ethtool-channels`, `ethtool-gro`, and `snapshots`
3432
and these stages are described below. You can also use the stage `all`
3533
which will configure everything.
3634

@@ -193,22 +191,6 @@ Firedancer. It has no dependencies on any other stage.
193191
Changing device settings with `ethtool-gro` requires root privileges, and
194192
cannot be performed with capabilities.
195193

196-
## ethtool-loopback
197-
XDP is incompatible with localhost UDP traffic using a feature called
198-
`tx-udp-segmentation`. This feature must be disabled when connecting Agave
199-
clients to Firedancer over loopback, or when using Frankendancer.
200-
201-
The command run by the stage is `ethtool --offload lo tx-udp-segmentation
202-
off`. We can check that it worked:
203-
204-
<<< @/snippets/ethtool-loopback.ansi
205-
206-
The stage only needs to be run once after boot but before running
207-
Firedancer. It has no dependencies on any other stage.
208-
209-
Changing device settings with `ethtool-loopback` requires root privileges,
210-
and cannot be performed with capabilities.
211-
212194
## snapshots
213195
When starting up, validators must load a snapshot to catch up to the
214196
current state of the blockchain. Snapshots are downloaded from other

book/guide/internals/net_tile.md

Lines changed: 0 additions & 22 deletions
Original file line numberDiff line numberDiff line change
@@ -426,28 +426,6 @@ completion ring.
426426
427427
The net tile moves completed frames back to the free ring.
428428
429-
## Loopback
430-
431-
The first net tile (`net:0`) sets up XDP on the loopback device, for
432-
two main reasons:
433-
434-
* For testing and development.
435-
* The Agave code sends local traffic to itself to as part of routine
436-
operation (e.g., when it's the leader it sends votes to its own TPU
437-
socket).
438-
439-
The Linux kernel routes outgoing packets addressed to IP addresses
440-
owned by the system via loopback. (See `ip route show table local`)
441-
The net tile partially matches this behavior. For better performance
442-
and simplicity, a second XDP socket is used.
443-
444-
Alternatively, the net tile could have sent such traffic out to the
445-
public gateway, in hopes that the traffic gets mirrored back.
446-
447-
But for now, Firedancer also binds XDP to loopback. This is a small performance hit for other traffic, but otherwise won't interfere.
448-
449-
The loopback device only supports XDP in SKB mode.
450-
451429
## Development
452430
453431
### Network Namespace

book/snippets/commands/configure-check.ansi

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,5 +3,4 @@ $ fdctl configure check all
33
WARNING sysctl ... not configured ... kernel parameter `/proc/sys/vm/max_map_count` is too low (got 65536 but expected at least 1000000)
44
WARNING ethtool-channels ... not configured ... device `ens3f0` does not have right number of channels (got 1 but expected 2)
55
WARNING ethtool-gro ... not configured ... device `ens3f0` has generic-receive-offload enabled. Should be disabled
6-
WARNING ethtool-loopback ... not configured ... device `lo` has tx-udp-segmentation enabled. Should be disabled
76
ERR  failed to configure some stages

book/snippets/commands/configure-init.ansi

Lines changed: 0 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -23,6 +23,3 @@ $ fdctl configure init all
2323
NOTICE  ethtool-gro ... unconfigured ... device `ens3f0` has generic-receive-offload enabled. Should be disabled
2424
NOTICE  ethtool-gro ... configuring
2525
NOTICE  ethtool-gro ... RUN: `ethtool --offload ens3f0 generic-receive-offload off`
26-
NOTICE  ethtool-loopback ... unconfigured ... device `lo` has tx-udp-segmentation enabled. Should be disabled
27-
NOTICE  ethtool-loopback ... configuring
28-
NOTICE  ethtool-loopback ... RUN: `ethtool --offload lo tx-udp-segmentation off`

book/snippets/configure.ansi

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -7,4 +7,3 @@
77
NOTICE  sysctl ... already valid
88
NOTICE  ethtool-channels ... already valid
99
NOTICE  ethtool-gro ... already valid
10-
NOTICE  ethtool-loopback ... already valid

book/snippets/ethtool-loopback.ansi

Lines changed: 0 additions & 7 deletions
This file was deleted.

contrib/test/test_firedancer_leader.sh

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -85,7 +85,6 @@ echo "
8585
sudo $FD_DIR/$OBJDIR/bin/firedancer-dev configure init kill --config $(readlink -f firedancer-dev.toml)
8686
sudo $FD_DIR/$OBJDIR/bin/firedancer-dev configure init hugetlbfs --config $(readlink -f firedancer-dev.toml)
8787
sudo $FD_DIR/$OBJDIR/bin/firedancer-dev configure init ethtool-channels --config $(readlink -f firedancer-dev.toml)
88-
sudo $FD_DIR/$OBJDIR/bin/firedancer-dev configure init ethtool-gro ethtool-loopback --config $(readlink -f firedancer-dev.toml)
8988
sudo $FD_DIR/$OBJDIR/bin/firedancer-dev configure init keys --config $(readlink -f firedancer-dev.toml)
9089

9190
sudo gdb -iex="set debuginfod enabled on" -ex=r --args $FD_DIR/$OBJDIR/bin/firedancer-dev dev --no-configure --log-path $(readlink -f firedancer-dev.log) --config $(readlink -f firedancer-dev.toml)

src/app/fdctl/main.c

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -38,7 +38,6 @@ configure_stage_t * STAGES[] = {
3838
&fd_cfg_stage_hyperthreads,
3939
&fd_cfg_stage_ethtool_channels,
4040
&fd_cfg_stage_ethtool_gro,
41-
&fd_cfg_stage_ethtool_loopback,
4241
NULL,
4342
};
4443

src/app/fddev/main.h

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -48,7 +48,6 @@ configure_stage_t * STAGES[] = {
4848
&fd_cfg_stage_hyperthreads,
4949
&fd_cfg_stage_ethtool_channels,
5050
&fd_cfg_stage_ethtool_gro,
51-
&fd_cfg_stage_ethtool_loopback,
5251
&fd_cfg_stage_keys,
5352
&fd_cfg_stage_genesis,
5453
&fd_cfg_stage_blockstore,

0 commit comments

Comments
 (0)