Skip to content

Lower memory footprint and packet pool #666

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
guvenc merged 1 commit into from
Apr 15, 2025
Merged

Lower memory footprint and packet pool #666

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 15 additions & 3 deletions include/dpdk_layer.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,13 +33,25 @@ extern "C" {
// there are three periodic messages (ARP, ND, ND-RA) that could be sent at once
#define DP_PERIODIC_Q_SIZE (DP_MAX_PORTS * 3)

// 40Gb/s with 1500B packets means ~3.3M packets/s
// assuming 0.1s delay in processing means ~350k mbufs needed
// While dpservice is processing RTE_GRAPH_BURST_SIZE, the NIC will store packets here
// Seen recommendations to keep this at 2*RTE_GRAPH_BURST_SIZE or 4*RTE_GRAPH_BURST_SIZE
#define DP_RX_QUEUE_SIZE (4 * RTE_GRAPH_BURST_SIZE)
// Seen this recommended to be bigger than Rx because multiple Rx streams share the same Tx
// but in our case we are only using one worker thread, so there is no concurrent Tx
#define DP_TX_QUEUE_SIZE DP_RX_QUEUE_SIZE

#ifdef ENABLE_PYTEST
#define DP_MBUF_POOL_SIZE (50*1024)
#else
#define DP_MBUF_POOL_SIZE (350*1024)
// packet pool needs to be able to hold all packets from all port Rx queues
// (as requested in the configure stage via DP_RX_QUEUE_SIZE)
// then also all packets in a burst can allocate a chunk - RTE_GRAPH_BURST_SIZE
// then there are service queues/periodic/grpc/...
// thus a headroom of 4k should be OK
#define DP_MBUF_POOL_HEADROOM 4096
#define DP_MBUF_POOL_SIZE ((DP_NR_STD_RX_QUEUES * DP_RX_QUEUE_SIZE) * DP_MAX_PORTS + DP_MBUF_POOL_HEADROOM)
#endif

// max Ether MTU 1500 + frame header 14 + frame footer 4 + IPv6 tunnel header 40
#define DP_MBUF_BUF_SIZE (1558 + RTE_PKTMBUF_HEADROOM)

Expand Down
4 changes: 2 additions & 2 deletions src/dp_port.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -123,7 +123,7 @@ static int dp_port_init_ethdev(struct dp_port *port, struct rte_eth_dev_info *de

/* RX and TX queues config */
for (uint16_t i = 0; i < DP_NR_STD_RX_QUEUES; ++i) {
ret = rte_eth_rx_queue_setup(port->port_id, i, 1024,
ret = rte_eth_rx_queue_setup(port->port_id, i, DP_RX_QUEUE_SIZE,
port->socket_id,
&rxq_conf,
dp_layer->rte_mempool);
Expand All @@ -137,7 +137,7 @@ static int dp_port_init_ethdev(struct dp_port *port, struct rte_eth_dev_info *de
txq_conf.offloads = port_conf.txmode.offloads;

for (uint16_t i = 0; i < DP_NR_STD_TX_QUEUES; ++i) {
ret = rte_eth_tx_queue_setup(port->port_id, i, 2048,
ret = rte_eth_tx_queue_setup(port->port_id, i, DP_TX_QUEUE_SIZE,
port->socket_id,
&txq_conf);
if (DP_FAILED(ret)) {
Expand Down
Loading