Segmentation fault in picoquic-dpdk client and multiple test failures on Ubuntu 24.04 with DPDK 21.11

[demartinomarco]

Issue Description

I tried compiling picoquic-dpdk in a virtual machine running Ubuntu Server 24.04.3, but encountered two problems:

1. Not all unit tests pass during the test suite
2. When trying to retrieve an HTML file from the server using the demo client, it segfaults after connection establishment

Steps to Reproduce

1. Build using the provided script (see below)
2. Run unit tests: ./picoquic_ct -n -r
3. Start server: ./picoquicdemo -p 4433
4. Run client: ./picoquicdemo 127.0.0.1 4433 index.html

Expected Behavior

• All unit tests should pass
• Client should successfully retrieve the HTML file without segmentation faults

Actual Behavior

• Multiple unit tests fail (see test output below)
• Client establishes connection but segfaults when opening stream 0 to GET /index.html

Client Log Output

Starting Picoquic (v1.01) connection to server = 127.0.0.1, port = 4433
No token file present. Will create one as <demo_token_store.bin>.
Testing scenario: <index.html>
Max stream id bidir remote before start = 0 (0)
Starting client connection. Version = 1, I-CID: fcf09fa346eaa85a
Max stream id bidir remote after start = 0 (0)
Waiting for packets.
Client port (AF=2): 36606.
Negotiated ALPN: h3
Almost ready!

Connection established. Version = 1, I-CID: fcf09fa346eaa85a, verified: 1
Opening stream 0 to GET /index.html
Segmentation fault (core dumped)

Build Environment

• OS: Ubuntu Server 24.04.3 (Virtual Machine)
• DPDK Version: 21.11.9
• picoquic-dpdk: Latest from GitHub (https://github.com/IPNetworkingLab/picoquic-dpdk)

Build Script

#!/bin/bash

# Update system and install dependencies
sudo apt-get update
sudo apt-get install -y libssl-dev libnuma-dev meson ninja-build pkg-config python3-pip wget cmake make

# Install Python dependencies
pip3 install pyelftools

# Install DPDK 21.11
wget https://fast.dpdk.org/rel/dpdk-21.11.9.tar.xz
tar xf dpdk-21.11.9.tar.xz
cd dpdk-stable-21.11.9
meson build
ninja -C build
sudo ninja -C build install
sudo ldconfig
cd ..

# Build picoquic 
git clone https://github.com/IPNetworkingLab/picoquic-dpdk
cd picoquic-dpdk
./ci/build_picotls.sh

# Set environment variables and build
export PKG_CONFIG_PATH=/usr/local/lib/x86_64-linux-gnu/pkgconfig:$PKG_CONFIG_PATH
cmake .
make

# Perform Unit Tests
ulimit -c unlimited -S
./picoquic_ct -n -r

Test Results Summary

Starting test number 0, connection_id_print
    Success.
Starting test number 1, connection_id_parse
    Success.
Starting test number 2, sprintf
    Success.
Starting test number 3, memcmp
    Success.
Starting test number 4, threading
    Success.
Starting test number 5, picohash
    Success.
Starting test number 6, bytestream
    Success.
Starting test number 7, splay
    Success.
Starting test number 8, cnxcreation
    Success.
Starting test number 9, parseheader
    Success.
Starting test number 10, incoming_initial
    Success.
Starting test number 11, header_length
    Success.
Starting test number 12, pn2pn64
    Success.
Starting test number 13, intformat
    Success.
Starting test number 14, varint
    Success.
Starting test number 15, ack_sack
    Success.
Starting test number 16, skip_frames
    Success.
Starting test number 17, parse_frames
    Success.
Starting test number 18, logger
    Success.
Starting test number 19, binlog
    Success.
Starting test number 20, app_message_overflow
    Success.
Starting test number 21, TlsStreamFrame
    Success.
Starting test number 22, StreamZeroFrame
    Success.
Starting test number 23, stream_splay
    Success.
Starting test number 24, stream_output
    Success.
Starting test number 25, stream_retransmit_copy
    Success.
Starting test number 26, stream_retransmit_format
    Success.
Starting test number 27, stateless_blowback
    Success.
Starting test number 28, ack_send
    Success.
Starting test number 29, ack_range
    Success.
Starting test number 30, ack_disorder
    Success.
Starting test number 31, ack_horizon
    Success.
Starting test number 32, ack_of_ack
    Success.
Starting test number 33, sim_link
    Success.
Starting test number 34, clear_text_aead
    Success.
Starting test number 35, pn_ctr
    Success.
Starting test number 36, cleartext_pn_enc
    Success.
Starting test number 37, cid_for_lb
    Success.
Starting test number 38, cid_for_lb_cli
    Success.
Starting test number 39, retry_protection_vector
    Success.
Starting test number 40, draft17_vector
    Success.
Starting test number 41, esni
    Success.
Starting test number 42, pn_enc_1rtt
    Success.
Starting test number 43, new_cnxid_stash
    Success.
Starting test number 44, new_cnxid
    Success.
Starting test number 45, pacing
    Success.
Starting test number 46, tls_api
    Success.
Starting test number 47, tls_api_inject_hs_ack
    Success.
Starting test number 48, null_sni
    Success.
Starting test number 49, silence_test
    Success.
Starting test number 50, version_negotiation
    Success.
Starting test number 51, version_invariant
    Success.
Starting test number 52, version_negotiation_spoof
    Success.
Starting test number 53, first_loss
    Success.
Starting test number 54, second_loss
    Success.
Starting test number 55, SH_loss
    Success.
Starting test number 56, client_losses
    Success.
Starting test number 57, server_losses
    Success.
Starting test number 58, many_losses
    Success.
Starting test number 59, datagram
    Success.
Starting test number 60, datagram_rt
    Success.
Starting test number 61, datagram_loss
    Success.
Starting test number 62, datagram_size
    Success.
Starting test number 63, datagram_small
    Success.
Starting test number 64, ddos_amplification
    Success.
Starting test number 65, ddos_amplification_0rtt
    Success.
Starting test number 66, ddos_amplification_8k
    Success.
Starting test number 67, blackhole
    Fails, error: -1.
Starting test number 68, no_ack_frequency
    Success.
Starting test number 69, connection_drop
    Success.
Starting test number 70, transport_param_stream_id
    Success.
Starting test number 71, vn_tp
    Success.
Starting test number 72, vn_compat
    Success.
Starting test number 73, stream_rank
    Success.
Starting test number 74, stream_id_to_rank
    Success.
Starting test number 75, transport_param
    Success.
Starting test number 76, tls_api_sni
    Success.
Starting test number 77, tls_api_alpn
    Success.
Starting test number 78, tls_api_wrong_alpn
    Success.
Starting test number 79, tls_api_oneway_stream
    Success.
Starting test number 80, tls_api_q_and_r_stream
    Success.
Starting test number 81, tls_api_q2_and_r2_stream
    Success.
Starting test number 82, stateless_reset
    Success.
Starting test number 83, stateless_reset_bad
    Success.
Starting test number 84, stateless_reset_client
    Success.
Starting test number 85, stateless_reset_handshake
    Success.
Starting test number 86, tls_api_very_long_stream
    Success.
Starting test number 87, tls_api_very_long_max
    Success.
Starting test number 88, tls_api_very_long_with_err
    Success.
Starting test number 89, tls_api_very_long_congestion
    Fails, error: -1.
Starting test number 90, many_short_loss
    Success.
Starting test number 91, retry
    Success.
Starting test number 92, retry_large
    Success.
Starting test number 93, retry_token
    Success.
Starting test number 94, retry_token_valid
    Success.
Starting test number 95, two_connections
    Success.
Starting test number 96, multiple_versions
    Success.
Starting test number 97, keep_alive
    Success.
Starting test number 98, integrity_limit
    Success.
Starting test number 99, excess_repeat
    Fails, error: -1.
Starting test number 100, netperf_basic
    Success.
Starting test number 101, netperf_bbr
    Success.
Starting test number 102, nat_attack
    Success.
Starting test number 103, sockets
    Success.
Starting test number 104, socket_ecn
    Success.
Starting test number 105, ticket_store
    Success.
Starting test number 106, ticket_seed
    Fails, error: -1.
Starting test number 107, ticket_seed_from_bdp_frame
    Fails, error: -1.
Starting test number 108, token_store
    Success.
Starting test number 109, token_reuse_api
    Success.
Starting test number 110, session_resume
    Success.
Starting test number 111, zero_rtt
    Success.
Starting test number 112, zero_rtt_loss
    Success.
Starting test number 113, stop_sending
    Fails, error: -1.
Starting test number 114, unidir
    Success.
Starting test number 115, mtu_discovery
    Success.
Starting test number 116, mtu_blocked
    Success.
Starting test number 117, mtu_delayed
    Success.
Starting test number 118, mtu_required
    Success.
Starting test number 119, mtu_max
    Success.
Starting test number 120, mtu_drop
    Fails, error: -1.
Starting test number 121, red_cc
    Fails, error: -1.
Starting test number 122, multi_segment
    Fails, error: -1.
Starting test number 123, pacing_cc
    Fails, error: -1.
Starting test number 124, spurious_retransmit
    Success.
Starting test number 125, tls_zero_share
    Success.
Starting test number 126, transport_param_log
    Success.
Starting test number 127, bad_certificate
    Success.
Starting test number 128, set_verify_certificate_callback_test
    Success.
Starting test number 129, virtual_time
    Success.
Starting test number 130, different_params
    Success.
Starting test number 131, quant_params
    Success.
Starting test number 132, set_certificate_and_key
    Success.
Starting test number 133, request_client_authentication
    Success.
Starting test number 134, bad_client_certificate
    Success.
Starting test number 135, nat_rebinding
    Success.
Starting test number 136, nat_rebinding_loss
    Success.
Starting test number 137, nat_rebinding_zero
    Success.
Starting test number 138, nat_rebinding_latency
    Success.
Starting test number 139, nat_rebinding_fast
    Fails, error: -1.
Starting test number 140, spin_bit
    Fails, error: -1.
Starting test number 141, loss_bit
    Success.
Starting test number 142, client_error
    Success.
Starting test number 143, client_only
    Success.
Starting test number 144, packet_enc_dec
    Success.
Starting test number 145, pn_vector
    Success.
Starting test number 146, zero_rtt_spurious
    Success.
Starting test number 147, zero_rtt_retry
    Success.
Starting test number 148, zero_rtt_no_coal
    Success.
Starting test number 149, zero_rtt_many_losses
    Success.
Starting test number 150, zero_rtt_long
    Success.
Starting test number 151, zero_rtt_delay
    Success.
Starting test number 152, random_tester
    Success.
Starting test number 153, random_gauss
    Success.
Starting test number 154, random_public_tester
    Success.
Starting test number 155, cnxid_transmit
    Success.
Starting test number 156, cnxid_transmit_disable
    Success.
Starting test number 157, cnxid_transmit_r_before
    Success.
Starting test number 158, cnxid_transmit_r_disable
    Success.
Starting test number 159, probe_api
    Success.
Starting test number 160, migration
    Success.
Starting test number 161, migration_long
    Success.
Starting test number 162, migration_with_loss
    Success.
Starting test number 163, migration_zero
    Success.
Starting test number 164, migration_fail
    Success.
Starting test number 165, preferred_address
    Success.
Starting test number 166, preferred_address_dis_mig
    Success.
Starting test number 167, preferred_address_zero
    Success.
Starting test number 168, cnxid_renewal
    Success.
Starting test number 169, retire_cnxid
    Success.
Starting test number 170, not_before_cnxid
    Success.
Starting test number 171, server_busy
    Success.
Starting test number 172, initial_close
    Success.
Starting test number 173, initial_server_close
    Success.
Starting test number 174, new_rotated_key
    Success.
Starting test number 175, key_rotation
    Fails, error: 1064.
Starting test number 176, key_rotation_server
    Fails, error: -1.
Starting test number 177, key_rotation_client
    Success.
Starting test number 178, false_migration
    Success.
Starting test number 179, nat_handshake
    Success.
Starting test number 180, key_rotation_vector
    Success.
Starting test number 181, key_rotation_stress
    Success.
Starting test number 182, short_initial_cid
    Success.
Starting test number 183, stream_id_max
    Success.
Starting test number 184, padding_test
    Success.
Starting test number 185, packet_trace
    Fails, error: -1.
Starting test number 186, qlog_trace
    Fails, error: -1.
Starting test number 187, qlog_trace_auto
    Fails, error: -1.
Starting test number 188, qlog_trace_only
    Fails, error: -1.
Starting test number 189, qlog_trace_ecn
    Fails, error: -1.
Starting test number 190, path_packet_queue
    Success.
Starting test number 191, perflog
    Fails, error: -1.
Starting test number 192, nat_rebinding_stress
    Success.
Starting test number 193, random_padding
    Success.
Starting test number 194, ec00_zero
    Success.
Starting test number 195, ec2f_second_flight
    Success.
Starting test number 196, eccf_corrupted_fuzz
    Success.
Starting test number 197, eca1_amplification_loss
    Success.
Starting test number 198, ecf1_final_loss
    Success.
Starting test number 199, ec5c_silly_cid
    Success.
Starting test number 200, ec9a_preemptive_amok
    Success.
Starting test number 201, error_reason
    Success.
Starting test number 202, ready_to_send
    Fails, error: -1.
Starting test number 203, ready_to_skip
    Fails, error: -1.
Starting test number 204, ready_to_zfin
    Fails, error: -1.
Starting test number 205, ready_to_zero
    Fails, error: -1.
Starting test number 206, cubic
    Fails, error: -1.
Starting test number 207, cubic_jitter
    Fails, error: -1.
Starting test number 208, fastcc
    Fails, error: -1.
Starting test number 209, fastcc_jitter
    Fails, error: -1.
Starting test number 210, bbr
    Fails, error: -1.
Starting test number 211, bbr_jitter
    Fails, error: -1.
Starting test number 212, bbr_long
    Success.
Starting test number 213, bbr_performance
    Fails, error: -1.
Starting test number 214, bbr_slow_long
    Fails, error: -1.
Starting test number 215, bbr_one_second
    Fails, error: -1.
Starting test number 216, bbr_gbps
    Success.
Starting test number 217, bbr_asym100
    Fails, error: -1.
Starting test number 218, bbr_asym100_nodelay
    Fails, error: -1.
Starting test number 219, bbr_asym400
    Fails, error: -1.
Starting test number 220, long_rtt
    Success.
Starting test number 221, high_latency_basic
    Success.
Starting test number 222, high_latency_bbr
    Fails, error: -1.
Starting test number 223, high_latency_cubic
    Success.
Starting test number 224, high_latency_probeRTT
    Fails, error: -1.
Starting test number 225, satellite_basic
    Fails, error: -1.
Starting test number 226, satellite_seeded
    Fails, error: -1.
Starting test number 227, satellite_loss

Questions

1. Have I made an error in the compilation picoquic-dpdk?
2. Do I have to configure something else, so that the client and the tests work?

[tbarbette]

We did not run the testsuite; I'm not even sure the testsuite was working before using DPDK as we used some random base git commit.

The DPDK version cannot run locally because you need a real device to connect to DPDK. Are you doing this for an academic work?

[demartinomarco]

Thank you for your reply! Yes, analyzing the performance of picoquic-dpdk will be part of my bachelor thesis (If I can get it running).

How can I run the DPDK version?

[tbarbette]

You have a [newClient.sh](https://github.com/IPNetworkingLab/picoquic-dpdk/blob/main/newClient.sh) and newServer.sh. Have you read Nikita's master thesis? It explains much needed information.

[demartinomarco]

No, I have not read it, since I do not have access to it (This is the link I tried https://dial.uclouvain.be/downloader/downloader.php?pid=thesis%3A37954&datastream=PDF_01&cover=cover-mem)

Both newClient.sh and newServer.sh give me the error:
Invalid port: (null)

[tbarbette]

From there ? https://thesis.dial.uclouvain.be/entities/masterthesis/bb17984a-35e4-4ac9-a3b4-26b8be80b6f8

You should A) be proficient with DPDK and B) be proficient with picoquic before trying this. Please try again when you're at ease with both. For DPDK you want a setup with two machines etc.