HTTP 3 Documentation

Building ATS with HTTP/3 support using Docker images

1. Build the Docker image (Work in progress, don't use yet)

Fedora 37

git clone [email protected]:apache/trafficserver-ci.git
cd trafficserver-ci
cd docker/http3/fedora37
sed -i "s/UID=1000/UID=$(id -u)/" Dockerfile
sed -i "s/GID=1000/GID=$(id -g)/" Dockerfile
sed -i "s/username/$(whoami)/" Dockerfile
docker-compose up -d

ssh config

echo -e "\nHost fedora_h3" >> ~/.ssh/config
echo "HostName localhost" >> ~/.ssh/config
echo "Port 666" >> ~/.ssh/config

ssh to Docker

ssh fedora_h3

Jump to Section 3 below and continue with those steps.

---

Building the HTTP/3 tools and libraries form scratch

1. Installing packages to build Traffic Server

Fedora 37

sudo dnf -y install ccache make pkgconfig bison flex gcc-c++ clang autoconf automake libtool sudo git rpm-build distcc-server file wget openssl hwloc nghttp2 libnghttp2-devel sudo git rpm-build distcc-server file wget openssl hwloc nghttp2 libnghttp2-devel sudo git rpm-build distcc-server file wget openssl hwloc nghttp2 libnghttp2-devel ImageMagick-devel ImageMagick-c++-devel hiredis-devel zlib-devel libmaxminddb-devel perl-ExtUtils-MakeMaker perl-Digest-SHA perl-URI curl tcl-devel java python3 httpd-tools procps-ng nmap-ncat python3-pip python3-gunicorn python3-requests python3-devel python3-psutil telnet golang openssl-devel pcre-devel initscripts

Ubuntu

apt-get -y install sudo
sudo apt-get -y install build-essential llvm ccache make pkgconf bison flex g++ clang gettext libc++-dev autoconf automake libtool autotools-dev git distcc file wget openssl hwloc intltool-debian clang-tools-14 clang-14 libssl-dev libexpat1-dev libpcre3-dev libcap-dev libhwloc-dev zlib1g-dev libjemalloc-dev libluajit-5.1-dev liblzma-dev libhiredis-dev libbrotli-dev libncurses-dev libgeoip-dev libmagick++-dev libmaxminddb-dev libcjose-dev libcjose0 libjansson-dev
sudo apt-get -y install libunwind-dev

2. Install package dependencies for HTTP/3 tools

Fedora 37

sudo dnf -y install libev-devel jemalloc-devel python2-devel libxml2-devel c-ares-devel libevent-devel jansson-devel zlib-devel systemd-devel cargo cmake perl-FindBin

Ubuntu

sudo apt -y install libev-dev libjemalloc-dev python2-dev libxml2-dev libpython2-dev libc-ares-dev libsystemd-dev libevent-dev libjansson-dev zlib1g-dev cargo cmake

3. Build and install the HTTP/3 tools and libraries

There will be HTTTP/3 versions of curl and h2load under the /opt/bin directory

git clone [email protected]:apache/trafficserver.git || git clone https://github.com/apache/trafficserver.git
cd trafficserver
git checkout -b 10-Dev origin/10-Dev
cd tools
./build_h3_tools.sh
cd ..

4. Building trafficserver with HTTP/3 support

autoreconf -if
mkdir target
cd target
sudo mkdir -p /opt/ats
USER=$(whoami) sudo chown $USER: /opt/ats

gcc configure

../configure --prefix=/opt/ats --enable-ccache --enable-werror --enable-experimental-plugins --enable-example-plugins --with-quiche=/opt/quiche --enable-expensive-tests

ASAN and clang configure

LUAJIT_CFLAGS=-Wno-unused-command-line-argument CC=/bin/clang CXX=/bin/clang++ CLANG_TIDY=/bin/clang-tidy ../configure --prefix=/opt/ats --enable-ccache --enable-werror --enable-experimental-plugins --enable-example-plugins --with-quiche=/opt/quiche --enable-asan --enable-expensive-tests

build and install

make -j install

5. Configuring trafficserver with HTTP/3 support

You will need to generate TLS certificates (see below) if you don't have any already. Below is an example configuration that will work with the benchmarking example.

ETC_DIR=/opt/ats/etc/trafficserver
RECORDS_CONFIG=$ETC_DIR/records.config
REMAP_CONFIG=$ETC_DIR/remap.config
sed -i 's/8080 8080:ipv6/8080 4443:ssl 4443:quic/' $RECORDS_CONFIG
echo "CONFIG proxy.config.udp.threads INT 1" >> $RECORDS_CONFIG
echo "CONFIG proxy.config.diags.show_location INT 2" >> $RECORDS_CONFIG
echo "CONFIG proxy.config.quic.initial_max_streams_bidi_in INT 100000" >> $RECORDS_CONFIG
echo "CONFIG proxy.config.quic.initial_max_streams_bidi_out INT 100000" >> $RECORDS_CONFIG
sed -i 's/CONFIG proxy.config.http.insert_response_via_str INT 0/CONFIG proxy.config.http.insert_response_via_str INT 3/' $RECORDS_CONFIG
echo map / http://127.0.0.1/ @plugin=generator.so >> $REMAP_CONFIG
echo "dest_ip=* ssl_cert_name=$ETC_DIR/localhost.crt ssl_key_name=$ETC_DIR/localhost.key" >> $ETC_DIR/ssl_multicert.config

6. Generating TLS certificates

Create a certificate configuration file

tee -a apache.conf << EOF
[req]
distinguished_name = req_distinguished_name
x509_extensions = v3_req
prompt = no
[req_distinguished_name]
C = US
ST = CA
L = Mountain View
O = Traffic Server
CN = trafficserver.org
[v3_req]
keyUsage = nonRepudiation, digitalSignature, keyEncipherment
extendedKeyUsage = serverAuth
subjectAltName = @alt_names
[alt_names]
DNS.1 = *.trafficserver.org
EOF

Create the private and public keys and move them to Traffic Server's configuration directory

openssl req -nodes -x509 -newkey rsa:4096 -keyout private-key.pem -out cert.pem -config apache.conf -sha256 -days 730
ETC_DIR=/opt/ats/etc/trafficserver
mv private-key.pem $ETC_DIR/localhost.key
mv cert.pem $ETC_DIR/localhost.crt

7. Benchmarking HTTP/3

Start ATS

/opt/ats/bin/trafficserver start

Test one request

/opt/bin/curl -k --http3 https://127.0.0.1:4443/cache/1024

Run h2load

/opt/bin/h2load -n 500000 -c 100 --npn-list=h3 https://127.0.0.1:4443/cache/1024/asdfasdf

You should see output like this below

starting benchmark...
spawning thread #0: 100 total client(s). 500000 total requests
TLS Protocol: TLSv1.3
Cipher: TLS_AES_128_GCM_SHA256
Server Temp Key: X25519 253 bits
Application protocol: h3
progress: 10% done
progress: 20% done
progress: 30% done
progress: 40% done
progress: 50% done
progress: 60% done
progress: 70% done
progress: 80% done
progress: 90% done
progress: 100% done

finished in 8.53s, 58611.15 req/s, 66.12MB/s
requests: 500000 total, 500000 started, 500000 done, 500000 succeeded, 0 failed, 0 errored, 0 timeout
status codes: 500000 2xx, 0 3xx, 0 4xx, 0 5xx
traffic: 564.07MB (591470968) total, 72.93MB (76470968) headers (space savings 35.47%), 488.28MB (512000000) data
UDP datagram: 753325 sent, 1500566 received
                     min         max         mean         sd        +/- sd
time for request:      132us     86.27ms      1.67ms      1.03ms    91.72%
time for connect:    14.81ms     39.15ms     28.11ms      6.83ms    61.00%
time to 1st byte:    54.53ms    105.88ms     75.15ms     11.71ms    69.00%
req/s           :     586.36      623.81      595.43        8.92    83.00%