Skip to content

feat: python perf - yamux window auto-tuning#62

Open
asabya wants to merge 2 commits intolibp2p:migrate/pr808-python-perffrom
asabya:feat/python-perf-yamux-auto-tuning
Open

feat: python perf - yamux window auto-tuning#62
asabya wants to merge 2 commits intolibp2p:migrate/pr808-python-perffrom
asabya:feat/python-perf-yamux-auto-tuning

Conversation

@asabya
Copy link
Copy Markdown

@asabya asabya commented Mar 13, 2026

Summary

  • Point python-v0.x perf image at asabya/py-libp2p with yamux receive window auto-tuning (256KB → 16MB)
  • Remove commented-out local python config block from images.yaml

Changes

perf/images.yaml: Switch python-v0.x source from libp2p/py-libp2p to asabya/py-libp2p (commit 9a9a4a6) which includes yamux window auto-tuning. Clean up commented-out local config.

Test plan

  • Verify perf Docker image builds successfully from the new commit
  • Run python-v0.x perf tests (tcp + ws, noise + tls, yamux + mplex)
  • Confirm throughput improvement with auto-tuned yamux windows

Point python-v0.x at asabya/py-libp2p with yamux receive window
auto-tuning (256KB→16MB).
@asabya asabya requested a review from dhuseby as a code owner March 13, 2026 08:04
@asabya asabya changed the base branch from master to migrate/pr808-python-perf March 13, 2026 08:05
@asabya
Copy link
Copy Markdown
Author

asabya commented Mar 26, 2026

AI Assisted Debugging:

The PR introduced yamux receive window auto-tuning (256KB → 16MB) to improve throughput. The
window size controls flow control — how much data a sender can have in-flight before waiting
for acknowledgment. Larger windows mean higher throughput, especially on high-latency links.

The problem: in py-libp2p, yamux's write() used send_window as the only limit on frame size.

As the window auto-tuned upward, yamux started sending multi-megabyte frames down to the
security layer. Two things broke:

Failure 1: Noise encryption rejects large payloads (4 tests, instant failure)

The Noise protocol encrypts messages using ChaCha20-Poly1305 with a hard 65535-byte transport
message limit (2-byte length prefix). When yamux sent a frame larger than that, the
noiseprotocol library's encrypt() raised:

NoiseInvalidMessage: Data must be bytes and less or equal 65535 bytes in length

Failure 2: WebSocket + TLS choked on large frames (2 tests, 5-min timeout)

WS+TLS tests didn't crash — they just ran extremely slow (~0.15 Gbps vs expected ~5 Gbps) and
timed out at 300s. The large yamux frames caused excessive buffering and backpressure through
the WebSocket framing layer (4KB default buffer) and TLS record layer (16KB records), creating
a bottleneck.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant