feat(perf): migrate python perf image config from test-plans #808#37
feat(perf): migrate python perf image config from test-plans #808#37
Conversation
(cherry picked from commit 285b6d3b7d3e4004e19975ae4426a07b27dd7d5f)
|
Hi @seetadev, this was the same on the old test-plans I checked the failing run and pulled the artifacts/logs for Result summary
Passing cases
Failing cases
Direct failure reason
So this is a genuine test-execution failure (timeouts + one QUIC runtime error), not an artifact upload/workflow issue. |
It sounds like that is incorrect. We shouldn't fail if an impl is super slow and it times out. However 300s is 5 minutes...any idea what is taking so long? I'm ok with doubling this but if that's not long enough I think py-libp2p needs some optimization work. |
|
|
@acul71 the issues are deep in python. I ran the perf test with Here is the (tcp, tls, yamux) log file: So you can see that with 1 MB transfers it completes all 10 iterations and then times out anyway. Something else is going on other than just being slow. |
…n-perf Made-with: Cursor # Conflicts: # perf/images.yaml
I've just discovered that increasing the # Larger default window reduces flow-control churn on high-throughput links.
DEFAULT_WINDOW_SIZE = 16 * 1024 * 1024But I discovered that number with testing, I don't know what the full implications are of this mod. For I'm going to review the About |
Pin perf Python tests to the intended py-libp2p snapshot and avoid docker compose project/container name collisions between reruns by using a per-test-pass project suffix. Made-with: Cursor
|
Why this change was needed:
What the fix does:
Result:
|
|
In CI/CD this test is failing for timeout python-v0_x_x_python-v0_x__ws__tls__mplex__4a514541_dialer | Connected to listener
python-v0_x_x_python-v0_x__ws__tls__mplex__4a514541_dialer | Upload 1/10: 0.43 Gbps
python-v0_x_x_python-v0_x__ws__tls__mplex__4a514541_dialer | Upload 2/10: 0.43 Gbps
python-v0_x_x_python-v0_x__ws__tls__mplex__4a514541_dialer | Upload 3/10: 0.43 Gbps
python-v0_x_x_python-v0_x__ws__tls__mplex__4a514541_dialer | Upload 4/10: 0.43 Gbps
python-v0_x_x_python-v0_x__ws__tls__mplex__4a514541_dialer | Upload 5/10: 0.43 Gbps
python-v0_x_x_python-v0_x__ws__tls__mplex__4a514541_dialer | Upload 6/10: 0.43 Gbps
python-v0_x_x_python-v0_x__ws__tls__mplex__4a514541_dialer | Upload 7/10: 0.43 Gbps
python-v0_x_x_python-v0_x__ws__tls__mplex__4a514541_dialer | Upload 8/10: 0.43 Gbps
python-v0_x_x_python-v0_x__ws__tls__mplex__4a514541_dialer | Upload 9/10: 0.43 Gbps
python-v0_x_x_python-v0_x__ws__tls__mplex__4a514541_dialer | Upload 10/10: 0.43 Gbps
python-v0_x_x_python-v0_x__ws__tls__mplex__4a514541_dialer | Download 1/10: 0.49 Gbps
python-v0_x_x_python-v0_x__ws__tls__mplex__4a514541_dialer | Download 2/10: 0.50 Gbps
python-v0_x_x_python-v0_x__ws__tls__mplex__4a514541_dialer | Download 3/10: 0.49 Gbps
python-v0_x_x_python-v0_x__ws__tls__mplex__4a514541_dialer | Download 4/10: 0.49 Gbps
python-v0_x_x_python-v0_x__ws__tls__mplex__4a514541_dialer | Download 5/10: 0.50 Gbps
Gracefully stopping... (press Ctrl+C again to force)in my linux-box works (faster hardware I think) python-v0_x_x_python-v0_x__ws__tls__mplex__216ffd9a_dialer | Connected to listener
python-v0_x_x_python-v0_x__ws__tls__mplex__216ffd9a_dialer | Upload 1/10: 1.05 Gbps
python-v0_x_x_python-v0_x__ws__tls__mplex__216ffd9a_dialer | Upload 2/10: 1.10 Gbps
python-v0_x_x_python-v0_x__ws__tls__mplex__216ffd9a_dialer | Upload 3/10: 1.07 Gbps
python-v0_x_x_python-v0_x__ws__tls__mplex__216ffd9a_dialer | Upload 4/10: 1.04 Gbps
python-v0_x_x_python-v0_x__ws__tls__mplex__216ffd9a_dialer | Upload 5/10: 1.06 Gbps
python-v0_x_x_python-v0_x__ws__tls__mplex__216ffd9a_dialer | Upload 6/10: 0.99 Gbps
python-v0_x_x_python-v0_x__ws__tls__mplex__216ffd9a_dialer | Upload 7/10: 0.91 Gbps
python-v0_x_x_python-v0_x__ws__tls__mplex__216ffd9a_dialer | Upload 8/10: 0.99 Gbps
python-v0_x_x_python-v0_x__ws__tls__mplex__216ffd9a_dialer | Upload 9/10: 1.01 Gbps
python-v0_x_x_python-v0_x__ws__tls__mplex__216ffd9a_dialer | Upload 10/10: 1.07 Gbps
python-v0_x_x_python-v0_x__ws__tls__mplex__216ffd9a_dialer | Download 1/10: 1.23 Gbps
python-v0_x_x_python-v0_x__ws__tls__mplex__216ffd9a_dialer | Download 2/10: 1.23 Gbps
python-v0_x_x_python-v0_x__ws__tls__mplex__216ffd9a_dialer | Download 3/10: 1.26 Gbps
python-v0_x_x_python-v0_x__ws__tls__mplex__216ffd9a_dialer | Download 4/10: 1.22 Gbps
python-v0_x_x_python-v0_x__ws__tls__mplex__216ffd9a_dialer | Download 5/10: 1.28 Gbps
python-v0_x_x_python-v0_x__ws__tls__mplex__216ffd9a_dialer | Download 6/10: 1.23 Gbps
python-v0_x_x_python-v0_x__ws__tls__mplex__216ffd9a_dialer | Download 7/10: 1.21 Gbps
python-v0_x_x_python-v0_x__ws__tls__mplex__216ffd9a_dialer | Download 8/10: 1.28 Gbps
python-v0_x_x_python-v0_x__ws__tls__mplex__216ffd9a_dialer | Download 9/10: 1.23 Gbps
python-v0_x_x_python-v0_x__ws__tls__mplex__216ffd9a_dialer | Download 10/10: 1.25 Gbps
python-v0_x_x_python-v0_x__ws__tls__mplex__216ffd9a_dialer | Latency iterations done
python-v0_x_x_python-v0_x__ws__tls__mplex__216ffd9a_dialer | upload:
python-v0_x_x_python-v0_x__ws__tls__mplex__216ffd9a_dialer | iterations: 10
python-v0_x_x_python-v0_x__ws__tls__mplex__216ffd9a_dialer | min: 0.91
python-v0_x_x_python-v0_x__ws__tls__mplex__216ffd9a_dialer | q1: 1.00
python-v0_x_x_python-v0_x__ws__tls__mplex__216ffd9a_dialer | median: 1.05
python-v0_x_x_python-v0_x__ws__tls__mplex__216ffd9a_dialer | q3: 1.07
python-v0_x_x_python-v0_x__ws__tls__mplex__216ffd9a_dialer | max: 1.10
python-v0_x_x_python-v0_x__ws__tls__mplex__216ffd9a_dialer | outliers: []
python-v0_x_x_python-v0_x__ws__tls__mplex__216ffd9a_dialer | samples: [0.91, 0.99, 0.99, 1.01, 1.04, 1.05, 1.06, 1.07, 1.07, 1.1]
python-v0_x_x_python-v0_x__ws__tls__mplex__216ffd9a_dialer | unit: Gbps
python-v0_x_x_python-v0_x__ws__tls__mplex__216ffd9a_dialer | download:
python-v0_x_x_python-v0_x__ws__tls__mplex__216ffd9a_dialer | iterations: 10
python-v0_x_x_python-v0_x__ws__tls__mplex__216ffd9a_dialer | min: 1.21
python-v0_x_x_python-v0_x__ws__tls__mplex__216ffd9a_dialer | q1: 1.23
python-v0_x_x_python-v0_x__ws__tls__mplex__216ffd9a_dialer | median: 1.23
python-v0_x_x_python-v0_x__ws__tls__mplex__216ffd9a_dialer | q3: 1.26
python-v0_x_x_python-v0_x__ws__tls__mplex__216ffd9a_dialer | max: 1.28
python-v0_x_x_python-v0_x__ws__tls__mplex__216ffd9a_dialer | outliers: []
python-v0_x_x_python-v0_x__ws__tls__mplex__216ffd9a_dialer | samples: [1.21, 1.22, 1.23, 1.23, 1.23, 1.23, 1.25, 1.26, 1.28, 1.28]
python-v0_x_x_python-v0_x__ws__tls__mplex__216ffd9a_dialer | unit: Gbps
python-v0_x_x_python-v0_x__ws__tls__mplex__216ffd9a_dialer | latency:
python-v0_x_x_python-v0_x__ws__tls__mplex__216ffd9a_dialer | iterations: 100
python-v0_x_x_python-v0_x__ws__tls__mplex__216ffd9a_dialer | min: 1.310
python-v0_x_x_python-v0_x__ws__tls__mplex__216ffd9a_dialer | q1: 1.804
python-v0_x_x_python-v0_x__ws__tls__mplex__216ffd9a_dialer | median: 2.266
python-v0_x_x_python-v0_x__ws__tls__mplex__216ffd9a_dialer | q3: 2.831
python-v0_x_x_python-v0_x__ws__tls__mplex__216ffd9a_dialer | max: 4.360
python-v0_x_x_python-v0_x__ws__tls__mplex__216ffd9a_dialer | outliers: [4.952]
python-v0_x_x_python-v0_x__ws__tls__mplex__216ffd9a_dialer | samples: [1.31, 1.322, 1.34, 1.345, 1.365, 1.377, 1.378, 1.393, 1.468, 1.481, 1.514, 1.567, 1.625, 1.635, 1.635, 1.652, 1.653, 1.669, 1.697, 1.739, 1.739, 1.757, 1.77, 1.775, 1.782, 1.811, 1.831, 1.855, 1.873, 1.904, 1.923, 1.936, 1.937, 1.956, 1.96, 2.005, 2.005, 2.006, 2.062, 2.178, 2.189, 2.191, 2.217, 2.221, 2.229, 2.235, 2.249, 2.252, 2.262, 2.265, 2.268, 2.282, 2.283, 2.297, 2.322, 2.326, 2.346, 2.36, 2.366, 2.384, 2.4, 2.477, 2.486, 2.487, 2.51, 2.519, 2.549, 2.577, 2.578, 2.587, 2.66, 2.746, 2.759, 2.762, 2.83, 2.833, 2.847, 2.908, 2.952, 2.958, 2.996, 3.083, 3.099, 3.12, 3.121, 3.143, 3.212, 3.221, 3.247, 3.319, 3.402, 3.421, 3.66, 3.669, 3.735, 3.959, 4.103, 4.187, 4.36, 4.952]
python-v0_x_x_python-v0_x__ws__tls__mplex__216ffd9a_dialer | unit: ms
python-v0_x_x_python-v0_x__ws__tls__mplex__216ffd9a_dialer exited with code 0
Compose Stopping Aborting on container exit...
Container python-v0_x_x_python-v0_x__ws__tls__mplex__216ffd9a_dialer Stopping
Container python-v0_x_x_python-v0_x__ws__tls__mplex__216ffd9a_dialer Stopped
Container python-v0_x_x_python-v0_x__ws__tls__mplex__216ffd9a_listener Stopping
Container python-v0_x_x_python-v0_x__ws__tls__mplex__216ffd9a_listener Stopped
python-v0_x_x_python-v0_x__ws__tls__mplex__216ffd9a_listener exited with code 143
[2026-03-07 04:40:13] INFO: ✓ Test complete
Container python-v0_x_x_python-v0_x__ws__tls__mplex__216ffd9a_dialer Stopping
Container python-v0_x_x_python-v0_x__ws__tls__mplex__216ffd9a_dialer Stopped
Container python-v0_x_x_python-v0_x__ws__tls__mplex__216ffd9a_dialer Removing
Container python-v0_x_x_python-v0_x__ws__tls__mplex__216ffd9a_dialer Removed
Container python-v0_x_x_python-v0_x__ws__tls__mplex__216ffd9a_listener Stopping
Container python-v0_x_x_python-v0_x__ws__tls__mplex__216ffd9a_listener Stopped
Container python-v0_x_x_python-v0_x__ws__tls__mplex__216ffd9a_listener Removing
Container python-v0_x_x_python-v0_x__ws__tls__mplex__216ffd9a_listener Removed |
|
@acul71 it seems like you and @sumanjeet0012 are trying to do the same thing. @sumanjeet0012 has this PR: #52 I think you two should combine efforts. |
Summary
libp2p/test-plans#808intolibp2p/unified-testing.perf/images.yamlto includepython-v0.xin perf aliases and add python implementation configuration for perf runs.Context
Test plan
--test-select "~python".Made with Cursor