Improve reliability of server_key_algorithms test
#66
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
To make this work, have the test programs flush stdout before abort. stdout is block-buffered if it is a pipe.
Each case takes ~1s, so running them in series is quite slow. This also increases the failure rate.
ctz
force-pushed
the
jbp-server-algorithms-test
branch
from
a571635 to
4875185
Compare
cpu
approved these changes
Apr 23, 2025
Member
cpu
left a comment
cpu
left a comment
There was a problem hiding this comment.
The root cause here was the test runner choosing a port for the test servers to run on, but losing when that port was used by something else.
Ahhh yup, that'll do it 🤦♂️ Thanks for catching that.
djc
approved these changes
Apr 23, 2025
tests/runner.rs
Outdated
| .args([ | ||
| "-v", | ||
| "--header", | ||
| "Host: localhost:port", |
Member
There was a problem hiding this comment.
What does this do? Seems surprising to have port literally here?
Member
Author
There was a problem hiding this comment.
Removed the :port for clarity. Because tests/server.c dumps all the data it receives from curl, and because we check its output matches (whether run against openssl or rustls libssl), we need the data sent by curl to be determinstic. This commit stopped that being true, because the server port now varies between runs.
Formerly, the test runner chose a port for each server invocation, but this wasn't robust if the port it chose was already in use. Now the test runner asks for port zero, which means the kernel chooses an ephemeral port that is free. The server then prints the port it is listening on, and the runner parses that out. Remove the now-variable port sent by curl in the `Host:` header.
ctz
force-pushed
the
jbp-server-algorithms-test
branch
from
4875185 to
1bdf3e1
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The root cause here was the test runner choosing a port for the test servers to run on, but losing when that port was used by something else. Now the test server relies on the kernel to choose a free port.
fixes #37