Skip to content

server : speed up tests #15836

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 5 commits into from
Sep 6, 2025
Merged

server : speed up tests #15836

merged 5 commits into from
Sep 6, 2025

Conversation

@ngxson
Copy link
Collaborator

@ngxson ngxson commented Sep 6, 2025 •
edited
Loading

In this PR:

  • Speed up server tests by firstly loading all models, then use --offline flag to bypass model file validation
  • Clean up some abusive usages of server.start(CUSTOM_TIMEOUT)
  • Bump default timeout to 60s to make everyone happy
master:
./tools/server/tests/tests.sh  ... 6:28.06 total

PR:
./tools/server/tests/tests.sh  ... 4:35.78 total

@github-actions github-actions bot added script Script related examples python python script changes server labels Sep 6, 2025
@ngxson
Copy link
Collaborator Author

ngxson commented Sep 6, 2025

Hmm it doesn't seem to be faster on the CI. Probably because internet connection on CI is already good.

But I think this can still be an useful thing. Sometimes CI fails due to HF server timeout. This PR resolves this problem by sending less requests to HF server

@ngxson ngxson requested a review from ggerganov September 6, 2025 12:25
@ngxson ngxson merged commit 3c3635d into master Sep 6, 2025
10 checks passed
walidbr pushed a commit to walidbr/llama.cpp that referenced this pull request Sep 7, 2025
@walidbr
server : speed up tests (ggml-org#15836)
8084099 
* server : speed up tests

* clean up

* restore timeout_seconds in some places

* flake8

* explicit offline
njsyw1997 pushed a commit to aizip/llama.cpp that referenced this pull request Sep 10, 2025
@ngxson @njsyw1997
server : speed up tests (ggml-org#15836)
37b254a 
* server : speed up tests

* clean up

* restore timeout_seconds in some places

* flake8

* explicit offline
@ngxson ngxson deleted the xsn/server_speedup_test branch October 5, 2025 11:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@ggerganov ggerganov ggerganov approved these changes

Assignees

No one assigned

Labels

examples python python script changes script Script related server

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@ngxson @ggerganov