-
Notifications
You must be signed in to change notification settings - Fork 541
test: Add HF Space Dockerfile using pre-built leaderboard image #3838
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
3 commits
Select commit
Hold shift + click to select a range
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,73 @@ | ||
| name: Build HF Space Docker Image | ||
|
|
||
| on: | ||
| push: | ||
| branches: [ main ] | ||
| paths: | ||
| - 'Dockerfile.hf-space' | ||
| - '.github/workflows/hf_space_docker.yml' | ||
| pull_request: | ||
| branches: [ main ] | ||
| paths: | ||
| - 'Dockerfile.hf-space' | ||
| - '.github/workflows/hf_space_docker.yml' | ||
| workflow_dispatch: | ||
|
|
||
| jobs: | ||
| build-hf-space: | ||
| runs-on: ubuntu-latest | ||
|
|
||
| steps: | ||
| - name: Checkout code | ||
| uses: actions/checkout@v4 | ||
|
|
||
| - name: Set up Docker Buildx | ||
| uses: docker/setup-buildx-action@v3 | ||
|
|
||
| - name: Build HF Space Docker image | ||
| run: | | ||
| docker build -f Dockerfile.hf-space -t mteb-hf-space:test . | ||
| echo "✅ Docker image built successfully" | ||
|
|
||
| - name: Test Docker image | ||
| run: | | ||
| # Test that the image can be created and the container starts | ||
| docker run -d --name mteb-test -p 7860:7860 mteb-hf-space:test | ||
|
|
||
| # Give the container a moment to start | ||
| sleep 5 | ||
|
|
||
| # Check if container is running | ||
| if docker ps | grep -q mteb-test; then | ||
| echo "✅ Container is running" | ||
| docker logs mteb-test | ||
| else | ||
| echo "❌ Container failed to start" | ||
| docker logs mteb-test | ||
| exit 1 | ||
| fi | ||
|
|
||
| # Clean up | ||
| docker stop mteb-test | ||
| docker rm mteb-test | ||
|
|
||
| - name: Validate HF Space configuration | ||
| run: | | ||
| # Check that required environment variables and ports are set | ||
| docker run --rm mteb-hf-space:test sh -c ' | ||
| if [ "$GRADIO_SERVER_NAME" = "0.0.0.0" ]; then | ||
| echo "✅ GRADIO_SERVER_NAME is correctly set" | ||
| else | ||
| echo "❌ GRADIO_SERVER_NAME is not set to 0.0.0.0" | ||
| exit 1 | ||
| fi | ||
| ' | ||
|
|
||
| # Check exposed port | ||
| EXPOSED_PORT=$(docker inspect mteb-hf-space:test --format='{{range $key, $value := .Config.ExposedPorts}}{{$key}}{{end}}' | grep -o '7860') | ||
| if [ "$EXPOSED_PORT" = "7860" ]; then | ||
| echo "✅ Port 7860 is exposed" | ||
| else | ||
| echo "❌ Port 7860 is not exposed" | ||
| exit 1 | ||
| fi | ||
isaac-chung marked this conversation as resolved.
Show resolved
Hide resolved
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,6 @@ | ||
| FROM ghcr.io/embeddings-benchmark/mteb/leaderboard:latest | ||
|
|
||
| ENV GRADIO_SERVER_NAME="0.0.0.0" | ||
| EXPOSE 7860 | ||
isaac-chung marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
|
||
| CMD ["make", "run-leaderboard"] | ||
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think this action won't be triggered
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Did you check the right sections? https://github.com/embeddings-benchmark/mteb/actions/runs/20725664296/job/59500466946?pr=3838
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I checked this. This will be triggered only on changed in dockerfile for spaces, but I'm not sure if it will be enough to test then
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think it's better to move these checks to
leaderboard_docker.ymlto test this more frequently and make sure that our leaderboard will workThere was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Correct, that's what the paths section of the workflow file shows. I'm not sure there's new info here.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Keeping these tests separate makes more sense.