Conversation
Reviewer's guide (collapsed on small PRs)Reviewer's GuideThis PR enhances GPU support by adding conditional host /dev/dri mapping in the Makefile, parameterizes the target architecture for selecting the correct Llama server binary, and standardizes Docker image references to docker.io for improved Podman and Docker compatibility. Class diagram for Makefile variables and Dockerfile ARGsclassDiagram
class Makefile {
BASE_IMAGE : string
DOCKER_IMAGE : string
PORT : int
MODELS_PATH : string
GPU_DEVICE_FLAG : string
LLAMA_ARGS : string
}
class Dockerfile {
GO_VERSION : string
LLAMA_SERVER_VERSION : string
LLAMA_SERVER_VARIANT : string
TARGETARCH : string
LLAMA_BINARY_PATH : string
BASE_IMAGE : string
}
Makefile --> Dockerfile : "Uses Dockerfile ARGs in build/run"
Makefile : +docker-run()
Dockerfile : +builder stage
Dockerfile : +llama-server stage
Dockerfile : +final stage
File-Level Changes
Tips and commandsInteracting with Sourcery
Customizing Your ExperienceAccess your dashboard to:
Getting Help
|
Summary of ChangesHello @ericcurtin, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request enhances the build and runtime environment by enabling automatic GPU detection and utilization within Docker containers, and improves container runtime compatibility by explicitly defining image registry paths in the Dockerfile. These changes aim to streamline development and debugging, particularly when comparing GPU access behavior between Docker and Podman. Highlights
Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here. You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension. Footnotes
|
There was a problem hiding this comment.
Pull Request Overview
This PR enables GPU access for the Docker container when GPU devices are available and includes podman-compatibility improvements. The changes allow the application to leverage GPU acceleration when the hardware is present while maintaining compatibility with different container runtimes.
Key changes:
- Auto-detection of GPU devices in the Makefile to conditionally pass GPU flags
- Addition of podman-compatible image references in the Dockerfile
- Architecture variable initialization for better cross-platform support
Reviewed Changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
| Makefile | Adds GPU device detection and conditional GPU flag passing to docker run command |
| Dockerfile | Updates image references for podman compatibility and adds TARGETARCH variable initialization |
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
![sourcery-ai[bot]](https://avatars.githubusercontent.com/in/48477?s=60&v=4){kind=small}
There was a problem hiding this comment.
Hey there - I've reviewed your changes and they look great!
Prompt for AI Agents
Please address the comments from this code review:
## Individual Comments
### Comment 1
<location> `Makefile:10` </location>
<code_context>
DOCKER_IMAGE := docker/model-runner:latest
PORT := 8080
MODELS_PATH := $(shell pwd)/models-store
+GPU_DEVICE_FLAG := $(shell if [ -d /dev/dri ]; then echo "--device /dev/dri"; fi)
LLAMA_ARGS ?=
</code_context>
<issue_to_address>
**suggestion:** GPU_DEVICE_FLAG only checks for /dev/dri, which may not cover all GPU device scenarios.
This logic may not detect all GPU types, such as NVIDIA devices under /dev/nvidia*. Please update to support a wider range of GPU hardware.
```suggestion
GPU_DEVICE_FLAG := $(shell \
FLAGS=""; \
if [ -d /dev/dri ]; then FLAGS="$$FLAGS --device /dev/dri"; fi; \
if ls /dev/nvidia* 1>/dev/null 2>&1; then \
for dev in /dev/nvidia*; do \
FLAGS="$$FLAGS --device $$dev"; \
done; \
fi; \
echo "$$FLAGS" \
)
```
</issue_to_address>Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request introduces GPU support by passing through the device to the Docker container and improves podman compatibility by using fully qualified image names in the Dockerfile. The changes are a good step forward. My main feedback is to make the GPU detection in the Makefile more robust by adding support for NVIDIA GPUs, which are common for AI/ML workloads. The current implementation only supports GPUs using /dev/dri (like Intel/AMD). I've provided a specific suggestion to improve this.
ericcurtin
force-pushed
the
Makefile-changes
branch
2 times, most recently
from
eefca6e to
77999fe
Compare
There was a problem hiding this comment.
Pull Request Overview
Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
ericcurtin
force-pushed
the
Makefile-changes
branch
from
77999fe to
fb5d842
Compare
ericcurtin
force-pushed
the
Makefile-changes
branch
from
fb5d842 to
7d01be2
Compare
There was a problem hiding this comment.
Pull Request Overview
Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
ericcurtin
force-pushed
the
Makefile-changes
branch
from
7d01be2 to
e466cf4
Compare
ericcurtin
force-pushed
the
Makefile-changes
branch
from
e466cf4 to
5bbc084
Compare
There was a problem hiding this comment.
Pull Request Overview
Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
ericcurtin
force-pushed
the
Makefile-changes
branch
from
5bbc084 to
8e82338
Compare
ericcurtin
force-pushed
the
Makefile-changes
branch
from
8e82338 to
ce3b16d
Compare
There was a problem hiding this comment.
Pull Request Overview
Copilot reviewed 3 out of 3 changed files in this pull request and generated no new comments.
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
Also some podman-compatibility fixes, trying to compare why GPU access works in podman and not docker and vice versa for debugging reasons. Also, make "docker run" a shell script, mainly so we can run shellcheck on it. Signed-off-by: Eric Curtin <eric.curtin@docker.com>
ericcurtin
force-pushed
the
Makefile-changes
branch
from
ce3b16d to
f741826
Compare
|
This ready for re-review @doringeman |
|
Tiny fix here: #187. |
Also some podman-compatibility fixes, trying to compare why GPU access works in podman and not docker and vice versa for debugging reasons.
Summary by Sourcery
Enable GPU device passthrough in the Makefile and improve podman compatibility in the Dockerfile
Enhancements: