driver: mount wsl lib folder for docker-container driver #3320
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
3 tasks
3 tasks
|
Following up on #3295 (comment). |
Thanks for your feedback, can you show the output of |
|
@pechersky Seems you're not using a development version of buildx to test changes of this PR. You can build it on your side with https://github.com/docker/buildx?tab=readme-ov-file#building: After testing don't forget to remove the binary: |
|
Great! After deleting the previous builder, and rerunning, I get the expected output, I think. |
Yes looks good great! |
Signed-off-by: CrazyMax <[email protected]>
|
Strangely, if I rerun the |
Can you show the output of |
|
|
Just to note -- if I delete the builder, then recreate it, then rerun the |
Hum I'm not able to repro. First build: Second build: Third build: I'm not sure what would make |
|
@tonistiigi Btw I think there is an issue with ondemand and autoallow behavior. When I create a builder with nvidia build tag: And inspect it before doing a build I have: After first build then I get: If I restart the Docker Engine and inspecting this builder again I have: I think as the generated CDI specification is loaded from the build, it takes precedence over the OnDemand and therefore the device is not automatically allowed anymore. |
|
Sorry for the noise. I understand now that the |
It's available within the builder container on device request right before the build.
Great 👍 |
tonistiigi
reviewed
Aug 11, 2025
There was a problem hiding this comment.
It seems requesting GPU on container builder creation introduced in #3063 is not enough as some libraries are missing on WSL.
If that is the case, isn't this a bug in Moby if --gpus does not work correctly?
It works but not for nested containers, I will make a repro with dind to make sure of it. |
|
In that case, I think we need to detect the dind environment more precisely. I see you have |
It's not really detecting the dind environment. When we create the container builder and request gpus: buildx/driver/docker-container/driver.go Lines 161 to 163 in c6d1e39 Moby will run the nvidia container runtime prestart hook that checks whether NVIDIA devices are requested and ensures GPU access is configured using the In such case, necessary libraries are mounted and GPU is detected with cc @elezar |
|
As a naive question: Some of the output above referred to @crazy-max in the WSL case, where is the "outer" CDI spec being created (or should it be created)? Is this in the WSL2 distribution? Could a spec be generated For Docker Desktop as the VM is being started? On a related note, I have created moby/moby#50228 as a placeholder for migrating the |
Yes we have the device name format defined here: https://docs.docker.com/reference/dockerfile/#run---device:
We don't handle CDI specs from the host for the container builder case atm. We use the
Yes good point, that might be something Docker Desktop could do.
Ah that looks ideal, I see the |
tonistiigi
approved these changes
Aug 13, 2025
3 tasks
aevesdocker
pushed a commit
to docker/docs
that referenced
this pull request
Aug 26, 2025
follow-up #23069 (comment) ## Related issues or tickets * docker/buildx#3320 ## Reviews <!-- Notes for reviewers here --> <!-- List applicable reviews (optionally @tag reviewers) --> - [ ] Technical review - [ ] Editorial review - [ ] Product review Signed-off-by: CrazyMax <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
relates to #3295 (comment)
When generating the CDI spec during build for a container builder created on WSL it fails with:
It seems requesting GPU on container builder creation introduced in #3063 is not enough as some libraries are missing on WSL.
To fix this issue we can mount the WSL libraries in the container if docker context is local.
Inspecting the CDI spec we can see
libdxcore.sois properly detected and mounted: