Skip to content

[BUG] nvflare poc start gpu flags do not work with docker #4445

@Vithor112

Description

@Vithor112

Describe the bug
Even after running nvflare poc prepare -d [image] -n 2 followed by nvflare poc start -gpu 0, the spawned containers do not have GPU capabilities enabled upon startup.

I am currently using a workaround by setting export GPU2USE='--gpus=0' manually before starting the POC.

To Reproduce
Run: nvflare poc prepare -d -n 2
Run: nvflare poc start -gpu 0
Check for GPU availability inside the container (e.g., check if PyTorch detects CUDA or run nvidia-smi).
Result: GPU is not detected unless the environment variable is manually exported.

Expected behavior
The -gpu flag in the nvflare poc start command should automatically configure the containers with the appropriate GPU capabilities without requiring manual environment variable exports.

Desktop:

  • OS: ubuntu 22.04
  • Python Version 3.10.12
  • NVFlare Version 2.7.2

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions