https://github.com/GoogleCloudPlatform/kubernetes-engine-samples/blob/main/ai-ml/llm-finetuning-gemma/Dockerfile
The dependabot update causes the fine-tune job to fail due to not detecting the GPU.
GKE Autopilot 1.33.x
The updates causes the issue on the following image versions tested:
nvidia/cuda:12.9.1-runtime-ubuntu22.04
nvidia/cuda:12.9.1-runtime-ubuntu24.04
The original version nvidia/cuda:12.2.0-runtime-ubuntu22.04 still works.