[Issue]: Cannot use local docker image when `--pull` is hard coded into docker build

[xinyazhang]

Problem Description

madengine/src/madengine/tools/run_models.py

Line 593 in eca77f7

+ " --pull -f "

Example command

madengine run --tags pyt_deepspeed_megatron_gpt2 --live-output --additional-context "{'guest_os':'UBUNTU', 'docker_build_arg':{'BASE_DOCKER':'compute-artifactory.amd.com:5000/rocm-plus-docker/framework/compute-rocm-rel-6.4:114_ubuntu22.04_py3.10_pytorch_release-2.6_6b7dbfa'}}" --timeout 0 --clean-docker-cache

The command fails with the following logs if replacing the BASE_DOCKER argument with any local image.

#2 [internal] load metadata for docker.io/library/swdev-542227:latest
#2 ERROR: pull access denied, repository does not exist or may require authorization: server message: insufficient_scope: authorization failed                                                                  ------
 > [internal] load metadata for docker.io/library/swdev-542227:latest:
------
pyt_deepspeed_megatron_gpt2.ubuntu.amd.Dockerfile:3
--------------------
   1 |     # CONTEXT {'gpu_vendor': 'AMD', 'guest_os': 'UBUNTU'}
   2 |     ARG BASE_DOCKER=rocm/pytorch:latest
   3 | >>> FROM $BASE_DOCKER
   4 |     ENV WORKSPACE_DIR=/workspace
   5 |     RUN mkdir -p $WORKSPACE_DIR
--------------------
ERROR: failed to solve: swdev-542227:latest: failed to resolve source metadata for docker.io/library/swdev-542227:latest: pull access denied, repository does not exist or may require authorization: server message: insufficient_scope: authorization failed

Confirmed removing --pull can fix the problem.

Operating System

Does not matter

CPU

Does not matter

GPU

Does not matter

ROCm Version

Does not matter

ROCm Component

No response

Steps to Reproduce

No response

(Optional for Linux users) Output of /opt/rocm/bin/rocminfo --support

No response

Additional Information

No response