You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Hi everyone, recently I ran into a niche and bizzare error when trying to build the ros2 image using isaaclab's container.py script, and would like to share the how I resolved it at the end, just so anyone else out there don't have to spend a full hour debugging this.
Issue
Essentially the error occurs because the line FROM isaac-lab-base${DOCKER_NAME_SUFFIX} AS ros2 in docker/Dockerfile.ros2 can not find isaac-lab-base locally, leading it to try to find it on docker hub and fails due to permission error. As you can see in the terminal output below, isaac-lab-base built successfully, but when the next build stage failed to see this built image locally.
$ ./container.py start ros2
[INFO] Using container profile: ros2
[INFO] X11 Forwarding is configured as '1' in '.container.cfg'.
To disable X11 forwarding, set 'X11_FORWARDING_ENABLED=0' in '.container.cfg'.
[INFO] Building the docker image and starting the container 'isaac-lab-ros2' in the background...
Compose can now delegate builds to bake for better performance.
To do so, set COMPOSE_BAKE=true.
[+] Building 178.7s (20/20) FINISHED docker-container:mybuilder
=> [isaac-lab-base internal] load build definition from Dockerfile.base 0.0s
=> => transferring dockerfile: 4.48kB 0.0s
=> WARN: InvalidDefaultArgInFrom: Default value for ARG ${ISAACSIM_BASE_IMAGE_ARG}:${ISAACSIM_VERSION_ARG} results in empty or invalid base image name (line 12) 0.0s
=> [isaac-lab-base internal] load metadata for nvcr.io/nvidia/isaac-sim:5.0.0 0.8s
=> [isaac-lab-base internal] load .dockerignore 0.0s
=> => transferring context: 487B 0.0s
=> [isaac-lab-base base 1/13] FROM nvcr.io/nvidia/isaac-sim:5.0.0@sha256:499e1fd7a3ffc68c6c6fff466d4a6a6dc9ab1573e0a28dbd5f100501ea64e410 0.0s
=> => resolve nvcr.io/nvidia/isaac-sim:5.0.0@sha256:499e1fd7a3ffc68c6c6fff466d4a6a6dc9ab1573e0a28dbd5f100501ea64e410 0.0s
=> [isaac-lab-base internal] load build context 0.1s
=> => transferring context: 144.27kB 0.1s
=> CACHED [isaac-lab-base base 2/13] RUN --mount=type=cache,target=/var/cache/apt apt-get update && apt-get install -y --no-install-recommends build-essential cmake git libglib2.0-0 0.0s
=> [isaac-lab-base base 3/13] COPY ../ /workspace/isaaclab 0.2s
=> [isaac-lab-base base 4/13] RUN chmod +x /workspace/isaaclab/isaaclab.sh 0.1s
=> [isaac-lab-base base 5/13] RUN ln -sf /isaac-sim /workspace/isaaclab/_isaac_sim 0.1s
=> [isaac-lab-base base 6/13] RUN /workspace/isaaclab/isaaclab.sh -p -m pip install toml 1.0s
=> [isaac-lab-base base 7/13] RUN --mount=type=cache,target=/var/cache/apt /workspace/isaaclab/isaaclab.sh -p /workspace/isaaclab/tools/install_deps.py apt /workspace/isaaclab/source && apt -y a 0.2s
=> [isaac-lab-base base 8/13] RUN mkdir -p /isaac-sim/kit/cache && mkdir -p /root/.cache/ov && mkdir -p /root/.cache/pip && mkdir -p /root/.cache/nvidia/GLCache && mkdir -p /root/.nv/Co 0.1s
=> [isaac-lab-base base 9/13] RUN touch /bin/nvidia-smi && touch /bin/nvidia-debugdump && touch /bin/nvidia-persistenced && touch /bin/nvidia-cuda-mps-control && touch /bin/nvidia-cuda-m 0.1s
=> [isaac-lab-base base 10/13] RUN --mount=type=cache,target=/root/.cache/pip /workspace/isaaclab/isaaclab.sh --install 106.4s
=> [isaac-lab-base base 11/13] RUN /workspace/isaaclab/isaaclab.sh -p -m pip uninstall -y quadprog 0.6s
=> [isaac-lab-base base 12/13] RUN echo "export ISAACLAB_PATH=/workspace/isaaclab" >> ${HOME}/.bashrc && echo "alias isaaclab=/workspace/isaaclab/isaaclab.sh" >> ${HOME}/.bashrc && echo "alias py 0.1s
=> [isaac-lab-base base 13/13] WORKDIR /workspace/isaaclab 0.1s
=> [isaac-lab-base] exporting to docker image format 68.5s
=> => exporting layers 36.9s
=> => exporting manifest sha256:8b2ff34c01f4c8a1ae0437a31c3e4b11cead9a74a58ed909a83b638ee0c7f303 0.0s
=> => exporting config sha256:0587389ca238d52afa750aca47e2b01f7148e30878e1cd01d8a42222cc1de35f 0.0s
=> => sending tarball 31.6s
=> [isaac-lab-base] importing to docker 12.8s
=> => loading layer 578a0e953b62 2.23MB / 2.23MB 12.8s
=> => loading layer 5f70bf18a086 32B / 32B 12.3s
=> => loading layer 5043a07f1ec0 158B / 158B 12.2s
=> => loading layer c34ff5b44d40 3.66MB / 3.66MB 12.2s
=> => loading layer de1a2d9e7af0 31.28kB / 31.28kB 12.0s
=> => loading layer 5807db8dd0f7 377B / 377B 11.9s
=> => loading layer 1a862c3bef2f 309B / 309B 11.9s
=> => loading layer b2884da00d5f 699.31MB / 699.31MB 11.8s
=> => loading layer 77f9e5ca83bf 3.15kB / 3.15kB 0.6s
=> => loading layer d6956ef9177e 1.71kB / 1.71kB 0.6s
=> [isaac-lab-base] resolving provenance for metadata file 0.0s
[+] Building 1/1
✔ isaac-lab-base Built 0.0s
Compose can now delegate builds to bake for better performance.
To do so, set COMPOSE_BAKE=true.
[+] Building 1.8s (20/21) docker-container:mybuilder
=> [isaac-lab-ros2 internal] load build definition from Dockerfile.ros2 0.0s
[+] Building 2.9s (21/21) FINISHED docker-container:mybuilder
=> [isaac-lab-ros2 internal] load build definition from Dockerfile.ros2 0.0s
=> => transferring dockerfile: 1.83kB 0.0s
=> [isaac-lab-base internal] load build definition from Dockerfile.base 0.1s
=> => transferring dockerfile: 4.48kB 0.0s
=> WARN: InvalidDefaultArgInFrom: Default value for ARG ${ISAACSIM_BASE_IMAGE_ARG}:${ISAACSIM_VERSION_ARG} results in empty or invalid base image name (line 12) 0.1s
=> ERROR [isaac-lab-ros2 internal] load metadata for docker.io/library/isaac-lab-base:latest 1.8s
=> [isaac-lab-base internal] load metadata for nvcr.io/nvidia/isaac-sim:5.0.0 0.9s
=> [isaac-lab-ros2 auth] library/isaac-lab-base:pull token for registry-1.docker.io 0.0s
=> [isaac-lab-base internal] load .dockerignore 0.0s
=> => transferring context: 487B 0.0s
=> [isaac-lab-base base 1/13] FROM nvcr.io/nvidia/isaac-sim:5.0.0@sha256:499e1fd7a3ffc68c6c6fff466d4a6a6dc9ab1573e0a28dbd5f100501ea64e410 0.0s
=> => resolve nvcr.io/nvidia/isaac-sim:5.0.0@sha256:499e1fd7a3ffc68c6c6fff466d4a6a6dc9ab1573e0a28dbd5f100501ea64e410 0.0s
=> [isaac-lab-base internal] load build context 0.1s
=> => transferring context: 139.29kB 0.1s
=> CACHED [isaac-lab-base base 2/13] RUN --mount=type=cache,target=/var/cache/apt apt-get update && apt-get install -y --no-install-recommends build-essential cmake git libglib2.0-0 0.0s
=> CACHED [isaac-lab-base base 3/13] COPY ../ /workspace/isaaclab 0.0s
=> CACHED [isaac-lab-base base 4/13] RUN chmod +x /workspace/isaaclab/isaaclab.sh 0.0s
=> CACHED [isaac-lab-base base 5/13] RUN ln -sf /isaac-sim /workspace/isaaclab/_isaac_sim 0.0s
=> CACHED [isaac-lab-base base 6/13] RUN /workspace/isaaclab/isaaclab.sh -p -m pip install toml 0.0s
=> CACHED [isaac-lab-base base 7/13] RUN --mount=type=cache,target=/var/cache/apt /workspace/isaaclab/isaaclab.sh -p /workspace/isaaclab/tools/install_deps.py apt /workspace/isaaclab/source && a 0.0s
=> CACHED [isaac-lab-base base 8/13] RUN mkdir -p /isaac-sim/kit/cache && mkdir -p /root/.cache/ov && mkdir -p /root/.cache/pip && mkdir -p /root/.cache/nvidia/GLCache && mkdir -p /root 0.0s
=> CACHED [isaac-lab-base base 9/13] RUN touch /bin/nvidia-smi && touch /bin/nvidia-debugdump && touch /bin/nvidia-persistenced && touch /bin/nvidia-cuda-mps-control && touch /bin/nvidia 0.0s
=> CACHED [isaac-lab-base base 10/13] RUN --mount=type=cache,target=/root/.cache/pip /workspace/isaaclab/isaaclab.sh --install 0.0s
=> CACHED [isaac-lab-base base 11/13] RUN /workspace/isaaclab/isaaclab.sh -p -m pip uninstall -y quadprog 0.0s
=> CACHED [isaac-lab-base base 12/13] RUN echo "export ISAACLAB_PATH=/workspace/isaaclab" >> ${HOME}/.bashrc && echo "alias isaaclab=/workspace/isaaclab/isaaclab.sh" >> ${HOME}/.bashrc && echo "a 0.0s
=> CACHED [isaac-lab-base base 13/13] WORKDIR /workspace/isaaclab 0.0s
=> CANCELED [isaac-lab-base] exporting to docker image format 0.8s
=> => exporting layers 0.0s
=> => exporting manifest sha256:8b2ff34c01f4c8a1ae0437a31c3e4b11cead9a74a58ed909a83b638ee0c7f303 0.0s
=> => exporting config sha256:0587389ca238d52afa750aca47e2b01f7148e30878e1cd01d8a42222cc1de35f 0.0s
=> => sending tarball 0.8s
------
> [isaac-lab-ros2 internal] load metadata for docker.io/library/isaac-lab-base:latest:
------
failed to solve: isaac-lab-base: failed to resolve source metadata for docker.io/library/isaac-lab-base:latest: pull access denied, repository does not exist or may require authorization: server message: insufficient_scope: authorization failed
$ docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
isaac-lab-base latest 5eff70ff0b7e 3 minutes ago 17.7GB
Solution
For me, this was because I accidently created a custom builder -- buildx_buildkit_mybuilder0 -- from another project (elevation_mapping_cupy if anyone wonders).
$ docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
isaac-lab-base latest b50df75c7da0 52 minutes ago 17.7GB
nvcr.io/nvidia/isaac-sim 5.0.0 6e095b37a05b 5 weeks ago 15.2GB
moby/buildkit buildx-stable-1 44f5d48d5c6c 2 months ago 219MB
$ docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
07accd36c592 moby/buildkit:buildx-stable-1 "buildkitd --allow-i…" 6 days ago Up 17 minutes buildx_buildkit_mybuilder0
To fix it, simply remove the custom builder. Note that below mybuilder came from buildx_buildkit_mybuilder0 which is in the format buildx_buildkit_<builder-name><node-number>.
docker buildx stop mybuilder
docker buildx rm mybuilder
# optional: remove the builder image too
docker rmi moby/buildkit:buildx-stable-1
then you check if you reverted back to default build (should show "default"):
reacted with thumbs up emoji reacted with thumbs down emoji reacted with laugh emoji reacted with hooray emoji reacted with confused emoji reacted with heart emoji reacted with rocket emoji reacted with eyes emoji
Uh oh!
There was an error while loading. Please reload this page.
-
Hi everyone, recently I ran into a niche and bizzare error when trying to build the ros2 image using isaaclab's
container.pyscript, and would like to share the how I resolved it at the end, just so anyone else out there don't have to spend a full hour debugging this.Issue
Essentially the error occurs because the line
FROM isaac-lab-base${DOCKER_NAME_SUFFIX} AS ros2indocker/Dockerfile.ros2can not findisaac-lab-baselocally, leading it to try to find it on docker hub and fails due to permission error. As you can see in the terminal output below,isaac-lab-basebuilt successfully, but when the next build stage failed to see this built image locally.Solution
For me, this was because I accidently created a custom builder --
buildx_buildkit_mybuilder0-- from another project (elevation_mapping_cupyif anyone wonders).To fix it, simply remove the custom builder. Note that below
mybuildercame frombuildx_buildkit_mybuilder0which is in the formatbuildx_buildkit_<builder-name><node-number>.then you check if you reverted back to default build (should show "default"):
Then after rmoving the custom builder, I ran
./container.py start ros2with no problem.Beta Was this translation helpful? Give feedback.
All reactions