Skip to content

Conversation

Drei-E3
Copy link

@Drei-E3 Drei-E3 commented Jun 8, 2025

  • closes Metadata generation failed when installing PyQt5 #61037
  • [Tests not added and not passed]Test: Manually ran pytest inside the devcontainer to confirm PyQt5 and pytest-qt are functional.
  • Adding platform: linux/amd64 to the docker-compose.yml dev service to work around image compatibility issues on Apple Silicon.
  • Switching from direct Dockerfile builds to docker-compose.yml via updates in .devcontainer/devcontainer.json:
    Removed "dockerFile" setting
    Added "service", "workspaceFolder", and "dockerComposeFile"
  • Installing qt5-qmake and qtbase5-dev via apt-get to support pytest-qt, which is required for the test suite. However this is commented in order to avoid redundant tools installed on not arm/Arch64 plattform just like the original file. The user should uncomment it.

These changes resolve build failures seen on Apple Silicon when using the VS Code Remote - Containers extension.

Why this matters:

Apple Silicon machines often encounter architecture compatibility issues when building development containers, especially when Python packages need to compile C/C++ or Qt-based code. These changes ensure a smooth devcontainer build experience on both ARM64 and x86_64 environments.

Introduction

If you use mac Silicon, you should uncomment "# platform: linux/amd64" of docker-compose.yml file and #-y qt5-qmake qtbase5-dev\ of Dockerfile. If you use another plattform, just build dev container or docker container as usual, nothing was changed.

Zeming Leng added 2 commits June 8, 2025 19:43
…r-compose for M chip Mac compatibility

- Added docker-compose.yml to specify platform linux/amd64 as an option
- Adjusted devcontainer.json to use docker-compose setup
- This fixes build issues on Apple Silicon Macs when building the dev container
Copy link
Member

@rhshadrach rhshadrach left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the PR. Just as a heads up, I think the Docker-related files in the pandas repo are largely unmaintained and may be removed in future.

context: .
dockerfile: Dockerfile
# Uncomment the line below if you are using Mac with M1/M2/M3 and encounter build issues.
# platform: linux/amd64
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You're adding docker-compose because platform cannot be specified in .devcontainer.json, is that right?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. yes VS Code Dev Containers does not support a plain Dockerfile to automatically detect or adjust for architecture differences. By default, it tries to build the image using the host architecture—arm64 on Apple Silicon. docker-compose.yml makes it possible to force the dev containers build with amd 64 instead

  2. actually an other architecture like intel x86 user can also use this docker-compose.ym even if "platform: linux/amd64" is not uncommit. Docker will detected whether the architecture fit or not. If not, docker will build it with default suitable platform. However i dont have another pc to test so just in case i comment this line

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we should add docker-compose.yml here; as @rhshadrach already mentioned we have very little support activity in our Docker space, so I don't think we should expand it at the moment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. yes VS Code Dev Containers does not support a plain Dockerfile to automatically detect or adjust for architecture differences. By default, it tries to build the image using the host architecture—arm64 on Apple Silicon. docker-compose.yml makes it possible to force the dev containers build with amd 64 instead
  2. actually an other architecture like intel x86 user can also use this docker-compose.ym even if "platform: linux/amd64" is not uncommit. Docker will detected whether the architecture fit or not. If not, docker will build it with default suitable platform. However i dont have another pc to test so just in case i comment this line

If you put
FROM --platform=linux/amd64 python:3.11.13 into the Dockerfile itself, it will be built for amd64, even on apple silicon Mac

@rhshadrach rhshadrach added the Build Library building on various platforms label Jun 22, 2025
Copy link
Member

@rhshadrach rhshadrach left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm lukewarm here; this is making the use of the container more complex (adding a file and users need to conditionally configure it). It is also not something we can reasonably ensure stays working (e.g. a build on the CI).

Looking for other's thoughts here. cc @mroeschke @WillAyd

context: .
dockerfile: Dockerfile
# Uncomment the line below if you are using Mac with M1/M2/M3 and encounter build issues.
# platform: linux/amd64
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we should add docker-compose.yml here; as @rhshadrach already mentioned we have very little support activity in our Docker space, so I don't think we should expand it at the moment

apt-get --no-install-recommends -y install \
build-essential \
bash-completion \
# Install Qt5 dependencies for pytest-qt, only for m chip Macs
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are you sure this only affects Macs? I vaguely recall there being some kind of issue with qt on all platforms, and since this is in a Dockerfile anyway I'm not sure why the host should matter

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

apparently not only Macs, but arm64 arch in general?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think this has anything to do with the platform architecture, but rather when running a headless environment (like a Docker container)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was trying to fix this issue too. My approach was to use PyQt6 (#62176).

The main problem is that PyQt5 doesn't provide pre-compiled binaries for linux-arm64.

If you want to have PyQt5 in the container, you must compile it from source. For that, you will need to install some Qt5 libraries. I was verifying this approach with QEMU emulation, but after 3h building PyQt5, I gave up on the idea.

Another solution is to remove the Qt dependencies from the image for ARM64.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The qt dependency is only for a very small set of tests in the code base. If it's problematic for environment creation, I think it would be fine to separate that out

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @WillAyd, I applied your idea on #62422.

@mroeschke
Copy link
Member

Thanks for the pull request, but it appears to have gone stale. If interested in continuing, please merge in the main branch, address any review comments and/or failing tests, and we can reopen.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Build Library building on various platforms

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Metadata generation failed when installing PyQt5

6 participants