cibuildwheel with skbuild-conan - libcap/2.69 fails to build #2540

hgeorgsch · 2025-08-01T04:38:52Z

hgeorgsch
Aug 1, 2025

Hi,
I am new to packaging python libraries. For cibuildwork I find precious few examples of how it is set up in practice in different combinations with other toolchains. Does anyone have experience with integration with skbuild-conan? Or conan at all?

I use skbuild-conan to build my package, and it works fine without cibuildwork (Debian, Ubuntu 22.04 and a clean Ubuntu 24.04 docker, as well as Ubuntu 24.04 github runner).

With cibuildworks it fails to build libcap/2.64 which is an indirect dependency, which I cannot immediately change. The only direct dependencies are symengine and opencv.

Initially, I get an error message that O2 is not found, which I can get around by setting BUILD_CC=gcc in the environment, but this only changes it to a similar complaint about m64,

make: m64: Command not found
make: [Makefile:135: cap_text.o] Error 127 (ignored)
m64 -fPIC -O3 -Wall -Wwrite-strings -Wpointer-arith -Wcast-qual -Wcast-align -Wstrict-prototypes -Wmissing-prototypes -Wnested-externs -Winline -Wshadow -Wunreachable-code  -DNDEBUG -Dlinux -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 -I/root/.conan2/p/b/libcae030e9e99998f/b/src/libcap/../libcap/include/uapi -I/root/.conan2/p/b/libcae030e9e99998f/b/src/libcap/../libcap/include -c cap_file.c -o cap_file.o
make: m64: Command not found
make: [Makefile:132: cap_file.o] Error 127 (ignored)
ar rcs libcap.a cap_alloc.o cap_proc.o cap_extint.o cap_flag.o cap_text.o cap_file.o
ar: cap_alloc.o: No such file or directory
make: *** [Makefile:99: libcap.a] Error 1

Does anyone have any ideas what could be wrong or how to fix it?

All my own ideas have me stuck,

Installing libcap/2.69 system level would probably work, but it seems that only 2.48 is available.
If I can change the conan profile, it may be possible to override the dependency, but how would one do that inside cibuildwork (without messing up the rest of the profile)?
Ideally, there is some way to fix the environment so that conan works more robustly.

TIA

Answered by mayeut

Aug 2, 2025

quay.io/pypa/manylinux_2_28_x86_64

yes, the exact version of the image used by cibuildwheel is printed in the preamble

and crashes instead on libx264 with the error

In any case, you'll be rebuilding from sources so you might not want to rely on CCI default options for those dependencies. You should probably start by disabling everything you don't need from the OpenCV build, that might lower the number of indirect dependencies needed to build (e.g. do you need videoio / highgui / imgcodecs / ... modules & if not, disable them) and build that in the before-all step of cibuildwheel.

Some experiments:

[root@6a8364deab62 ~]# pipx install meson ninja conan
# required for OpenSSL build
[root…

View full answer

hgeorgsch · 2025-08-01T07:39:37Z

hgeorgsch
Aug 1, 2025
Author

I see that it might be possible to build only for more recent standard, such as manylinux_2_31 or _39, but whatever I try, cibuildwheel tries to build on 2_28. If anyone has a working example of a toml file restricting the manylinux builds in such a way, that would be greatly appreciated.

10 replies

hgeorgsch Aug 1, 2025
Author

No, it already tries to build libcap in the right version, but the build fails. On Debian bookworm it also builds libcap, since the native version is 2.66, and here it works, also when I build the package with python -m build. Hence, there is something in the build environment set up by cibuildwheel which is incompatible with the rest of the stack.

I am not ruling out the possibility that this could be fixed by some reconfiguration of conan. However, firstly I have to access the configuration via skbuild-conan and/or cibuildwheel, which bypass the config files used in a stand-alone build, and secondly, the configuration needs to take particular docker image into account.

mayeut Aug 2, 2025
Maintainer

Have you tried running python -m build in a manylinux container without using cibuildwheel ? It's not clear to me from your various attempts.

You can also try creating the conan profile used by skbuild-conan before it tries to do so by adding the following lines to your pyproject.toml:

[tool.cibuildwheel.linux]
before-all = ["set -x", "pipx install conan", "conan profile detect --name skbuild_conan_py", "conan install --profile:all=skbuild_conan_py --requires=libcap/2.69 '--build=*'"]

On manylinux_2_28_x86_64, you should get a log like:

Running before_all...

    + /opt/python/cp39-cp39/bin/python -c 'import sys, json, os; json.dump(os.environ.copy(), sys.stdout)'
    + sh -c 'set -x && pipx install conan && conan profile detect --name skbuild_conan_py && conan install --profile:all=skbuild_conan_py --requires=libcap/2.69 '"'"'--build=*'"'"' && exit 1'
+ pipx install conan
creating virtual environment...
installing conan...
done! ✨ 🌟 ✨
  installed package conan 2.19.1, installed using Python 3.12.11
  These apps are now globally available
    - conan
+ conan profile detect --name skbuild_conan_py
detect_api: Found cc=gcc-14.2.1
detect_api: gcc>=5, using the major as version
detect_api: gcc C++ standard library: libstdc++11

Detected profile:
WARN: This profile is a guess of your environment, please check it.
WARN: The output of this command is not guaranteed to be stable and can change in future Conan versions.
WARN: Use your own profile files for stability.
Saving detected profile to /root/.conan2/profiles/skbuild_conan_py
[settings]
arch=x86_64
build_type=Release
compiler=gcc
compiler.cppstd=gnu17
compiler.libcxx=libstdc++11
compiler.version=14
os=Linux

+ conan install --profile:all=skbuild_conan_py --requires=libcap/2.69 '--build=*'

======== Input profiles ========
Profile host:
[settings]
arch=x86_64
build_type=Release
compiler=gcc
compiler.cppstd=gnu17
compiler.libcxx=libstdc++11
compiler.version=14
os=Linux

Profile build:
[settings]
arch=x86_64
build_type=Release
compiler=gcc
compiler.cppstd=gnu17
compiler.libcxx=libstdc++11
compiler.version=14
os=Linux


======== Computing dependency graph ========
libcap/2.69: Not found in local cache, looking in remotes...
libcap/2.69: Checking remote: conancenter
Connecting to remote 'conancenter' anonymously
libcap/2.69: Downloaded recipe revision 7ef2d60864c2e58e89db957be936dc49
Graph root
    cli
Requirements
    libcap/2.69#7ef2d60864c2e58e89db957be936dc49 - Downloaded (conancenter)

======== Computing necessary packages ========
libcap/2.69: Forced build from source
Requirements
    libcap/2.69#7ef2d60864c2e58e89db957be936dc49:584fac9f8a19ed0436c5e532dd09aa36968270cf - Build

======== Installing packages ========
libcap/2.69: Sources downloaded from 'conancenter'
libcap/2.69: Calling source() in /root/.conan2/p/libcaec99049bf6e22/s/src
libcap/2.69: Uncompressing libcap-2.69.tar.xz to .

-------- Installing package libcap/2.69 (1 of 1) --------
libcap/2.69: Building from source
libcap/2.69: Package libcap/2.69:584fac9f8a19ed0436c5e532dd09aa36968270cf
libcap/2.69: settings: os=Linux arch=x86_64 compiler=gcc compiler.version=14 build_type=Release
libcap/2.69: options: fPIC=True psx_syscals=False shared=False
libcap/2.69: Copying sources to build folder
libcap/2.69: Building your package in /root/.conan2/p/b/libca82b11bf67c973/b
libcap/2.69: Calling generate()
libcap/2.69: Generators folder: /root/.conan2/p/b/libca82b11bf67c973/b/build-release/conan
libcap/2.69: Generating aggregated env files
libcap/2.69: Generated aggregated env files: ['conanbuild.sh', 'conanrun.sh']
libcap/2.69: Calling build()
libcap/2.69: Apply patch (conan): allow to configure fPIC option from conan recipe
libcap/2.69: Apply patch (conan): allow to override compiler via environment variables
libcap/2.69: RUN: make -j10
sed -e 's,@prefix@,/,' \
	-e 's,@exec_prefix@,/,' \
	-e 's,@libdir@,//lib,' \
	-e 's,@includedir@,//include,' \
	-e 's,@VERSION@,2.69,' \
	-e 's,@deps@,,' \
	libcap.pc.in >libcap.pc
=> making cap_names.list.h from /root/.conan2/p/b/libca82b11bf67c973/b/src/libcap/../libcap/include/uapi/linux/capability.h
grep -E '^#define\s+CAP_([^\s]+)\s+[0-9]+\s*$' include/uapi/linux/capability.h | sed -e 's/^#define\s\+/{"/' -e 's/\s*$/},/' -e 's/\s\+/",/' -e 'y/ABCDEFGHIJKLMNOPQRSTUVWXYZ/abcdefghijklmnopqrstuvwxyz/' > cap_names.list.h
cc -O2 -Dlinux -Wall -Wwrite-strings -Wpointer-arith -Wcast-qual -Wcast-align -Wstrict-prototypes -Wmissing-prototypes -Wnested-externs -Winline -Wshadow -Wunreachable-code  -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 -I/root/.conan2/p/b/libca82b11bf67c973/b/src/libcap/../libcap/include/uapi -I/root/.conan2/p/b/libca82b11bf67c973/b/src/libcap/../libcap/include _makenames.c -o _makenames
./_makenames > cap_names.h
cc  -m64 -fPIC -O3 -Wall -Wwrite-strings -Wpointer-arith -Wcast-qual -Wcast-align -Wstrict-prototypes -Wmissing-prototypes -Wnested-externs -Winline -Wshadow -Wunreachable-code  -DNDEBUG -Dlinux -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 -I/root/.conan2/p/b/libca82b11bf67c973/b/src/libcap/../libcap/include/uapi -I/root/.conan2/p/b/libca82b11bf67c973/b/src/libcap/../libcap/include -c cap_alloc.c -o cap_alloc.o
cc  -m64 -fPIC -O3 -Wall -Wwrite-strings -Wpointer-arith -Wcast-qual -Wcast-align -Wstrict-prototypes -Wmissing-prototypes -Wnested-externs -Winline -Wshadow -Wunreachable-code  -DNDEBUG -Dlinux -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 -I/root/.conan2/p/b/libca82b11bf67c973/b/src/libcap/../libcap/include/uapi -I/root/.conan2/p/b/libca82b11bf67c973/b/src/libcap/../libcap/include -c cap_proc.c -o cap_proc.o
cc  -m64 -fPIC -O3 -Wall -Wwrite-strings -Wpointer-arith -Wcast-qual -Wcast-align -Wstrict-prototypes -Wmissing-prototypes -Wnested-externs -Winline -Wshadow -Wunreachable-code  -DNDEBUG -Dlinux -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 -I/root/.conan2/p/b/libca82b11bf67c973/b/src/libcap/../libcap/include/uapi -I/root/.conan2/p/b/libca82b11bf67c973/b/src/libcap/../libcap/include -c cap_extint.c -o cap_extint.o
cc  -m64 -fPIC -O3 -Wall -Wwrite-strings -Wpointer-arith -Wcast-qual -Wcast-align -Wstrict-prototypes -Wmissing-prototypes -Wnested-externs -Winline -Wshadow -Wunreachable-code  -DNDEBUG -Dlinux -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 -I/root/.conan2/p/b/libca82b11bf67c973/b/src/libcap/../libcap/include/uapi -I/root/.conan2/p/b/libca82b11bf67c973/b/src/libcap/../libcap/include -c cap_flag.c -o cap_flag.o
cc  -m64 -fPIC -O3 -Wall -Wwrite-strings -Wpointer-arith -Wcast-qual -Wcast-align -Wstrict-prototypes -Wmissing-prototypes -Wnested-externs -Winline -Wshadow -Wunreachable-code  -DNDEBUG -Dlinux -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 -I/root/.conan2/p/b/libca82b11bf67c973/b/src/libcap/../libcap/include/uapi -I/root/.conan2/p/b/libca82b11bf67c973/b/src/libcap/../libcap/include  -c cap_text.c -o cap_text.o
cc  -m64 -fPIC -O3 -Wall -Wwrite-strings -Wpointer-arith -Wcast-qual -Wcast-align -Wstrict-prototypes -Wmissing-prototypes -Wnested-externs -Winline -Wshadow -Wunreachable-code  -DNDEBUG -Dlinux -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 -I/root/.conan2/p/b/libca82b11bf67c973/b/src/libcap/../libcap/include/uapi -I/root/.conan2/p/b/libca82b11bf67c973/b/src/libcap/../libcap/include -c cap_file.c -o cap_file.o
ar rcs libcap.a cap_alloc.o cap_proc.o cap_extint.o cap_flag.o cap_text.o cap_file.o
ranlib libcap.a

libcap/2.69: Package '584fac9f8a19ed0436c5e532dd09aa36968270cf' built
libcap/2.69: Build folder /root/.conan2/p/b/libca82b11bf67c973/b/build-release
libcap/2.69: Generating the package
libcap/2.69: Packaging in folder /root/.conan2/p/b/libca82b11bf67c973/p
libcap/2.69: Calling package()
libcap/2.69: RUN: make install-common-cap -j10
mkdir -p -m 0755 /root/.conan2/p/b/libca82b11bf67c973/p//include/sys
mkdir -p -m 0755 /root/.conan2/p/b/libca82b11bf67c973/p//lib/pkgconfig
mkdir -p -m 0755 /root/.conan2/p/b/libca82b11bf67c973/p//lib
install -m 0644 include/sys/capability.h /root/.conan2/p/b/libca82b11bf67c973/p//include/sys
install -m 0644 libcap.pc /root/.conan2/p/b/libca82b11bf67c973/p//lib/pkgconfig/libcap.pc

libcap/2.69: RUN: make install-static-cap -j10
mkdir -p -m 0755 /root/.conan2/p/b/libca82b11bf67c973/p//include/sys
mkdir -p -m 0755 /root/.conan2/p/b/libca82b11bf67c973/p//lib/pkgconfig
mkdir -p -m 0755 /root/.conan2/p/b/libca82b11bf67c973/p//lib
install -m 0644 include/sys/capability.h /root/.conan2/p/b/libca82b11bf67c973/p//include/sys
install -m 0644 libcap.pc /root/.conan2/p/b/libca82b11bf67c973/p//lib/pkgconfig/libcap.pc
install -m 0644 libcap.a /root/.conan2/p/b/libca82b11bf67c973/p//lib/libcap.a

libcap/2.69: package(): Packaged 1 file: License
libcap/2.69: package(): Packaged 1 '.a' file: libcap.a
libcap/2.69: package(): Packaged 1 '.h' file: capability.h
libcap/2.69: Created package revision dcfc4e35ddebc5df488b7874fa56634d
libcap/2.69: Package '584fac9f8a19ed0436c5e532dd09aa36968270cf' created
libcap/2.69: Full package reference: libcap/2.69#7ef2d60864c2e58e89db957be936dc49:584fac9f8a19ed0436c5e532dd09aa36968270cf#dcfc4e35ddebc5df488b7874fa56634d
libcap/2.69: Package folder /root/.conan2/p/b/libca82b11bf67c973/p

======== Finalizing install (deploy, generators) ========
cli: Generating aggregated env files
cli: Generated aggregated env files: ['conanbuild.sh', 'conanrun.sh']
Install finished successfully

hgeorgsch Aug 2, 2025
Author

Good point. I have had some trouble identifying the authoritative source to pull the docker image from.
Would quay.io/pypa be the right identifier to use? E.g.

docker pull quay.io/pypa/manylinux_2_28_x86_64

Using that image, interestingly, skbuild-conan still fails on libcap, but the direct call to conan does not, and rerunning python3 -m build it relies on the cached installation for libcap, moving past that critical step, and crashes instead on libx264 with the error

No working C compiler found.

I am not sure what this means, nor which hacks would be kosher and which might break the standards for the wheel, but I'll continue testing with running conan separately to preinstall, and see where it gets me.

The only difference between the compiler versions on manylinux and my home box is the compiler version @ 14 and 12 respectively. On manylinux, the skbuild profile and the default one are identical.

Thanks a lot for your input. It is a good step forward.

mayeut Aug 2, 2025
Maintainer

quay.io/pypa/manylinux_2_28_x86_64

yes, the exact version of the image used by cibuildwheel is printed in the preamble

and crashes instead on libx264 with the error

In any case, you'll be rebuilding from sources so you might not want to rely on CCI default options for those dependencies. You should probably start by disabling everything you don't need from the OpenCV build, that might lower the number of indirect dependencies needed to build (e.g. do you need videoio / highgui / imgcodecs / ... modules & if not, disable them) and build that in the before-all step of cibuildwheel.

Some experiments:

[root@6a8364deab62 ~]# pipx install meson ninja conan
# required for OpenSSL build
[root@6a8364deab62 ~]# yum -y install perl-core
[root@6a8364deab62 ~]# cat /root/.conan2/profiles/skbuild_conan_py
[settings]
arch=x86_64
build_type=Release
compiler=gcc
compiler.cppstd=gnu17
compiler.libcxx=libstdc++11
compiler.version=14
os=Linux

[conf]
tools.system.package_manager:mode=install

[options]
ffmpeg/*:with_vaapi=False
ffmpeg/*:with_vdpau=False
# ffmpeg/*:with_xcb=False
# ffmpeg/*:with_xlib=False
opencv/*:with_wayland=False
# pulseaudio/*:with_x11=False

# [replace_requires]
# expat/*: expat/system
# zlib/*: zlib/system

[replace_tool_requires]
cmake/*: cmake/system
meson/*: meson/system
ninja/*: ninja/system

[platform_requires]
expat/system
zlib/system

[platform_tool_requires]
cmake/system
meson/system
ninja/system

# the build succeeds but takes quite some time
[root@6a8364deab62 ~]# conan install --profile:all=skbuild_conan_py --requires=opencv/4.12.0 --requires=symengine/0.14.0 '--build=*'

# the following profile allows for a much faster build:
[root@6a8364deab62 ~]# cat /root/.conan2/profiles/skbuild_conan_py
[settings]
arch=x86_64
build_type=Release
compiler=gcc
compiler.cppstd=gnu17
compiler.libcxx=libstdc++11
compiler.version=14
os=Linux

[conf]
tools.system.package_manager:mode=install

[options]
# ffmpeg/*:with_vaapi=False
# ffmpeg/*:with_vdpau=False
# ffmpeg/*:with_xcb=False
# ffmpeg/*:with_xlib=False
# opencv/*:with_wayland=False
# pulseaudio/*:with_x11=False
opencv/*:highgui=False
opencv/*:videoio=False
opencv/*:imgcodecs=False
opencv/*:dnn=False

[replace_requires]
# expat/*: expat/system
zlib/*: zlib/system

[replace_tool_requires]
cmake/*: cmake/system
meson/*: meson/system
ninja/*: ninja/system

[platform_requires]
expat/system
zlib/system

[platform_tool_requires]
cmake/system
meson/system
ninja/system

Answer selected by hgeorgsch

hgeorgsch Aug 3, 2025
Author

Thank you. Great. Now I am able to produce the wheelhouse for 2_34/x86_64.

I just wonder, should I not also get wheels for i686, when pyproject.toml says this?

[tool.cibuildwheel]
build = [ "*-manylinux_x86_64", "*-manylinux_i686" ]

[tool.cibuildwheel.linux]
manylinux-x86_64-image = "manylinux_2_34"
manylinux-i686-image = "manylinux_2_34"

(It is not that I think I need i686, I just don't like lose ends.)

Essentially, the trick is, as you suggest, to call conan directly to build each troublesome dependency, in the before-all variable.
In case anybody else runs across similar problems, the other tricks I found critical are,

Many failures when conan runs through skbuild-conan can be resolved by running conan as a standalone script, and preinstalling troublesome packages.
It is useful to run cibuildwheel in the docker image to identify each package which requires yum installation or conan installation. (Not running the entire process from scratch every time saves time.)
It is very useful to limit the build to the most familiar architecture. Failures on different architectures is confusing, e.g.

[tool.cibuildwheel]
build = "cp313-manylinux_x86_64"
[tool.cibuildwheel.linux]
manylinux-x86_64-image = "manylinux_2_34"

joerick Aug 10, 2025
Maintainer

I just wonder, should I not also get wheels for i686, when pyproject.toml says this?

We removed building i686 by default in v3.0 - but you can still enable it using the archs option on an x86_64 runner.

henryiii Aug 10, 2025
Maintainer

(Add auto32)

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

cibuildwheel with skbuild-conan - libcap/2.69 fails to build #2540

Uh oh!

{{title}}

Uh oh!

Replies: 1 comment 10 replies

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Select a reply

Uh oh!

cibuildwheel with skbuild-conan - libcap/2.69 fails to build #2540

Uh oh!

hgeorgsch Aug 1, 2025

Replies: 1 comment · 10 replies

Uh oh!

hgeorgsch Aug 1, 2025 Author

Uh oh!

hgeorgsch Aug 1, 2025 Author

Uh oh!

mayeut Aug 2, 2025 Maintainer

Uh oh!

hgeorgsch Aug 2, 2025 Author

Uh oh!

mayeut Aug 2, 2025 Maintainer

Uh oh!

hgeorgsch Aug 3, 2025 Author

Uh oh!

joerick Aug 10, 2025 Maintainer

Uh oh!

henryiii Aug 10, 2025 Maintainer

hgeorgsch
Aug 1, 2025

Replies: 1 comment 10 replies

hgeorgsch
Aug 1, 2025
Author

hgeorgsch Aug 1, 2025
Author

mayeut Aug 2, 2025
Maintainer

hgeorgsch Aug 2, 2025
Author

mayeut Aug 2, 2025
Maintainer

hgeorgsch Aug 3, 2025
Author

joerick Aug 10, 2025
Maintainer

henryiii Aug 10, 2025
Maintainer