Skip to content

Conversation

@logic-wei
Copy link

More than one building task of the libbpf and bpftool will be created with -j20. For the libbpf, several same install step running at the same time will cause the compiling error.

See the issus: #354

To solve this, change the dependencies of the external project "xxx" from "xxx-build" to "xxx".

More than one building task of the libbpf and bpftool will be created
with -j20. For the libbpf, several same install step running at the
same time will cause the compiling error.

See the issus: libbpf#354

To solve this, change the dependencies of the external project "xxx"
from "xxx-build" to "xxx".

Signed-off-by: weipeng <[email protected]>
Copy link
Collaborator

@danielocfb danielocfb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For my understanding, why won't this just shift the problem to the build process now? Won't that still happen in parallel (and, hence, have potential to cause conflicts) or is there something serializing that that I am missing?

@logic-wei
Copy link
Author

logic-wei commented Dec 2, 2025

For my understanding, why won't this just shift the problem to the build process now? Won't that still happen in parallel (and, hence, have potential to cause conflicts) or is there something serializing that that I am missing?

Let me show you what happend.

I did a test about cmake.

Create a project as below:

.
├── CMakeLists.txt
├── ext
│   └── Makefile
├── main2.c
└── main.c

The CMakeLists.txt (target helle and hello2 depend on target ext):

project("hello")

include(ExternalProject)
ExternalProject_Add(ext
  PREFIX ext
  SOURCE_DIR ${CMAKE_CURRENT_SOURCE_DIR}/ext/
  CONFIGURE_COMMAND ""
  BUILD_COMMAND ${MAKE}
  BUILD_IN_SOURCE TRUE
  INSTALL_COMMAND ""
  STEP_TARGETS build
)

add_executable(hello main.c)
add_dependencies(hello ext-build)

add_executable(hello2 main2.c)
add_dependencies(hello2 ext-build)

The Makefile:

all:
	sleep 2
	echo test >> output.txt

Build with -j20:

mkdir build
cd build
cmake ..
make -j20

You can see:

[  5%] Creating directories for 'ext'
[ 11%] Creating directories for 'ext'
[ 16%] No download step for 'ext'
[ 22%] No download step for 'ext'
[ 27%] No update step for 'ext'
[ 33%] No update step for 'ext'
[ 44%] No patch step for 'ext'
[ 44%] No patch step for 'ext'
[ 55%] No configure step for 'ext'
[ 55%] No configure step for 'ext'
[ 66%] Performing build step for 'ext'
[ 66%] Performing build step for 'ext'
[ 72%] No install step for 'ext'
[ 72%] Built target ext-build
[ 83%] Building C object CMakeFiles/hello.dir/main.o
[ 83%] Building C object CMakeFiles/hello2.dir/main2.o
[ 88%] Completed 'ext'
[100%] Linking C executable hello
[100%] Linking C executable hello2
[100%] Built target ext
[100%] Built target hello
[100%] Built target hello2

Two ext are building at the same time.

Check the content of the output.txt:

test
test

It show that the ext project dose building for twice.

So that's the key point.

For libbpf the command "install" will be called when doing make. The command "install" will check if the file exist before install it. If the file exist, it will raise the error "install: cannot create regular file xxx': File exists". So if two libbpf were building at the same time, the two "install" will run at the same time and then it will fail.

To fix this problem we can write cmake like this:

project("hello")

include(ExternalProject)
ExternalProject_Add(ext
  PREFIX ext
  SOURCE_DIR ${CMAKE_CURRENT_SOURCE_DIR}/ext/
  CONFIGURE_COMMAND ""
  BUILD_COMMAND ${MAKE}
  BUILD_IN_SOURCE TRUE
  INSTALL_COMMAND ""
  STEP_TARGETS build
)

add_executable(hello main.c)
add_dependencies(hello ext)

add_executable(hello2 main2.c)
add_dependencies(hello2 ext)

Build with -j20:

[  8%] Creating directories for 'ext'
[ 16%] No download step for 'ext'
[ 25%] No update step for 'ext'
[ 33%] No patch step for 'ext'
[ 41%] No configure step for 'ext'
[ 50%] Performing build step for 'ext'
[ 58%] No install step for 'ext'
[ 66%] Completed 'ext'
[ 66%] Built target ext
[ 83%] Building C object CMakeFiles/hello2.dir/main2.o
[ 83%] Building C object CMakeFiles/hello.dir/main.o
[100%] Linking C executable hello
[100%] Linking C executable hello2
[100%] Built target hello
[100%] Built target hello2

Depend on ext rather than ext-build. The ext will be built for only once.

@danielocfb
Copy link
Collaborator

My point is that while the install step won't be run in parallel anymore, the build still will, no? That may not blow up right now, but I fail to see why it couldn't given a certain scheduling and/or upstream changes. Anyway, this seems like an improvement, so we can probably get it in.

@danielocfb danielocfb merged commit 537677c into libbpf:master Dec 2, 2025
14 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants