--parallel-workers not working as expected

[matteogrechi]

When running colcon build --parallel-workers 1 on cpp code the number of compiler threads is not 1.
To have the "expected" behaviour I should run instead

export MAKEFLAGS=-j1
colcon build --parallel-workers 1

Is this behaviour intended?
Is it possible to cite this fact directly in the colcon documentation?

Thank you very much!

[christophebedard]

As far as I know, that is the expected behaviour with colcon. --parallel-workers controls the number of "jobs" (i.e., packages in this case) being built in parallel, but then each package itself may or may not be built in parallel using multiple threads. That part depends on the underlying built system, e.g., CMake. As you've found, using the MAKEFLAGS environment variable is the proper way to control this for CMake packages.

[xiaoxiaozengzen]

I tried using --executor sequential , and the result was the same as above

[christophebedard]

--executor sequential only means that colcon will process (e.g., build or test) 1 package at a time.

[nyxrobotics]

I have heard that colcon is standard in ROS2, why is such an important issue still unresolved?
Do ROS2 users use other build systems?

[Ryanf55]

no, we just keep increasing our runner CPU/RAM when builds get OOM-killed. It's not sustainable though.

[peci1]

Adding this to my pain points list . Just staring at my laptop with full swap, waiting until the swapped oom killer gets unswapped... Common people, how can anyone develop without a make job server integration like catkin had?

[S-Fichtl]

What I usually care about is not how many threads each individual package is built with, or how many packages are being built in parallel, but instead I care about A: that my system is fully utilized and B: that my system is not overloaded. So I'd like to give a single value as the number of total compiler threads that I know my system can reliably process at once and want the build system to distribute those threads as it wants to build as fast as possible. If I say -j12 colcon may decide whether it wants to build 12 packages with 1 thread each or 1 package with 12 threads or 3 packages with 4 threads each.

[ardabbour]

Did someone actually think about this and say, "Yes, this makes sense?"