Avoid command line length limits in gmake by adding objects to `ar` incrementally and using response file for linking

[mercury233]

What problem will this solve?

When building with MinGW on Windows, a project with a large number of object files can hit the Windows command line length limit.

In the generated gmake files, static libraries are currently created by passing all object files to ar in a single command:

LINKCMD = $(AR) -rcs "$@" $(OBJECTS)

If $(OBJECTS) expands to many files, the command line can exceed the Windows limit (commonly cited as 8191 characters) and the build fails.

The final link step in the generated gmake files also does not use response files, which means large projects may encounter the same command length limitation during linking.

What might be a solution?
Submit object files to ar individually, and use a response file during the link step.

What other alternatives have you already considered?
This workaround seems don't work in beta8.

sed -i '/OBJECTS += $(OBJDIR)\/.*\.o/d' irrlicht.make
sed -i '/^# Rules/i\OBJECTS += $(OBJDIR)/*.o' irrlicht.make

Anything else we should know?
This issue tends to be less visible on typical Linux gmake environments because the supported command line length is significantly larger than on Windows.

[nickclark2016]

If you use a posix shell rather than cmd, do you still have this issue? Also, can you confirm if the Ninja exporter has this issue?

[Jarod42]

I tried with msys, and I reproduce failure with both gmake/codelite/ninja/premake-ninja. (CodeLite try to use response file, but its logic to create it is not totally correct), msvc succeeds.
I should have a fix for premake-ninja as ninja handles natively response file :-)

Adding repro project in https://github.com/Jarod42/premake-sample-projects/tree/master/projects/long_command_line

[mercury233]

If you use a posix shell rather than cmd, do you still have this issue? Also, can you confirm if the Ninja exporter has this issue?

I'm using mintty 3.8.1 shipped with msys2, and it still has this issue. I've never used Ninja so I cant test it.

[Jarod42]

For Ninja, fix is to use rspfile, rspfile_content, see https://ninja-build.org/manual.html#ref_rule.

For Makefile, Codelite (internally) splits $OBJECTS (OBJECT0,OBJECT1, OBJECT2, ..) of block of 100 files (And even 100 files might be too long) to create the response file. Criteria should be length.

[Jarod42]

premake-ninja "fixed".

ar from macOS doesn't support response file, so deactivated for it.

CMake seems to have the same issue on MacOS https://gitlab.kitware.com/cmake/cmake/-/issues/16731

Note: My test suite doesn't launch ninja on macos (yet?).

[nickclark2016]

From how I read that (and I'll need to do some more research), rspfile and rspfile_content seem to be trivial to implement. We could then just use a os.target() check to only do this for Windows targeted Ninja files. Will definitely need to do more reading on this, but this definitely seems needed on both Ninja and gmake.