What's the perfered structure for a library with generated code (e.g., uses utils.bin2c) ?

[find]

I have a project structured like this:

root/
- xmake.lua // add_files('res.bin', 'lib.cpp')
- res.bin // rule='utils.bin2c'
- lib.h
- lib.cpp // #include "res.bin.h"
- demo/
  - xmake.lua // includes('..')
  - main.cpp

And I notice that bin2c was not working when I try to build demo inside demo directory with xmake -P . as it will search for res.bin in the parent folder of root

this can be solved by change this line from

local argv = {"lua", "private.utils.bin2c", "-i", path(sourcefile_bin), "-o", path(headerfile)}

into

local argv = {"lua", "private.utils.bin2c", "-i", path.absolute(sourcefile_bin), "-o", path.absolute(headerfile)}

But I also find that almost every generator comes with xmake is doing the same thing: input path(sourcefile), output path(generatedfile). So the question is, should them all be path.absolute, or am I using xmake wrong, that the project can be structured in a more elegant way which this problem just doesn't exist?

[waruqi]

It works for me. you can try https://github.com/xmake-io/xmake/blob/master/tests/projects/other/bin2c/

ruki-2:bin2c ruki$ xmake -rv
[ 50%]: generating.bin2c src/data.bin
/Users/ruki/.local/bin/xmake lua private.utils.bin2c -i src/data.bin -o build/.gens/test/macosx/x86_64/release/rules/utils/bin2c/data.bin.h -w 16
[ 50%]: generating.bin2c src/image.png
/Users/ruki/.local/bin/xmake lua private.utils.bin2c -i src/image.png -o build/.gens/test/macosx/x86_64/release/rules/utils/bin2c/image.png.h -w 16
generating code data file from /Users/ruki/projects/personal/xmake/tests/projects/other/bin2c/src/data.bin ..
generating code data file from /Users/ruki/projects/personal/xmake/tests/projects/other/bin2c/src/image.png ..
/Users/ruki/projects/personal/xmake/tests/projects/other/bin2c/build/.gens/test/macosx/x86_64/release/rules/utils/bin2c/image.png.h generated!
/Users/ruki/projects/personal/xmake/tests/projects/other/bin2c/build/.gens/test/macosx/x86_64/release/rules/utils/bin2c/data.bin.h generated!
[ 70%]: cache compiling.release src/main.c
"/usr/bin/xcrun -sdk macosx clang" -c -Qunused-arguments -target x86_64-apple-macos11.4 -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX
.platform/Developer/SDKs/MacOSX11.3.sdk -fvisibility=hidden -O3 -Ibuild/.gens/test/macosx/x86_64/release/rules/utils/bin2c -DNDEBUG -o build/.objs/test/macos
x/x86_64/release/src/main.c.o src/main.c
[ 80%]: linking.release test
"/usr/bin/xcrun -sdk macosx clang++" -o build/macosx/x86_64/release/test build/.objs/test/macosx/x86_64/release/src/main.c.o -target x86_64-apple-macos11.4 -
isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.3.sdk -stdlib=libc++ -Wl,-x -lz
[100%]: build ok!
ruki-2:bin2c ruki$

then please add -vD argument and let me see the full output if it does not work for you.

[find]

Yes this project is all fine, my problem happens when I includes another xmake project, I will post my log below

[find]

the project:

example-xmake-proj.zip

Logs:

E:\tmp\example-xmake-proj\demo>xmake -vD -P .
[ 22%]: generating.bin2c ..\res.bin
E:\utils\xmake\xmake.exe lua private.utils.bin2c -i ..\res.bin -o build\.gens\lib\mingw\x64\release\rules\utils\bin2c\res.bin.h
checking for x86_64-w64-mingw32-g++ ... E:\msys2.x64\mingw64\bin\x86_64-w64-mingw32-g++
checking for the c++ compiler (cxx) ... x86_64-w64-mingw32-g++
[ 22%]: ccache compiling.release main.cpp
E:\msys2.x64\mingw64\bin\x86_64-w64-mingw32-g++ -c -o build\.objs\demo\mingw\x64\release\main.cpp.obj main.cpp
error: E:\tmp\res.bin not found!
checking for E:\msys2.x64\mingw64\bin\x86_64-w64-mingw32-g++ ... ok
error: @programdir\modules\private\async\runjobs.lua:256: @programdir\core\sandbox\modules\os.lua:362: execv(E:\utils\xmake\xmake.exe lua private.utils.bin2c -i ..\res.bin -o build\.gens\lib\mingw\x64\release\rules\utils\bin2c\res.bin.h) failed(-1)
stack traceback:
    [C]: in function 'error'
    [@programdir\core\base\os.lua:872]:
    [@programdir\core\sandbox\modules\os.lua:362]:
    [@programdir\core\sandbox\modules\os.lua:275]: in function 'vrunv'
    [@programdir\modules\private\utils\batchcmds.lua:96]: in function 'script'
    [@programdir\modules\private\utils\batchcmds.lua:179]: in function '_runcmd'
    [@programdir\modules\private\utils\batchcmds.lua:186]: in function '_runcmds'
    [@programdir\modules\private\utils\batchcmds.lua:387]: in function 'callback'
    [@programdir\modules\core\project\depend.lua:189]: in function 'on_changed'
    [@programdir\modules\private\utils\batchcmds.lua:386]: in function 'runcmds'
    [@programdir\actions\build\kinds\object.lua:87]: in function 'jobfunc'
    [@programdir\modules\private\async\runjobs.lua:232]:

stack traceback:
        [C]: in function 'error'
        @programdir\core\base\os.lua:872: in function 'os.raiselevel'
        (...tail calls...)
        @programdir\modules\private\async\runjobs.lua:256: in field 'catch'
        @programdir\core\sandbox\modules\try.lua:127: in global 'try'
        @programdir\modules\private\async\runjobs.lua:218: in upvalue 'cotask'
        @programdir\core\base\scheduler.lua:388: in function <@programdir\core\base\scheduler.lua:385>

[waruqi]

But I also find that almost every generator comes with xmake is doing the same thing: input path(sourcefile), output path(generatedfile). So the question is, should them all be path.absolute, or am I using xmake wrong, that the project can be structured in a more elegant way which this problem just doesn't exist?

xmake will find the top-level xmake.lua directory as the root, even though you have set -P . to force the current path to be the root of the project. However, the xmake lua subprocess will still look for the root xmake.lua, so passing relative paths will be a problem.

Therefore, I do not recommend using includes("...") to reverse include the root xmake.lua, which can cause a lot of problems. You should always run xmake in the root directory .

root/xmake.lua

includes("demo")

root/demo/xmake.lua (remove includes)

[find]

Thanks! that works, and makes sense.
While I still think it'll be better if includes can handle projects like closures, I noticed when lib and demo are in the same level, like this:

root/
  demo/
    xmake.lua
  lib/
    xmake.lua

includes('../lib') from demo/xmake.lua works just fine, which fits my need very well

[waruqi]

you should add root/xmake.lua

and write includes("demo" , "lib") in root/xmake.lua

[find]

you should add root/xmake.lua

and write includes("demo" , "lib") in root/xmake.lua

I'm sure that will work, but that's exactly what I'm trying to avoid, In my real use case, lib is in fact a seperated project , and there can be many demos, each is also a independent project with its own dependencies, and I don't want to build them all together