Build directories do not remember their source

[davidchisnall]

Xmake Version

v2.7.1+master.20e5768e3

Operating System Version and Architecture

FreeBSD 14

Describe Bug

When I run a build from the build directory (as is generally best practice, as it keeps all build-related state distinct from the source and allows separate mount points for source [read only] and build [ephemeral]), I need to specify the location of the source directory for every xmake invocation.

$ mkdir build
$ cd build
$ mkdir .xmake # Apparently I need to do this manually, 
$ xmake f -P path-to-project --some-config-option -o $(pwd)
note: xmake.lua not found, try generating it (pass -y or --confirm=y/n/d to skip confirm)?
please input: n (y/n)

Expected Behavior

The last step in the above example should perform the build.

Project Configuration

No response

Additional Information and Error Logs

No response

[waruqi]

$ mkdir build
$ cd build
$ mkdir .xmake
$ xmake -P ..
[ 25%]: cache compiling.release src/main.cpp
[ 50%]: linking.release test
[100%]: build ok!

or

$ mkdir build
$ cd build
$ export XMAKE_CONFIGDIR=`pwd`
$ xmake -P ..

[davidchisnall]

$ mkdir build
$ cd build
$ mkdir .xmake
$ xmake -P ..

This works, however if I then run xmake a second time from that build directory then it expects to find an xmake.lua in the current directory. I would expect it to note the existence of a .xmake directory containing the build configuration and use that.

[davidchisnall]

In addition, sometimes, xmake in the build step forgets arguments passed to the configure (xmake f -P {source dir} --{whatever}) step. This doesn't happen on most builds, but only occasionally. I'm not sure what triggers it.

[waruqi]

I will continue to improve on some of the points you mentioned here. #3342

[waruqi]

I will take a look at it later and if you encounter problems, please open some separate issues (not discussions) and describe them in detail. I usually deal with them and consider improvements (although it may take some time or even wait for a few releases).

If no issues are provided, it is difficult for me to know what trouble the user is having and to keep track of its status.

[waruqi]

FYI: We have no open sourced our system. We have had a number of problems with xmake, but if you could take a look and let us know if they're user error (and how to fix them if they are), that would be fantastic.

I tried this project, but meet some problems about llvm. CHERIoT-Platform/cheriot-rtos#2

[waruqi]

The most annoying one is that this does not generate the .dump file unless we use xmake -v. We also have a load of places like this where we need to pass force = true because xmake thinks that our compiler does not support flags that it does, in fact, support (e.g. -NDEBUG).

I tried it. xmake cannot find ld.lld, so target:tool("ld") return nil

you can update xmake to dev and try it again. I added support for ld.lld

[ 98%]: linking firmware build/cheriot/cheriot/release/test-suite
/Users/ruki/projects/working/rtos/llvm-project/build/bin/ld.lld --script=build/test-suite-firmware.ldscript --relax -o build/cheriot/cheriot/release/test-suite build/.objs/cherimcu.switcher/cheriot/cheriot/release/__/sdk/core/switcher/entry.S.o build/.objs/test-suite.loader/cheriot/cheriot/release/__/sdk/core/loader/boot.S.o build/.objs/test-suite.loader/cheriot/cheriot/release/__/sdk/core/loader/boot.cc.o build/cheriot/cheriot/release/test_runner.compartment build/cheriot/cheriot/release/ccompile_test.compartment build/cheriot/cheriot/release/futex_test.compartment build/cheriot/cheriot/release/cherimcu.allocator.compartment build/cheriot/cheriot/release/crt.library build/cheriot/cheriot/release/test-suite.scheduler.compartment build/cheriot/cheriot/release/locks_test.compartment build/cheriot/cheriot/release/freestanding.library build/cheriot/cheriot/release/cxxrt.library build/cheriot/cheriot/release/thread_pool.compartment build/cheriot/cheriot/release/compartment_calls_test.compartment build/cheriot/cheriot/release/string.library build/cheriot/cheriot/release/allocator_test.compartment build/cheriot/cheriot/release/stack_exhaustion_trusted.compartment build/cheriot/cheriot/release/compartment_calls_inner.compartment build/cheriot/cheriot/release/atomic.library build/cheriot/cheriot/release/queue_test.compartment build/cheriot/cheriot/release/crash_recovery_outer.compartment build/cheriot/cheriot/release/stack_exhaustion_thread.compartment build/cheriot/cheriot/release/cherimcu.software_revoker.compartment build/cheriot/cheriot/release/crash_recovery_test.compartment build/cheriot/cheriot/release/multiwaiter_test.compartment build/cheriot/cheriot/release/stack_exhaustion_test.compartment build/cheriot/cheriot/release/crash_recovery_inner.compartment build/cheriot/cheriot/release/thread_pool_test.compartment

[ 98%]: Creating firmware dump build/cheriot/cheriot/release/test-suite.dump
/Users/ruki/projects/working/rtos/llvm-project/build/bin/llvm-objdump -glxsdrS build/cheriot/cheriot/release/test-suite
[100%]: build ok!

#3346

Also, if you encounter problems with xmake, you can open a new issue and I will continue to improve xmake.

[davidchisnall]

Also, if you encounter problems with xmake, you can open a new issue and I will continue to improve xmake.

Thanks. I opened several issues, but you moved them all to discussions and closed them because you couldn't reproduce them and I couldn't share enough of the build system to help. Hopefully now that it's open source, you can see all that we're doing.

I guess creating a symlink from ld -> ld.lld would work?

[waruqi]

I guess creating a symlink from ld -> ld.lld would work?

work, but xmake/dev has supported it.

Thanks. I opened several issues, but you moved them all to discussions and closed them because you couldn't reproduce them and I couldn't share enough of the build system to help. Hopefully now that it's open source, you can see all that we're doing.

Yes, it is difficult for me to locate and solve problems if I cannot reproduce them. But now that you have open source, it has helped me a lot to solve the problem.

[waruqi]

And, I've continued to improve the build directory experience, so that I can now remember the build directory outside of the project source code.

But it hasn't been merged yet, so you can test it by updating to the config branch.

$ xmake update -s github:xmake-io/xmake#config
$ cd workdir
$ xmake f -P ../projectdir
$ xmake
$ xmake run
...

- workdir
  - build (generated)
  - .xmake (generated)
- projectdir
  - src 
  - xmake.lua

see #3342

and related pr #3344