Build crystal darwin/x86_64 from darwin/arm64

[miry]

Bug Report

I am trying to build crystal x86_64 from MacOS M1.
It is required for crystal-lang/distribution-scripts#265

$ llvm-config --host-target
arm64-apple-darwin23.0.0
$ make clean_cache clean
$ make crystal target=x86_64-apple-darwin release=true
Using /usr/local/bin/llvm-config [version= 15.0.7]
c++ -c  -o src/llvm/ext/llvm_ext.o src/llvm/ext/llvm_ext.cc -I/usr/local/include -std=c++14   -fno-exceptions -fno-rtti -D_DEBUG -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS
CRYSTAL_CONFIG_BUILD_COMMIT="1f592eca6" CRYSTAL_CONFIG_PATH='$ORIGIN/../share/crystal/src' SOURCE_DATE_EPOCH="1699639716"  CRYSTAL_CONFIG_LIBRARY_PATH='$ORIGIN/../lib/crystal' ./bin/crystal build -D strict_multi_assign -D preview_overload_order --release --cross-compile --target x86_64-apple-darwin -Dwithout_interpreter  -o .build/crystal src/compiler/crystal.cr -D without_openssl -D without_zlib -D use_pcre2

cc .build/crystal.o -o .build/crystal  -rdynamic -L/opt/homebrew/Cellar/crystal/1.10.1/bin/../../../../lib /Users/miry/src/crystal/crystal/src/llvm/ext/llvm_ext.o `"/usr/local/bin/llvm-config" --libs --system-libs --ldflags 2> /dev/null` -lstdc++ -lpcre2-8 -lgc -L/opt/homebrew/Cellar/libevent/2.1.12_1/lib -levent -liconv

I found few details:

1. Make does not show any errors, only there is no resulted file.
2. The last line cc does not use recieve any target argument.
3. The last line cc includes -L/opt/homebrew/..., that's are arm64
4. If I run the last cc with -v, than it shows:

Run cc -v ...

Apple clang version 15.0.0 (clang-1500.0.40.1)
Target: arm64-apple-darwin23.0.0
Thread model: posix
InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin
 "/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ld" -demangle -export_dynamic -lto_library /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/libLTO.dylib -dynamic -arch arm64 -platform_version macos 14.0.0 14.0 -syslibroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk -o .build/crystal -L/opt/homebrew/Cellar/crystal/1.10.1/bin/../../../../lib -L/usr/local/lib -L/opt/homebrew/Cellar/libevent/2.1.12_1/lib -L/usr/local/lib .build/crystal.o /Users/miry/src/crystal/crystal/src/llvm/ext/llvm_ext.o -search_paths_first -headerpad_max_install_names -lLLVMWindowsManifest -lLLVMWindowsDriver -lLLVMXRay -lLLVMLibDriver -lLLVMDlltoolDriver -lLLVMCoverage -lLLVMLineEditor -lLLVMAArch64Disassembler -lLLVMAArch64AsmParser -lLLVMAArch64CodeGen -lLLVMAArch64Desc -lLLVMAArch64Utils -lLLVMAArch64Info -lLLVMX86TargetMCA -lLLVMX86Disassembler -lLLVMX86AsmParser -lLLVMX86CodeGen -lLLVMX86Desc -lLLVMX86Info -lLLVMOrcJIT -lLLVMMCJIT -lLLVMJITLink -lLLVMInterpreter -lLLVMExecutionEngine -lLLVMRuntimeDyld -lLLVMOrcTargetProcess -lLLVMOrcShared -lLLVMDWP -lLLVMDebugInfoGSYM -lLLVMOption -lLLVMObjectYAML -lLLVMObjCopy -lLLVMMCA -lLLVMMCDisassembler -lLLVMLTO -lLLVMPasses -lLLVMCFGuard -lLLVMCoroutines -lLLVMObjCARCOpts -lLLVMipo -lLLVMVectorize -lLLVMLinker -lLLVMInstrumentation -lLLVMFrontendOpenMP -lLLVMFrontendOpenACC -lLLVMExtensions -lLLVMDWARFLinker -lLLVMGlobalISel -lLLVMMIRParser -lLLVMAsmPrinter -lLLVMSelectionDAG -lLLVMCodeGen -lLLVMIRReader -lLLVMAsmParser -lLLVMInterfaceStub -lLLVMFileCheck -lLLVMFuzzMutate -lLLVMTarget -lLLVMScalarOpts -lLLVMInstCombine -lLLVMAggressiveInstCombine -lLLVMTransformUtils -lLLVMBitWriter -lLLVMAnalysis -lLLVMProfileData -lLLVMSymbolize -lLLVMDebugInfoPDB -lLLVMDebugInfoMSF -lLLVMDebugInfoDWARF -lLLVMObject -lLLVMTextAPI -lLLVMMCParser -lLLVMMC -lLLVMDebugInfoCodeView -lLLVMBitReader -lLLVMFuzzerCLI -lLLVMCore -lLLVMRemarks -lLLVMBitstreamReader -lLLVMBinaryFormat -lLLVMTableGen -lLLVMSupport -lLLVMDemangle -lm -lc++ -lpcre2-8 -lgc -levent -liconv -lSystem /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/15.0.0/lib/darwin/libclang_rt.osx.a
ld: warning: ignoring file '/Users/miry/src/crystal/crystal/.build/crystal.o': found architecture 'x86_64', required architecture 'arm64'
ld: Undefined symbols:
  _main, referenced from:
      
clang: error: linker command failed with exit code 1 (use -v to see invocation)

5. If I run the last cc with -v -target==x86_64-apple-darwin, than it shows:

Run cc -v -target x86_64-apple-darwin ...

cc .build/crystal.o -o .build/crystal  -rdynamic -L/opt/homebrew/Cellar/crystal/1.10.1/bin/../../../../lib /Users/miry/src/crystal/crystal/src/llvm/ext/llvm_ext.o `"/usr/local/bin/llvm-config" --libs --system-libs --ldflags 2> /dev/null` -lstdc++ -lpcre2-8 -lgc -L/opt/homebrew/Cellar/libevent/2.1.12_1/lib -levent -liconv -v -target x86_64-apple-darwin
Apple clang version 15.0.0 (clang-1500.0.40.1)
Target: x86_64-apple-darwin
Thread model: posix
InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin
 "/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ld" -demangle -export_dynamic -lto_library /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/libLTO.dylib -dynamic -arch x86_64 -platform_version macos 14.0.0 14.0 -syslibroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk -o .build/crystal -L/opt/homebrew/Cellar/crystal/1.10.1/bin/../../../../lib -L/usr/local/lib -L/opt/homebrew/Cellar/libevent/2.1.12_1/lib -L/usr/local/lib .build/crystal.o /Users/miry/src/crystal/crystal/src/llvm/ext/llvm_ext.o -search_paths_first -headerpad_max_install_names -lLLVMWindowsManifest -lLLVMWindowsDriver -lLLVMXRay -lLLVMLibDriver -lLLVMDlltoolDriver -lLLVMCoverage -lLLVMLineEditor -lLLVMAArch64Disassembler -lLLVMAArch64AsmParser -lLLVMAArch64CodeGen -lLLVMAArch64Desc -lLLVMAArch64Utils -lLLVMAArch64Info -lLLVMX86TargetMCA -lLLVMX86Disassembler -lLLVMX86AsmParser -lLLVMX86CodeGen -lLLVMX86Desc -lLLVMX86Info -lLLVMOrcJIT -lLLVMMCJIT -lLLVMJITLink -lLLVMInterpreter -lLLVMExecutionEngine -lLLVMRuntimeDyld -lLLVMOrcTargetProcess -lLLVMOrcShared -lLLVMDWP -lLLVMDebugInfoGSYM -lLLVMOption -lLLVMObjectYAML -lLLVMObjCopy -lLLVMMCA -lLLVMMCDisassembler -lLLVMLTO -lLLVMPasses -lLLVMCFGuard -lLLVMCoroutines -lLLVMObjCARCOpts -lLLVMipo -lLLVMVectorize -lLLVMLinker -lLLVMInstrumentation -lLLVMFrontendOpenMP -lLLVMFrontendOpenACC -lLLVMExtensions -lLLVMDWARFLinker -lLLVMGlobalISel -lLLVMMIRParser -lLLVMAsmPrinter -lLLVMSelectionDAG -lLLVMCodeGen -lLLVMIRReader -lLLVMAsmParser -lLLVMInterfaceStub -lLLVMFileCheck -lLLVMFuzzMutate -lLLVMTarget -lLLVMScalarOpts -lLLVMInstCombine -lLLVMAggressiveInstCombine -lLLVMTransformUtils -lLLVMBitWriter -lLLVMAnalysis -lLLVMProfileData -lLLVMSymbolize -lLLVMDebugInfoPDB -lLLVMDebugInfoMSF -lLLVMDebugInfoDWARF -lLLVMObject -lLLVMTextAPI -lLLVMMCParser -lLLVMMC -lLLVMDebugInfoCodeView -lLLVMBitReader -lLLVMFuzzerCLI -lLLVMCore -lLLVMRemarks -lLLVMBitstreamReader -lLLVMBinaryFormat -lLLVMTableGen -lLLVMSupport -lLLVMDemangle -lm -lc++ -lpcre2-8 -lgc -levent -liconv -lSystem /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/15.0.0/lib/darwin/libclang_rt.osx.a
ld: warning: ignoring file '/Users/miry/src/crystal/crystal/src/llvm/ext/llvm_ext.o': found architecture 'arm64', required architecture 'x86_64'
ld: warning: ignoring file '/opt/homebrew/Cellar/pcre2/10.42/lib/libpcre2-8.0.dylib': found architecture 'arm64', required architecture 'x86_64'
ld: warning: ignoring file '/opt/homebrew/Cellar/bdw-gc/8.2.4/lib/libgc.1.dylib': found architecture 'arm64', required architecture 'x86_64'
ld: warning: ignoring file '/opt/homebrew/Cellar/libevent/2.1.12_1/lib/libevent-2.1.7.dylib': found architecture 'arm64', required architecture 'x86_64'
ld: warning: no platform load command found in '/Users/miry/src/crystal/crystal/.build/crystal.o', assuming: macOS
ld: Undefined symbols:
  _GC_base, referenced from:
      _*Log::for:Log in crystal.o
...
clang: error: linker command failed with exit code 1 (use -v to see invocation)

[miry]

Continue working on building x86_64 build.
Made a preparation for libraries to have universal versions:

$ ls -1 ~/tmp/crystal/embedded/lib
libcord.a
libcord.la
libevent.a
libevent.la
libevent_core.a
libevent_core.la
libevent_extra.a
libevent_extra.la
libevent_pthreads.a
libevent_pthreads.la
libffi.8.dylib
libffi.a
libffi.dylib
libffi.la
libgc.a
libgc.la
libpcre2-8.a
libpcre2-8.la
libpcre2-posix.a
libpcre2-posix.la
pkgconfig

$ file ~/tmp/crystal/embedded/lib/libpcre2-8.a 
/Users/miry/tmp/crystal/embedded/lib/libpcre2-8.a: Mach-O universal binary with 2 architectures: [x86_64:current ar archive random library] [arm64:current ar archive random library]
/Users/miry/tmp/crystal/embedded/lib/libpcre2-8.a (for architecture x86_64):	current ar archive random library
/Users/miry/tmp/crystal/embedded/lib/libpcre2-8.a (for architecture arm64):	current ar archive random library

$ file ~/tmp/crystal/embedded/lib/libgc.a 
/Users/miry/tmp/crystal/embedded/lib/libgc.a: Mach-O universal binary with 2 architectures: [x86_64:current ar archive random library] [arm64:current ar archive random library]
/Users/miry/tmp/crystal/embedded/lib/libgc.a (for architecture x86_64):	current ar archive random library
/Users/miry/tmp/crystal/embedded/lib/libgc.a (for architecture arm64):	current ar archive random library

$ file ~/tmp/crystal/embedded/lib/libevent.a
/Users/miry/tmp/crystal/embedded/lib/libevent.a: Mach-O universal binary with 2 architectures: [x86_64:current ar archive random library] [arm64:current ar archive random library]
/Users/miry/tmp/crystal/embedded/lib/libevent.a (for architecture x86_64):	current ar archive random library
/Users/miry/tmp/crystal/embedded/lib/libevent.a (for architecture arm64):	current ar archive random library

Partially made some make command to build x86_64, but still required in the end to specify explictly for cc the argument -target x86_64-apple-darwin

The command generated by make:

cc /Users/miry/tmp/crystal/embedded/bin/crystal.o -o /Users/miry/tmp/crystal/embedded/bin/crystal -Wl,-rpath,/Users/miry/tmp/crystal/embedded/lib -L/Users/miry/tmp/crystal/embedded/lib -rdynamic -L/Users/miry/tmp/crystal/embedded/lib /Users/miry/src/crystal/crystal/src/llvm/ext/llvm_ext.o `"/usr/local/bin/llvm-config" --libs --system-libs --ldflags 2> /dev/null` -lstdc++ -lpcre2-8 -lgc -L/Users/miry/tmp/crystal/embedded/lib -levent -liconv

With modifing it with -target, the result was successfull:

$ file ~/tmp/crystal/embedded/bin/crystal
~/tmp/crystal/embedded/bin/crystal: Mach-O 64-bit executable x86_64

[miry]

I wonder how I can add extra flags to cc?

I tried overried LDFLAGS, that propogated to link-flags, still did not work:

CRYSTAL_CONFIG_BUILD_COMMIT="1f592eca6" CRYSTAL_CONFIG_PATH='$ORIGIN/../share/crystal/src' SOURCE_DATE_EPOCH="1699639716"  CRYSTAL_CONFIG_LIBRARY_PATH= ./bin/crystal build --no-debug -D strict_multi_assign -D preview_overload_order --release --stats --link-flags="-Wl,-rpath,/Users/miry/tmp/crystal/embedded/lib -L/Users/miry/tmp/crystal/embedded/lib -v -target x86_64-apple-darwin" --cross-compile --target x86_64-apple-darwin -Dwithout_interpreter  -o /Users/miry/tmp/crystal/embedded/bin/crystal src/compiler/crystal.cr -D without_openssl -D without_zlib -D use_pcre2
                          D | 2023-11-12T10:35:24+01:00 | Using compiled compiler at .build/crystal
                          D | 2023-11-12T10:35:42+01:00 | ld: warning: ignoring file '/Users/miry/.cache/crystal/Users-miry-tmp-local-src-crystal-src-ecr-process.cr/_main.o3.o': found architecture 'x86_64', required architecture 'arm64'
                          D | 2023-11-12T10:35:42+01:00 | ld: Undefined symbols:
                          D | 2023-11-12T10:35:42+01:00 |   _main, referenced from:
                          D | 2023-11-12T10:35:42+01:00 |       <initial-undefines>
                          D | 2023-11-12T10:35:42+01:00 | clang: error: linker command failed with exit code 1 (use -v to see invocation)
                          D | 2023-11-12T10:35:42+01:00 | Error: execution of command failed with exit status 1: cc "${@}" -o /Users/miry/.cache/crystal/Users-miry-tmp-local-src-crystal-src-ecr-process.cr/macro_run  -rdynamic -L/Users/miry/tmp/crystal/embedded/lib -lpcre2-8 -lgc -L/Users/miry/tmp/crystal/embedded/lib -levent -liconv
gmake[2]: *** [Makefile:239: /Users/miry/tmp/crystal/embedded/bin/crystal] Error 1          

[miry]

Found the problem. In my scenario i used embeded crystal. It seems it does not have cross-platform features. I managed at least to compile x86_64 version.