Skip to content

Conversation

@mrexodia
Copy link
Contributor

@mrexodia mrexodia commented Oct 26, 2025

This is a huge refactor doing the following things:

  • Add an alternative cxx-common implementation as a superbuild project. You can find more information here: https://github.com/LLVMParty/packages/blob/main/dependencies.md. This is of course optional, but makes it much easier to users to build the dependencies necessary for remill.
  • Cross-compile the semantics instead of the current hack with g++-multilib and -m32. The semantics are now compiled as -ffreestanding and just enough of the C++ standard library was implemented in a fake --sysroot to make it work.
  • Change how FPU exceptions are handled. Previously they would call std::fetestexcept and directly use FE_XXX flags. This has now been decoupled properly so you can implement FPU exceptions however you want. It looks like Adding new type 'MV256W' and enabling AArch64 STP SIMD semantics. #636 was discussing similar bugs, all of which have been fixed (I think).
  • Properly handle patches for sleigh without a separate Ghidra fork for PPC
  • Fix a few bugs in semantics uncovered by porting to -ffreestanding and test failures with later LLVM versions
  • The project now builds and runs on LLVM 15-21. There are some #ifdefs for LLVM versions, but I think it would be nice to widen the compatibility instead of only supporting LLVM 17. Due to bugs in LLVM JSON this required switching back to C++17, but the project uses no C++20 features, so I don't think this is an issue.
  • Build and run on Windows (clang and clang-cl)
  • Fix -fsanitize (@fvrmatteo's work)
  • Properly document the superbuild dependency management solution in the README. This pattern is poorly documented, but it solves essentially all problems with CMake dependency management and keeps things flexible for integration into package managers etc.
  • New GitHub Actions CI that builds dependencies from scratch (except LLVM, which is installed by the package manager)

Closes #678
Closes #718
Closes #663
Closes #573

Unfortunately splitting this up is going to be extremely tough. I can split the changes for LLVM15-21 compatibility, but the -ffreestanding changes are dependent on the FPU semantic changes and a bunch of CMake stuff. The sleigh "changes" (nothing actually changed) could be split out too, the diff just looks massive.

Note: The CI failure is something I cannot reproduce locally with any LLVM version. I suspect it is specific to the LLVM fork used in cxx-common. The docker images I use are official LLVM versions without changes, compiled with assertions enabled.

@mrexodia mrexodia force-pushed the cxx-common-alternative branch 9 times, most recently from aa16cb9 to 43b2024 Compare October 31, 2025 00:48
@mrexodia mrexodia changed the title cxx-common alternative Major refactor Oct 31, 2025
@mrexodia mrexodia force-pushed the cxx-common-alternative branch 2 times, most recently from b71a557 to f678379 Compare October 31, 2025 14:03
@mrexodia mrexodia marked this pull request as ready for review October 31, 2025 14:55
@mrexodia mrexodia force-pushed the cxx-common-alternative branch 3 times, most recently from f8bbbc9 to 8550a94 Compare October 31, 2025 15:59
@mrexodia mrexodia force-pushed the cxx-common-alternative branch from 8550a94 to 57d01d6 Compare November 1, 2025 01:33
// These hypercalls are not executed in normal test workflows anyway
__builtin_debugtrap();
break;
#else
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is okay based off the comments left in #625 but we are removing the explicit data dependencies. Without turning this into a naked block (which would be very brittle), not sure would could be done.

@mrexodia mrexodia force-pushed the cxx-common-alternative branch 2 times, most recently from 834a92c to a413225 Compare November 3, 2025 22:19
Copy link
Collaborator

@kyle-elliott-tob kyle-elliott-tob left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for all the hard work over the past few days, LGTM!

Waiting on @wizardengineer for review

Copy link
Collaborator

@wizardengineer wizardengineer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks again!

@cctv130
Copy link
Contributor

cctv130 commented Nov 5, 2025

  1. I compiled using the main branch of https://github.com/LLVMParty/packages and encountered an error where the type uint32_t was not found. Adding #include <cstdint> fixed this issue.
64ab69ba652566459201bb620ce4ac43 2. For the **cxx-common-alternative** version of Remill, which version of XED does it correspond to? I encountered an error, but when using the XED built from the **main** branch of *packages*, the following cases compile successfully:
case XED_ISA_SET_AVX512DQ_KOPB:
case XED_ISA_SET_AVX512DQ_KOPW:
8184caa9d221190affd8beea4ee1d0e3
  1. When using the cross-compilation option (-ffreestanding), I got a standard library missing error:

    fatal error: 'bits/c++config.h' file not found
    [build] 59 | #include <bits/c++config.h>
    
96f2d846c2d9a74a55e24c1d336063d1 4. ``` clang-17: warning: no such sysroot directory: '-m32' [-Wmissing-sysroot] /Users/runner/work/remill/remill/lib/Arch/AArch32/Runtime/BasicBlock.cpp:17:10: fatal error: 'algorithm' file not found 17 | #include ```

Today, when I compiled packages on an Ubuntu Server platform, I unfortunately encountered a similar “cannot find <algorithm>” error as in GitHub Actions.
I treated the -m32 message as just a warning and ignored it, focusing instead on fixing the <algorithm> issue.
Eventually, by ensuring all standard library headers were fully available and removing the -ffreestanding flag, the build succeeded.

However, when using the -ffreestanding flag, I got other errors such as:

error: no member named '__throw_domain_error' in namespace 'std'; did you mean '__throw_runtime_error'?

Although using -ffreestanding for cross-compilation might be a good idea in theory, there currently seems to be no good way to resolve these missing standard library issues.
For users who do not want to perform cross-compilation, it might be helpful to provide a configuration switch to decide whether to enable cross-compilation or not.

@mrexodia
Copy link
Contributor Author

mrexodia commented Nov 5, 2025

I compiled using the main branch

This is not a supported configuration, so there is no reason to try this. There also is no -m32 flag anymore, so you did something wrong. Newer XED versions are not supported and cause lifting problems, so I use the same one as the original cxx-common.

@mrexodia mrexodia force-pushed the cxx-common-alternative branch from a413225 to 0dfe75e Compare November 5, 2025 11:47
@cctv130
Copy link
Contributor

cctv130 commented Nov 5, 2025

I compiled using the main branch

This is not a supported configuration, so there is no reason to try this. There also is no -m32 flag anymore, so you did something wrong. Newer XED versions are not supported and cause lifting problems, so I use the same one as the original cxx-common.

Can cxx-common currently run on the Windows platform ,Are there other ways to avoid relying on vcpkg

@cctv130
Copy link
Contributor

cctv130 commented Nov 5, 2025

I compiled using the main branch

This is not a supported configuration, so there is no reason to try this. There also is no -m32 flag anymore, so you did something wrong. Newer XED versions are not supported and cause lifting problems, so I use the same one as the original cxx-common.

Can cxx-common currently run on the Windows platform ,Are there other ways to avoid relying on vcpkg

Sorry, my English isn’t great, but I understand you now.

@kyle-elliott-tob
Copy link
Collaborator

kyle-elliott-tob commented Nov 5, 2025

cxx-common will not compile out of the box on windows. There's (currently) a few different issues, all of which our resolved by this commit. Instructions will eventually be updated in the main readme to reflect this after the merge.

@mrexodia
Copy link
Contributor Author

mrexodia commented Nov 5, 2025

You can build the remill fully on Windows with the dependencies folder (previously named cxx-common, but that was confusing), I will soon push some updates with a README explaining how to do it.

@mrexodia mrexodia force-pushed the cxx-common-alternative branch 20 times, most recently from 99ad47b to 3524cf3 Compare November 5, 2025 23:23
@mrexodia mrexodia force-pushed the cxx-common-alternative branch from 3524cf3 to 3dd93c7 Compare November 5, 2025 23:55
@kyle-elliott-tob kyle-elliott-tob merged commit 4689097 into lifting-bits:master Nov 6, 2025
15 checks passed
@kyle-elliott-tob
Copy link
Collaborator

Thank you very much again @mrexodia, a truly herculean effort that is greatly appreciated!

@kyle-elliott-tob kyle-elliott-tob mentioned this pull request Nov 6, 2025
@mrexodia mrexodia deleted the cxx-common-alternative branch November 24, 2025 00:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Readme suggest LLVM 15+ but it seems only to work till LLVM 17 C++ header file not found windows下构建的问题

6 participants