Windows: how to override malloc / free / new / delete globally ?

[jcelerier]

The simple following program crashes if linking against snmallocshim-static (MSYS2, with clang64 but reproducible on any other MinGW environment):

main.cpp:

#include <QObject>

int main(int argc, char *argv[]) {
    delete new QObject;
    return 0;
}

$ clang++ main.cpp  \
   -I/clang64/include/qt6/ \
   -I/clang64/include/qt6/QtCore \
   -std=c++20 \
    /clang64/lib/libQt6Core.dll.a \
    libsnmallocshim-static.a \
   -lbcrypt -rdynamic

$ lldb ./a.exe
(lldb) r
Process 19300 stopped
* thread #1, stop reason = Exception 0x80000003 encountered at address 0x7ffbd2e9ca82
    frame #0: 0x00007ffbd2e9ca83 ntdll.dll`RtlIsZeroMemory + 163
ntdll.dll`RtlIsZeroMemory:
->  0x7ffbd2e9ca83 <+163>: jmp    0x7ffbd2e9ca91 ; <+177>
    0x7ffbd2e9ca85 <+165>: movq   0xf8(%rsp), %rdi
    0x7ffbd2e9ca8d <+173>: movl   0x20(%rsp), %ebx
    0x7ffbd2e9ca91 <+177>: movl   %ebx, 0x30(%rsp)

(lldb) bt
* thread #1, stop reason = Exception 0x80000003 encountered at address 0x7ffbd2e9ca82
  * frame #0: 0x00007ffbd2e9ca83 ntdll.dll`RtlIsZeroMemory + 163
    frame #1: 0x00007ffbd2ea5b5a ntdll.dll`__misaligned_access + 1066
    frame #2: 0x00007ffbd2ea5e3a ntdll.dll`__misaligned_access + 1802
    frame #3: 0x00007ffbd2eb1e35 ntdll.dll`__misaligned_access + 50949
    frame #4: 0x00007ffbd2dcc3bc ntdll.dll`RtlGetCurrentServiceSessionId + 4892
    frame #5: 0x00007ffbd2dcb001 ntdll.dll`RtlFreeHeap + 81
    frame #6: 0x00007ffbd016364b ucrtbase.dll`_free_base + 27
    frame #7: 0x00007ff65dd213f0 a.exe`main + 96
    frame #8: 0x00007ff65dd21303 a.exe`__tmainCRTStartup at crtexe.c:259:15
    frame #9: 0x00007ff65dd21366 a.exe`mainCRTStartup at crtexe.c:180:9
    frame #10: 0x00007ffbd134259d kernel32.dll`BaseThreadInitThunk + 29
    frame #11: 0x00007ffbd2deaf38 ntdll.dll`RtlUserThreadStart + 40

thus I assume I am linking something incorrectly and some allocations made with ucrt are free with snmalloc or conversely which then makes ucrt scream.

[mjp41]

Using the static library will only override the allocations in the exe, but I think you are picking up a DLL as well, which won't have the allocations overridden.

There are some horrible things other allocators do to achieve this by rewriting things inside UCRT. But I have never gotten around to this in snmalloc.

[mjp41]

Just going to add a comment of the various approaches I have found.

• There is minhooks
  • with an example for using it with an allocator here:
    https://gist.github.com/Erfan-Ahmadi/93369b8b7fbce77b1adf8e244ea33849
  • and an explanation of how it works here: https://www.codeproject.com/KB/winsdk/LibMinHook.aspx
• Micro-malloc has an open source override for Windows in here: https://github.com/Thermadiag/micromalloc/tree/main/micro_proxy
• tcmalloc the older version patches various functions here: https://github.com/gperftools/gperftools/blob/master/src/windows/patch_functions.cc

I don't know as I have much time to follow up on this at the moment, but adding this comment in case anyone else does. Otherwise, it is useful for me when I get more time to work on this.

[jcelerier]

I see, I thought that since mimalloc did it (https://microsoft.github.io/mimalloc/overrides.html) likely snmalloc too but I understand now that this was a bold assumption :)

[jcelerier]

That said, what I find weird is that in the call trace I see:

frame #6: 0x00007ffbd016364b ucrtbase.dll`_free_base + 27
frame #7: 0x00007ff65dd213f0 a.exe`main + 96

(I since did a -g -O0 build too and could confirm that it goes straight from my main to _free_base). I'd expect to go through snmalloc's operator delete in that case, no ?

[mjp41]

I just looked at the static library by default creates the symbols

  sn_malloc, sn_free, ...

You can override the prefix by passing to CMake:

-DSNMALLOC_STATIC_LIBRARY_PREFIX=""

This would create malloc, free, ..., etc but I have sometimes had problems with this on Windows depending on other build configurations.

If you don't want to override free/malloc/etc just the C++ new/delete, then you can link libsnmalloc-new-override.a instead. This I have had better success with on projects on Windows.

https://github.com/microsoft/snmalloc/blob/d6ad197d3c8452cd4d2548813e0b88a333ce11d6/CMakeLists.txt#L486C12-L486C33

I am going to take a bit of time to investigate building a proper override using Detours, which would work across multiple DLLs.

[SchrodingerZhu]

Also, windows has a _set_new_hander hook which we does not yet override.

[Andarwinux]

Is it possible to build a snmalloc dll that exports symbols prefixed with "mi" and disguise it as "mimalloc-override.dll" for use with minject(mimalloc-redirect.dll)?

[mjp41]

@jcelerier I got around to implementing the Windows heap override using Detours. If you are interested in trying it, then take a look at

#775

This PR is, I believe, functional but very lightly tested at the moment.