Ensure snmalloc properly linked

[maxtropets]

Ensure that if -DUSE_SNMALLOC=ON, then all CCF parts (and apps built on top of it) use snmalloc for both new/delete and malloc/free.

• Ensure linking/using snmalloc in e2e test apps (Check snmalloc linkage #7180).
• Provide convenient linkage check for ccf apps? Still learning this bit... UPD: won't really do, as stated in Ensure snmalloc properly linked #7172 (comment) it's up to the user to screw it up. Will do p1 though, to make sure that it's used by default on ccf apps
• Release the script to check snmalloc too, and check the user's app can be auto-cpacked and checked: Revisit CCF packaging #7187

[maxtropets]

Tried playing with app cmake

cmake_minimum_required(VERSION 3.21)

project(basic LANGUAGES C CXX)

set(CCF_PROJECT "ccf")

add_library(nasty_business SHARED piggyback.cpp)

target_link_options(nasty_business PRIVATE -static-libstdc++)

if(NOT TARGET "ccf")
  find_package(${CCF_PROJECT} REQUIRED)
endif()

add_ccf_app(basic SRCS basic.cpp ${CCF_DIR}/samples/apps/main.cpp)

target_link_libraries(basic PRIVATE nasty_business)

piggyback.cpp:

size_t nasty_business()
{
  fprintf(stderr, "oink, oink!\n");
  fflush(stderr);
  int* x = new int(32);
  fprintf(stderr, "oink, allocationk!\n");
  fflush(stderr);

  delete x;
  return (size_t)x;
}

Patched snmalloc to produce output on new/malloc, calling nasty_business in basic app triggers snmalloc new, however it's supposed to be statically linked to libstdc++, and it is:

maxtropets@great-vm [ ~/workspace/CCF/build ]$ ldd samples/apps/basic/libnasty_business.so
        linux-vdso.so.1 (0x00007ffca353b000)
        libm.so.6 => /usr/lib/libm.so.6 (0x0000717c746a0000)
        libgcc_s.so.1 => /usr/lib/libgcc_s.so.1 (0x0000717c7467b000)
        libc.so.6 => /usr/lib/libc.so.6 (0x0000717c74400000)
        /lib64/ld-linux-x86-64.so.2 (0x0000717c747ae000)
maxtropets@great-vm [ ~/workspace/CCF/build ]$ nm -C samples/apps/basic/libnasty_business.so  | grep new
000000000001a2c0 b (anonymous namespace)::__new_handler
0000000000007d70 T std::get_new_handler()
0000000000007d60 T std::set_new_handler(void (*)())
0000000000006ae0 T operator new(unsigned long)
0000000000006490 t operator new(unsigned long) [clone .cold]

But allocations say

snmalloc new
oink, oink!
snmalloc new
oink, allocationk!
snmalloc new
--
snmalloc new
oink, oink!
snmalloc new
oink, allocationk!
snmalloc new

How come?.. This's what ninja says

[1/1] : && /usr/bin/clang++ -g -Xlinker --no-undefined -Xlinker --undefined=enclave_create_node -Xlinker --undefined=enclave_run -rdynamic samples/apps/basic/CMakeFiles/basic.dir/basic.cpp.o samples/apps/basic/CMakeFiles/basic.dir/__/main.cpp.o -o samples/apps/basic/basic  -Wl,-rpath,/home/maxtropets/workspace/CCF/build/samples/apps/basic  libccf.a  libccf_launcher.a  samples/apps/basic/libnasty_business.so  libccf_js.a  libquickjs.a  libccf_endpoints.a  libccf_kv.a  -lnghttp2  libhttp_parser.a  libccfcrypto.a  libt_cose.a  libqcbor.a  -lcrypto  -lssl  3rdparty/exported/snmalloc/libsnmalloc-new-override.a  3rdparty/exported/snmalloc/libsnmallocshim-static.a  -luv  /usr/lib/libssl.so  -ldl  -lcurl

Is it because static libs go first, and than ld tries resolving symbols from dynamically linked nasty.so, but omits new/delete as already defined?..

[maxtropets]

Been fiddling with basic app's cmake more.

Tried statically linking against stdlibc++ to see what happens to new/delete implementations.

Added some recognisable stuff to all snmalloc allocation entry points

  const char* msg = "oopsie-doopsie-snmalloc-case-X";
  fprintf(stderr, "%s\n", msg);
  fflush(stderr);

Both with/without target_link_options(basic PRIVATE -static-libstdc++) getting snmalloc's versions.

I'm not sure why though, is it always guaranteed that any library will be linked before stdlib? IDK.

I've also checked some quick js internals, like for instance:

static void *js_def_malloc(JSMallocState *s, size_t size)
{
    void *ptr;

    /* Do not allocate zero bytes: behavior is platform dependent */
    assert(size != 0);

    if (unlikely(s->malloc_size + size > s->malloc_limit))
        return NULL;

    ptr = malloc(size);
    if (!ptr)
        return NULL;

    s->malloc_count++;
    s->malloc_size += js_def_malloc_usable_size(ptr) + MALLOC_OVERHEAD;
    return ptr;
}

Calls to 11928ec: e8 7f 34 1d 00 call 1365d70 <malloc> which is

0000000001365d70 <malloc>:
 1365d70:	48 81 ec 88 05 00 00 	sub    $0x588,%rsp
 1365d77:	48 89 bc 24 88 01 00 	mov    %rdi,0x188(%rsp)
 1365d7e:	00 
 1365d7f:	48 8b 84 24 88 01 00 	mov    0x188(%rsp),%rax
 1365d86:	00 
 1365d87:	48 89 84 24 90 01 00 	mov    %rax,0x190(%rsp)
 1365d8e:	00 
 1365d8f:	48 8b 05 9a 9c 50 00 	mov    0x509c9a(%rip),%rax        # 186fa30 <stderr@GLIBC_2.2.5>
 1365d96:	48 8b 38             	mov    (%rax),%rdi
 1365d99:	48 8d 35 ad da 18 00 	lea    0x18daad(%rip),%rsi        # 14f384d <_IO_stdin_used+0x7984d>
 1365da0:	31 c0                	xor    %eax,%eax
 1365da2:	e8 79 f7 72 ff       	call   a95520 <fprintf@plt>
 1365da7:	e9 00 00 00 00       	jmp    1365dac <malloc+0x3c>
 1365dac:	48 8b 05 7d 9c 50 00 	mov    0x509c7d(%rip),%rax        # 186fa30 <stderr@GLIBC_2.2.5>
 1365db3:	48 8b 38             	mov    (%rax),%rdi
 1365db6:	e8 95 e9 72 ff       	call   a94750 <fflush@plt>

which corresponds to the code snippet I've inserted.

[maxtropets]

I'm not sure why though, is it always guaranteed that any library will be linked before stdlib? IDK.

Mistery solved (hopefully). Seems like -static-libstdc++ is taking affect applying all the other libs. Thus, linking as

I'm not sure why though, is it always guaranteed that any library will be linked before stdlib? IDK.

/usr/bin/clang++  -static-libstdc++ ...

works just fine, because the overloaded snmalloc goes linked in first.

If anyone was to force-link libstdc++ as

/usr/bin/clang++  /path/to/libstdc++ ... 

then we have no power over this.