ERROR : fulcrum | <jemalloc> invalid page size

[Fredvs79]

I have a raspberryPi 5 that I was trying to run an instance of dojo
https://github.com/Samourai-Wallet/samourai-dojo
Latest maintained version here

The dojo software package optionally includes installing fulcrum as an indexer, but I was getting an error from fulcrum.
fulcrum | <jemalloc> unsupported system page size
I found references to this online here and here.

The issue is that rPi 5 running bookworm is optimized over the pi 4 to use 16K page sizes. While Debian arch64 uses 4K page sizes. My thought is that when dojo installs fulcrum, fulcrum is grabbing a pre-compiled binary of jemalloc during install which is default configured on the host machine for 4K page size.

A quick workaround is to tell RaspberryOS to use a different kernel image that uses 4K page size.
/boot/firmware/config.txt set kernel=kernel8.img
This indeed works.

However, I would like to see if it is possible for jemalloc to be configured for 16K? I found some discussion on this as well. I see that jemalloc does reference being able to set a specific page size during compile.

--with-lg-page=<lg-page>

• Specify the base 2 log of the allocator page size, which must in turn be at least as large as the system page size. By default the configure script determines the host's page size and sets the allocator page size equal to the system page size, so this option need not be specified unless the system page size may change between configuration and execution, e.g. when cross compiling.

So if I download the source and compile jemalloc myself and set --with-lg-page=4, it should run on default kernel with 16K page size. Yes?

My question is how does fulcrum integrate/install jemalloc? Isn't jemalloc a dependency? Does fulcrum specify any flags for jemalloc, or just defaults? I ask because I want to configure jemalloc to build as similarly to what fulcrum expects.

[Fredvs79]

I've tried sleuthing out a bit about how dojo installs jemalloc.
I notice that Debian bookworm uses jemalloc 5.2.21, while trixie uses 5.3.0.
If we're going to fix jemalloc might as well update it too.
The path I thought I was headed down was to grab the source, compile it with some flags, and install it over the existing libjemalloc.so.2 file.
Then I realized that I may not want to go that route as that would create a binary for /bin
However, because it's used by docker it's in the overlay2 union FS.
Unfortunately, trying to understand how docker builds the current version and sets built-time variables puts it outside my wheelhouse.

Is there any way docker can specify build-variables in the install script for jemalloc when installing fulcrum?
The two flags I want to set are
--with-lg-page=16 - this would set 64K page file size, which would be compatible with 64K as well as 16K and 4K kernel page sizes. Discussed here
--lg-hugepage=21 - this would optimize VSIZE as discussed here

[cculianu]

Hey I'm traveling for a conference but it looks like you may just have to build everything from scratch. Fulcrum builds jemalloc in a docker container using defaults for that machine which I guess in this case is 4kb pages. No, fulcrum doesn't pass any flags to jemalloc. If you want to sleuth more the trail starts at contrib/build/build.sh ...

[Fredvs79]

Thanks for answering. That does help, but since the fulcrum install scripts are called by dojo install scripts, I'm not sure I will be able to modify any of those fulcrum scripts.

I think my best hope is directly replace the library. ChatGPT to the rescue on how to do that.

• build jemalloc with the build flags I set
• identify the docker container docker images
• run the docker container with fulcrum docker run
• use the docker cp command to copy files into the container and replace the libarary
• commit the container changes to a new image