Continuous memory profiling

[ndr-ds]

Motivation

jemalloc is a more performant allocator (we do see a small performance improvement in benchmarks from it), and if we use it it's easier for us to have memory profiles.

Proposal

This adds support for using jemalloc as an allocator, as well as expose a couple endpoints:

• One that generates a pprof and returns it once the endpoint is queried
• One that generates a flamegraph and returns it once the endpoint is queried

Test Plan

Deployed a network with this and the linera-infra changes, saw the memory flamegraphs on Pyroscope, saw the memory profile Grafana dashboard, and managed to curl the flamegraph endpoint and get a memory profile svg returned to me.

Release Plan

• Nothing to do / These changes follow the usual release cycle.

[ndr-ds]

• Continuous memory profiling #4498 👈 (View in Graphite)
• Optimize RocksDB prefix searches #4487 : 1 other dependent PR (#4571 )
• eBPF Pyroscope #4469
• main

This stack of pull requests is managed by Graphite. Learn more about stacking.