Skip to content

Commit edb3865

Browse files
committed
lightningd: avoid high memory usage spike at startup.
I saw this while watching top, and tracked it down. We load all the plugins to checksum them at startup: ``` $ ms_print massif.out.3312805 | head -n 50 -------------------------------------------------------------------------------- Command: lightningd/lightningd.real --developer --log-level=trace --cltv-delta=6 --cltv-final=5 --watchtime-blocks=5 --rescan=1 --disable-dns --lightning-dir=/tmp/ltests-roazlc8h/test_xpay_fake_channeld_1/lightning-1/ --addr=127.0.0.1:46337 --allow-deprecated-apis=false --network=regtest --ignore-fee-limits=false --bitcoin-rpcuser=rpcuser --bitcoin-rpcpassword=rpcpass --bitcoin-datadir=/tmp/ltests-roazlc8h/test_xpay_fake_channeld_1/lightning-1/ --dev-fast-gossip --dev-bitcoind-poll=1 --log-file=- --log-file=/tmp/ltests-roazlc8h/test_xpay_fake_channeld_1/lightning-1/log --log-prefix=lightningd-1 --dev-fail-on-subdaemon-fail --dev-no-reconnect --autoconnect-seeker-peers=0 --subdaemon=channeld:../tests/plugins/channeld_fakenet --dev-throttle-gossip --grpc-port=37819 --dev-crash-after=3600 --bitcoin-rpcport=51623 Massif arguments: (none) ms_print arguments: massif.out.3312805 -------------------------------------------------------------------------------- MB 446.5^# |# |# |# |# |# |# |# |# |# |# |# |# |# |# |# |# |# |# |# :: : ::@@:@:::::::::::::::::::::::::::@@::::::::::::::@::::@::::::: 0 +----------------------------------------------------------------------->Gi 0 75.66 Number of snapshots: 57 Detailed snapshots: [1 (peak), 2, 10, 12, 33, 45, 49] -------------------------------------------------------------------------------- n time(i) total(B) useful-heap(B) extra-heap(B) stacks(B) -------------------------------------------------------------------------------- 0 0 0 0 0 0 1 295,677,530 468,189,872 447,087,069 21,102,803 0 95.49% (447,087,069B) (heap allocation functions) malloc/new/new[], --alloc-fns, etc. ->95.35% (446,440,727B) 0x2EFB99: tal_resize_ (tal.c:755) | ->94.23% (441,180,860B) 0x2EBCD1: grab_fd (grab_file.c:45) | | ->94.23% (441,180,860B) 0x2EBD54: grab_file (grab_file.c:63) | | ->94.23% (441,180,860B) 0x1A5CF7: file_checksum (plugin.c:315) | | ->94.23% (441,180,860B) 0x1A5EF1: plugin_register (plugin.c:355) | | ->94.23% (441,180,860B) 0x1AC62E: plugins_set_builtin_plugins_dir (plugin.c:2532) | | ->94.23% (441,180,860B) 0x16D614: find_subdaemons_and_plugins (lightningd.c:569) | | ->94.23% (441,180,860B) 0x16E9A1: main (lightningd.c:1226) | | | ->01.11% (5,219,417B) 0x2EBC32: grab_fd (grab_file.c:38) ``` Signed-off-by: Rusty Russell <[email protected]>
1 parent 922d78e commit edb3865

File tree

1 file changed

+6
-2
lines changed

1 file changed

+6
-2
lines changed

lightningd/plugin.c

Lines changed: 6 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -308,13 +308,17 @@ static void destroy_plugin(struct plugin *p)
308308
static u32 file_checksum(struct lightningd *ld, const char *path)
309309
{
310310
char *content;
311+
u32 crc;
311312

312313
if (ld->dev_no_plugin_checksum)
313314
return 0;
314315

315316
content = grab_file(tmpctx, path);
316-
if (content == NULL) return 0;
317-
return crc32c(0, content, tal_count(content));
317+
crc = crc32c(0, content, tal_count(content));
318+
/* We could leave this around, but we checksum many files in a loop,
319+
* causing 450MB of allocations at startup! */
320+
tal_free(content);
321+
return crc;
318322
}
319323

320324
struct plugin *plugin_register(struct plugins *plugins, const char* path TAKES,

0 commit comments

Comments
 (0)