geth takes over an hour to start with a lot of keys in the keystore

[Crypto2]

Hi,

This has been going on for a long time but geth takes longer and longer to start up. Currently using the pre-compiled 1.8.10 on Linux 64-bit. Pretty beefy machine with 128GB of RAM, NVMe storage, 3 GHz Xeon w/ 8 cores / 16 threads.

My best guess is it's because we have a lot of keys since it's for a wallet provider.

Quick example from today showing timestamps between these 2 lines of geth output:

INFO [06-04|04:26:23] Maximum peer count ETH=25 LES=0 total=25
INFO [06-04|05:38:10] Starting peer-to-peer node instance=Geth/v1.8.10-stable-eae63c51/linux-amd64/go1.10.

[jsvisa]

The time before node starting is the sync time, you can check out #16875 (comment) to see more

[Crypto2]

This is before syncing ever starts. These are the 1st two lines of geth output.

[timnugent]

Same issue here with 1.8.10 - also a powerful machine and previously synced up to block ~5770000. Only a few keys on this one.

[Crypto2]

Any update on this? It persists in 1.8.12.

[Crypto2]

At least in my case it does seem to be related to having a lot of keys, I renamed my keystore folder and it started up right away as would be expected.

[Crypto2]

@jsvisa Yep, just tried this on another machine with the same result.
with keystore = over an hour to start, without keystore = immediate start

[jsvisa]

@Crypto2 How much keys under keystore do you have? I'm try to start geth with 100 keys and no more much time to start in my local macOS.

[Crypto2]

Several million, they should be loaded/scanned in another thread or something?

[Crypto2]

Any updates on this?

[Crypto2]

Any updates? Still takes hours to start.

If I start it with keystore renamed then move the files back in will it at least scan them in the background then even if it takes a long time? So we can process the blockchain and do things besides sending at least?

[Crypto2]

@jsvisa Any updates? I can do the rename as indicated in my last post but it takes nearly a day for the keys to scan in the background. There has to be some way to speed this up?

[ligi]

How many keys do you have in your keystore? And what are your cache settings?

[holiman]

@Crypto2 could you provide your startup flags ?