Performance Client 1st Connection

[status21]

When I open a complex wallet for the 1st time a day, balances are calculated slow similar to electrs performance. Then I close the wallet and open it again and the balances show up immediately. There is nothing cached by the wallet itself. The performance keeps high for a while but when I test the next day it's the same behaviour. I didn't experience this with v1 ever, only after I upgraded to v2. Are there any caching options or something I can configure?

[cculianu]

Hmm. What makes the wallet complex? Lots of addresses? So it has to subscribe and pull status for tons of addresses?

What are your fulcrum.conf settings?

[status21]

Several "accounts" (derivation paths from /0/... to /X/...) and lot of addresses and transactions, well not really a lot and as I said, the second time I open it its fast.
My config:
tcp = 0.0.0.0:50003
ssl = 0.0.0.0:50004
ws = 0.0.0.0:50005
wss = 0.0.0.0:50006
cert = /../tls.cert
key = /../tls.key
peering = false
announce = false
admin = 8000
stats = 0.0.0.0:8090
zmq_allow_hashtx = true

[cculianu]

Yeah it's fast second time because lots of the data is cached in memory (be it memtables from rocksdb or internal Fulcrum app caches for other things).

[cculianu]

Hmm so you're using defaults which are generous. Only thing I can think of is somehow your DB's levels got less-than-ideally optimized during the conversion from 1.x -> 2.x format. :(

Your 1.x was probably better "leveled" because it was built organically as you synched the blockchain. The upgrade process did some bulk copying and levels got less than ideally oriented somehow. :/

I suspect if you were to pay the significant price for a full DB resynch from scratch using 2.x it should perform better...

Or you can wait it out and the levels sort themselves out over time I think.. or you can try running fulcrum with --compact-dbs and maybe that helps.. no idea.

Sorry I can't be more definite here.. rocksdb is a bit of a black box and requires lots of research to fully understand its dark corners.

[status21]

I did the sync from scratch for v2 but thank you anyway :)

[cculianu]

Did it change anything? Also try --compact-dbs (one time flag).. maybe it will help?

[status21]

I tried --compact-dbs and it looks good for the moment. After fulcrum compacted and ran normally everything was fast even at the 1st start of the client. Will watch and try further.

[cculianu]

Phew yeah so something with the levels was off.. for the record it would have eventually resolved itself I think… due to passive auto compaction..

Glad that did it though

EDIT: for testing, exit fulcrum and restart it after compaction (without compacting anymore obviously).. and confirm it’s ok on 1st run…

[status21]

Ok restarted without compaction and everything seems OK. 1st run is fast again!

[cculianu]

Maybe I should put in logic to detect out-of-balance levels and recommend compaction or maybe do auto-compaction or something.. but compaction can take a while so really the admin should probably explicitly ask for it else it could create inconvenience if it happens automatically.. hmm.. food for thought.