High memory usage despite persisting data.

[WinnerWind]

Hi.
My log file is about 1.27GiB of late.

Because of this, GoAccess has been eating up my RAM (about 1.4GiB on a small 8GiB memory system, which is highly undesirable). Because of this, I enabled on disk persistence hoping that the memory usage would decrease, but this has not been the case. After enabling disk persistence, memory usage has only increased by about 200MiB

My command (running via systemd)

goaccess /home/soham/access_combined.log -o /home/soham/GoAccess/access/index.html --log-format COMBINED --geoip-database=/usr/share/GeoIP/GeoLite2-City.mmdb --geoip-database=/usr/share/GeoIP/GeoLite2-Country.mmdb --geoip-database=/usr/share/GeoIP/GeoLite2-ASN.mmdb --real-time-html --ws-url={censored} --port=7890 --db-path=/home/soham/GoAccess/db-persist --persist --restore

[allinurl]

Great question. The persistence is only for saving state, when restored, everything is loaded into RAM, so GoAccess doesn't offload data to disk while running. You could try -q if you have a lot of query strings and don't need them in the requests, that should help. Let me know if it makes a difference.

For the record, I've been working on #117, which includes a major on-disk database designed, built in-house. It may make it upstream at some point, we'll see how things evolve... Stay tuned for updates: https://x.com/goaccess