Slow but unbounded grouth of memory cost with time?

[niusen]

Hi TensorKit developers,

thank you for this nice package and I used it a lot for my research. Now I have a problem with the increase of memory cost, probably it is related to TensorKit.jl. Below is brief description of my algorithm: I construct finite PEPS with U(1) symmetric tensor, and then generate Monte Carlo samples and do contractions to obtain the wavefunction amplitude. I expect the memory cost should saturate quickly. However, the memory occupation increases slowly and never stops.

Here is the simplified version of my codes that can reproduce the increase of memory cost:
https://github.com/niusen/PEPS_memory_test
Running "my_test.jl", then one can observe the change of occupied memory either from "top" linux command or from reading the output file. The slow memory increase is not visible within short time, but become evident after half day. For example, after 8 hours, the used memory increase from around 1.5Gb to around 8Gb.

I tried to tune some of the cache size "like TensorKit.treepermutercache" as well as use "GC.gc(true)", but the problem still exists. Is there a way to check the origin of the increase?

Thank you a lot.

[maartenvd]

There is varinfo(all=true, imported=true, recursive=true, sortby=:size, minsize=1024). But julia will also allocate memory and not directly release this back to the os (for example, when peak memory usage is 8gb, it may just keep 8gb around for later). You can get it to free some of it by ccalling malloc_trim.

[lkdvos]

Am I correct in assuming that you checked the sizes of the caches?
If these are not the ones that are "filling up" and causing that memory increase, it's very likely that this is simply what @maartenvd is referring to, and this should not grow further.

Note that these caches can always be safely emptied out, or you could even try further decreasing there maxsize. This problem has also appeared in PEPSKit.jl, we are still trying to figure out some reasonable values for these cache sizes, but it seems highly dependent on the use-case.

[niusen]

@maartenvd @lkdvos Thank you a lot for the answers. Finally I found combination of "malloc_trim" and "--heap-size-hint=6G" solves my problem.