Memory leak

[amigthea]

Describe the bug
The docker container has a memory leak, I'm on the latest image version.

After it has reached 17gb of host's ram usage I limited the container available memory to 512mb and since then the container spit out this kernel error three/four times a day:

kernel: Memory cgroup out of memory: Killed process 3305847 (python) total-vm:1743496kB, anon-rss:364896kB, file-rss:9984kB, shmem-rss:0kB, UID:0 pgtables:1012kB oom_score_adj:0

because it now kills the memory leaking process when the memory threshold is reached, instead of occupying more ram

Version
0.45.3

Screenshots
htop inside the container

Additional context
I have 12 active watches

any clue?

[dgtlmoon]

I also see that you have quite a lot of changedetection processes running....

or is it just one?

[Constantin1489]

By default, htop lists each thread of a process separately, while ps doesn't.
https://unix.stackexchange.com/a/10403/546660

[amigthea]

this screenshots are taken after a container restart

htop

ps -aux

[amigthea]

I took this after the first process killed for hitting the forced memory threshold

I can send whatever log do you need for troubleshooting

[adripo]

I am also experiencing the same issue. There is a discussion in this pr with more details, but no solution
#1831 (comment)

[adripo]

For now the workaround that I am using is to create a separate service that automatically restarts changedetectio each night.

https://github.com/adripo/changedetection.io/blob/patch-1/docker-compose.yml#L93
Rows 93-112

[amigthea]

thank you for joining the discussion @adripo

I would prefer to not add a service just for that, for now my workaround is bring up the memory limit to 1gb and change the check to once a day, no problem so far considering that I have 12 watches, each with visual filter selector . My previous settings was once every 15 minutes (I would like to go back to that), but even once every hour gave wasn't enough for 1gb of memory.

[adripo]

I also limited the memory in the first place, but then I discovered this issue with docker where it could kill other processes on the host when memory limit of the container is reached:

docker/for-linux#1001

[dgtlmoon]

I will say this

1. I will not add something that restarts peoples containers without them knowing about it, and certainly not before Test - shutdown test for #693 and #1850 #1853 is solved
2. In my digging around, it seems to be related to how LXML is handling the document, I do not believe its directly a problem in changedetection.io
3. If you want to be more productive in solving this, try to isolate a changedetection.io install where you can have just 1 watch in the setup that is able to cause the memory growth - without this, I cant help you, to handle this professionally and accurately you need to provide more exact information and a way to exactly reproduce the case with the absolute least amount of variables (just 1 watch,no notifications, etc etc - use the 'share' feature, thats what it is for)

[dgtlmoon]

Restarting the instance yourself by cron (just restart the container) is otherwise an excellent solution for now

[amigthea]

Restarting the instance yourself by cron (just restart the container) is otherwise an excellent solution for now

I would not prefer to restart the container every 15 minutes, my logs collection would be full of useless spam and I don't want to create overhead working on an exception to that; I think I'll leave the daily check and the memory limit for now. Thank you for your time and keep it up, it's a great project!

[dgtlmoon]

@amigthea so please see statement number 3 then :)

[amigthea]

If the memory problem can be solved by changing from one check every 15 minutes on 12 watches (>17Gb occupied mem) to check once every day on the same watches (stable on 600-700Mb), I would assume that the bench test you suggested would return also no problem

[dgtlmoon]

@amigthea so please see statement number 3 then :)

[dgtlmoon]

Also if anyone has input on this comment #1840 (comment) and how to move to that library