Change detected, but empty diff on ctrl+c

[norbertogriffiths]

I'm running changedetection.io on my local Debian desktop. I turn it on once a day by executing the command in a terminal, go through all the diffs and then turn it off again by pressing ctrl+c. I'm monitoring about 300 websites and usually around 20 change per day. "Ignore whitespace" is turned on.

On random days, maybe once every 2 weeks, unexpectedly, over 100 websites will be marked as changed. However, most of them will have no highlighted lines in the diff view (i.e. no changes). This only happens in clusters. Either there are at least a hundred websites that supposedly changed, but show no changes or none. It never happens with only a few websites. My expectation was that only websites that have changed lines will be marked as changed.

Is this an expected behavior that I don't understand or a bug?
Is maybe something going wrong, because I turn the system off every day?

[dgtlmoon]

It will trigger a change if

• the rules change (like you edit your global filters)
• the element moves (like the element cant be found anymore)

I was thinking to add some better logging, so it will log what/how it decided a change was found

does that help? did you do something like that?

are you able to repeat it with just 1 URL and share the "shared link" for it?

[norbertogriffiths]

It happened on several occasions without editing rules, so that can't be it.
If an element moves that should be visible in the diff, right? Or do you mean something other than text by 'element'?

I haven't found a way to reproduce this reliably yet, it just happens to seemingly random URLs at random times, so unfortunately I don't have any example I can share.

[dgtlmoon]

oh yeah, also ctrl+c might be related to this #693

because it might not be saving the database at that time

[dgtlmoon]

sounds like #693 is the fix you're after, funny thing is that i started looking at this just before your discussion here

[dgtlmoon]

as for ETA.. unsure, maybe a month or so

[norbertogriffiths]

@dgtlmoon The problem still appears, even if I wait for the JSON file to be saved first, so it appears that's not the issue. I managed to prepare an example now: empty_diff.zip

There are two folders: test_before and test_after. If I open test_before, press recheck on the "nena" watch and reload the page, nena will be highlighted as having changed. test_after is from that stage, at which nena is highlighted. But then if I click on "diff" for nena, there is actually no difference, so from my understanding the line shouldn't be highlighted.

[dgtlmoon]

Amazing debug info! thanks!!!

So, it maybe seems that when you are creating that empty_diff.zip file, are you editing any filters or ignore rules?

the JSON file says the previous md5 was af53890c6aaf472cf6f6df20b10fbf76 but when I look at whats on the disk..

dgtlmoon@dgtlmoon-HP-ProBook-440-G7:/tmp/data/1f0b700c-d921-4482-8bd6-f8c1d2715af0$ tail -n 1 history.txt 
1655829870,/home/user/Data/changedetection/data//1f0b700c-d921-4482-8bd6-f8c1d2715af0/bb903fec-83da-4cc5-9b61-8a115a9d209a.stripped.txt
dgtlmoon@dgtlmoon-HP-ProBook-440-G7:/tmp/data/1f0b700c-d921-4482-8bd6-f8c1d2715af0$ md5sum bb903fec-83da-4cc5-9b61-8a115a9d209a.stripped.txt 
076a2fb7f38ae3f7584f25f95547df7b  bb903fec-83da-4cc5-9b61-8a115a9d209a.stripped.txt
dgtlmoon@dgtlmoon-HP-ProBook-440-G7:/tmp/data/1f0b700c-d921-4482-8bd6-f8c1d2715af0$ 

last md5 without ignoring whitespace is 076a2fb7f38ae3f7584f25f95547df7b
last md5 according to the code which strips whitespace is 'f1a2e24d8f5c195e959658523b250ed1'

neither are af53890c6aaf472cf6f6df20b10fbf76...

and if i hit [check] again, i dont get any more false readings... it otherwise seems to work fine

do you have any other filters perhaps, that you removed, when creating that zip file?

[norbertogriffiths]

Sorry, you're right, I deleted some filters to make it easier, because I had tons of them. Didn't consider that that will prevent you from reconstructing the checksum. I don't have the data anymore now. Next time the issue happens I'll leave the filters in and send you the backup again.