Pruning of uBO filter*.txt lists

[ryanbr]

Thoughts on limiting the uBO filters*.txt lists. (to avoid bloat). I suspect there is many sites where filters aren't needed any more or need updating.

Currently:

• Currently, every year a new list is created with filters-[year].txt
• New rules are added to the new list, and updating on the other lists

Proposed:

• Every year a new list is created with filters-[year].txt (no change).
• At the start/end of the year (or whenever), Review the oldest list at the time (filters.txt, filters-2020.txt etc..), and go through each of the filters. (git blame/git log)
• Any adblock rules still confirmed current is moved into the latest list filters-[year].txt.
• The oldest filters list could be null'd eventually.

Why:

• There is rules that are many years old, which may or may not apply any more.
• Will avoid uBO list bloat over time as much as possible, and check if the filters need to be removed, changed or what ever.
• Keeps filter lists more reviewed more frequently, well at least yearly.

[ryanbr]

I know that this isn't the easiest of tasks.

But reviewing the oldest rules. and moving them into the updated list should be a good idea. Keeps the list more optimised, and more managable.

Something similar is done in Easylist, and its still manual (where's AI when you need it). Using git blame and git log. viewing the oldest additions and reviewing the commits. View the site, see if still applies and remove or change the filters.

[Yuki2718]

d5c3b06
Chose the top than the bottom so not to interfere with our practice of adding new rules. Next year we can move the whole contents to 2025.txt with new lines of Rules from filters-2024.txt added. Also created Movable to EL section. Please note EL addresses anti-adb only for adult sites, shortners/file hostings, and streaming/torrent/comic sites (see https://github.com/easylist/easylist). No need to move rules needing constant adjustment though, as members other than me don't have write access to EL.

[gorhill]

I think only filters from a set number of years should be moved to current year, like 2024 - 6 years ago = <=2018 , and once moved, they become "valid as of 2024", they will be checked again in 2030 since 2024 will become 6 years old. The idea is to garbage collect only old filters, not the one created in recent time.

[Yuki2718]

I don't see problem for moving rules like

! https://forums.lanik.us/viewtopic.php?f=62&t=42076
hotbabes.tv##+js(aopr, loadTool)
candid.tube,filesamba.*,hotbabes.tv,purelyceleb.com##+js(aopr, _wm)
cambabe.*,camgirlbay.net##+js(acs, onload)
cambabe.*##+js(acs, crakPopInParams)
cambabe.*##.topad
cambabe.*##.place
purelyceleb.com##.embed-container
##.exo-horizontal

in filters.txt to 2024.txt.

[gorhill]

Sure, all of filters.txt should be assumed to be very old. Just pointing out you created sections for 2020-2023, I don't think we should mind such "recent" filters, and also:

Next year we can move the whole contents to 2025.txt

The whole content of what? I am getting confused here. The way I see it to keep the workload as small and manageable is to just mind old filters (after we define "old"), and once an old filter has been validated as still being needed, we move it to current year list, then we do not validate the content of that list until it also become old in a few years.

[Yuki2718]

All right, got it. I was thinking to keep only filters.txt and the latest year's list. But maybe 6 is too long, I guess 3 or 5 years will be suffice.

[Yuki2718]

Initial commit 748b046
This actualy can be moved to EL once _specific_uBO.txt is added, but I haven't heard from other members about this.