local_allowlist_path, use_compression & deduplication flags are not available in config

[Keidro]

Hey, after configuring adblock I found that I wanted to set the use_compression & deduplication flags in config. Unfortunately, Adblock-lean throws me an error. The same thing happens if I change the default path for local_allowlist_path.

[friendly-bits]

Hi, please post console output for each of these issues, and the config file contents (cat /etc/adblock-lean/config) which causes them. Also the output of:

service adblock-lean version
cat /etc/config/dhcp

[Keidro]

service adblock-lean version
adblock-lean version: 'v0.6.2', update channel: 'release'.

cat /etc/adblock-lean/config

whitelist_mode="0"
blocklist_urls="https://raw.githubusercontent.com/hagezi/dns-blocklists/main/wildcard/pro-onlydomains.txt https://raw.githubusercontent.com/hagezi/dns-blocklists/main/wildcard/tif-onlydomains.txt"
blocklist_ipv4_urls=""
allowlist_urls=""
dnsmasq_blocklist_urls=""
dnsmasq_blocklist_ipv4_urls=""
dnsmasq_allowlist_urls=""
local_allowlist_path="/etc/banip/banip.allowlist"
local_blocklist_path="/etc/adblock-lean/blocklist"
deduplication
use_compression="1"
test_domains="google.com microsoft.com amazon.com"
list_part_failed_action="SKIP"
max_download_retries="3"
min_good_line_count="330000"
min_blocklist_part_line_count="1"
min_blocklist_ipv4_part_line_count="1"
min_allowlist_part_line_count="1"
max_file_part_size_KB="19000"
max_blocklist_file_size_KB="24000"
initial_dnsmasq_restart="0"
boot_start_delay_s="120"
custom_script=""
cron_schedule="0 5 * * *"
DNSMASQ_INSTANCE="cfg01411c"
DNSMASQ_INDEX="0"
DNSMASQ_CONF_D="/tmp/dnsmasq.cfg01411c.d"

cat /etc/config/dhcp

config dnsmasq
	option domainneeded '1'
	option localise_queries '1'
	option rebind_protection '1'
	option rebind_localhost '1'
	option local '/lan/'
	option domain 'lan'
	option expandhosts '1'
	option cachesize '1000'
	option authoritative '1'
	option readethers '1'
	option leasefile '/tmp/dhcp.leases'
	option resolvfile '/tmp/resolv.conf.d/resolv.conf.auto'
	option localservice '1'
	option ednspacket_max '1232'
	list addnmount '/bin/busybox'

config dhcp 'lan'
	option interface 'lan'
	option start '100'
	option limit '150'
	option leasetime '12h'
	option dhcpv4 'server'

config dhcp 'wan'
	option interface 'wan'
	option ignore '1'

config odhcpd 'odhcpd'
	option maindhcp '0'
	option leasefile '/tmp/hosts/odhcpd'
	option leasetrigger '/usr/sbin/odhcpd-update'
	option loglevel '4'

config dhcp 'iot'
	option interface 'iot'
	option start '100'
	option limit '150'
	option leasetime '12h'

config dhcp 'guest'
	option interface 'guest'
	option start '100'
	option limit '150'
	option leasetime '12h'

config dhcp 'home'
	option interface 'home'
	option start '100'
	option limit '150'
	option leasetime '12h'

output I will give in my next comment

[Keidro]

Deduplication error is not relevant, cause I re-created config using generation and see that
deduplication="1" so this part is missing from docs.

deduplication	Whether to perform sorting and deduplication of entries

I have tried several times to reproduce local_allowlist_path error messages, as if out of spite, they would not appear. I will try again in the next few days.

[friendly-bits]

so this part is missing from docs

The README says about deduplication:

Whether to perform sorting and deduplication of entries

Besides, the README says:

Each configuration option is internally documented with comments in /etc/adblock-lean/config.
And that the provided table is a short version.

The config file explains:

Whether to perform sorting and deduplication of entries (usually doesn't cause much slowdown, uses a bit more memory) - enable (1) or disable (0)

The config file also explains the format:

values must be enclosed in double-quotes

So I'm not sure what you would like to see in the docs that is not there already.

As to local_allowlist_path, I tested that option with a custom path and it worked fine. So my guess is that you didn't follow the config guidelines, or made a mistake in the path, but without specific error it's hard to tell what exactly went wrong.

Anyway, the upcoming release of adblock-lean will have the config parser re-implemented.

[lynxthecat]

I'm confused - what's the issue here @Keidro?

[friendly-bits]

I relied on the README you have on your site and built the config based on that.

So to my understanding, you did not notice the line above the table which says:

Each configuration option is internally documented with comments in /etc/adblock-lean/config.

And you did not notice the comment in the config which explains what the value should be.

We did our best to document each option in the config file, but doing this in the README as well would (from our perspective) add a lot of unnecessary clunk to the README which we rather avoid. I suppose we can change the font of the line which points to the explanation in the config file so it's bold and harder to miss.

[Keidro]

Wed Apr  9 17:55:18 2025 user.info adblock-lean: Found local allowlist. Sanitizing.
Wed Apr  9 17:55:18 2025 user.warn adblock-lean: Warning: Rogue element: '192.168.1.42/24' identified originating in allowlist file from: /etc/banip/banip.allowlist.
Wed Apr  9 17:55:18 2025 user.info adblock-lean: Skipping file and continuing.
Wed Apr  9 17:55:18 2025 user.info adblock-lean: Not using any allowlist for blocklist processing.

[friendly-bits]

Rogue element: '192.168.1.42/24' identified originating in allowlist file from: /etc/banip/banip.

adblock-lean normally works with domains, not IP addresses. adblock-lean validates entries included in source lists to make sure that invalid data doesn't get into the final list. Looks like your allowlist has some IP addresses which fail this validation, because they are not domains. Currently adblock-lean does not support ipv4 allowlists because nobody asked for this feature but maybe we can consider implementing that.

[Keidro]

It would be nice, especially for allowlist it will be useful for internal routing / firewall rules and extensions like banIP which translate addresses themselves. If you ever want to implement support for ipv4, start with allowlist :)

[friendly-bits]

Thinking about this a bit more, I'm not sure what the use case for this feature would be, except when ipv4 blocklist is also used in adblock-lean. In general, dnsmasq (and adblock-lean, by extension) doesn't block IP addresses, unless specifically told to. adblock-lean blocks domain name resolution to ip addresses for domains included in the blocklist. So you can communicate with any IP address you like regardless of adblock-lean configuration, except if you are using an ipv4 blocklist in adblock-lean (which is a supported feature, but currently only for downloaded lists, not local ones). If you are using such blocklist in adblock-lean then the specified IP addresses or IP ranges will be blocked, and an IPv4 allowlist would then be useful to unblock them (or perhaps to unblock certain IP's belonging to a blocked IP range).

Generally, while we can do this in adblock-lean, I'm not sure why we should. For example, in your case: you are using BanIP to block certain IP ranges and you are using a BanIP allowlist to allow some of those blocked IPs. Why do you need adblock-lean to do same blocking and unblocking, rather than just using BanIP for managing blocked IP's and adblock-lean for managing blocked domains?

[lynxthecat]

Could we also block a range vis the dnsmasq route?

In any case, it seems like there really is a use case for blocking IPs?

@Keidro why do you want to block certain IPs? Can you explain the use case? We’ve wondered about implementing this functionality in adblock-lean.

[friendly-bits]

@lynxthecat there is a use case of blocking ip's, of course, and it can be useful for adblocking purposes in some cases, however I believe that the correct way to implement that is via firewall rules. I'm not actually sure how dnsmasq can block connections to IP addresses at all, except when resolving a certain domain and then checking whether the resolved IP is included in the blocklist. Maybe I'm missing something.

[lynxthecat]

It would also be possible to have adblock-lean inject nftables code using an .nft file - see e.g. here. If that were to be faster than using dnsmasq, it could be an idea to translate any remote IP blocklists to such an .nft file to block those IP lists. Or even go belt and braces and have both dnsmasq and nftables block incoming/outgoing connections or lookups associated with IP address threats.