[Question] Verify if an IP exist in the nftables whitelist/balcklist

[n0obHere]

@friendly-bits First of all, thank you for this great software. I'm just getting started with it, and seems like its going to make my life much more easier around managing geo-ip whitelist/blacklist on my linux VM's.

I'm still reading through the docs and was curious to know if after installation, fetching IP lists from the chosen provider and maybe adding some IPs manually, if there's a way to check if a particular IP (or subnet maybe) now exists in the nftables or not? I'm aware of this script check-ip-in-source.sh, and if I'm understanding its use case correctly, I think this is to be used pre-installation and it only checks if the upstream provider has that IP in its IP lists or not, and not necessarily checks if the local nftables has that has IP in its whitelist/blacklist or not.

[friendly-bits]

Hi, geoip-shell currently does not implement this functionality, however this should be fairly easy to achieve either via the nft (for nftables) or the ipset (for iptables) CLI tools, by querying the loaded ipsets/nft sets. Alternatively (provided that geoip-shell backup functionality is enabled), you could run a decompression utility on the (by default compressed) iplist backups, then pipe the output into the grepcidr utility which would then match specified IP's or IP ranges against the backup IP lists. (Note that if using geoip-shell on OpenWrt, the backup functionality is disabled by default to prevent wearing out the onboard flash)

The check-ip-in-source script was originally intended for checks before installation but it works fine after installation as well. And yes, it downloads the IP lists from the provider and checks against the downloaded lists.

[friendly-bits]

Also note that if you are running geoip-shell inside LXC containers, those containers either need to be privileged (which may pose a security risk), or you may need to use the iptables backend. Otherwise loading large IP lists may run into errors because of a kernel bug/limitation (discussed in #24).

[n0obHere]

@friendly-bits For nftables, nft itself doesn’t really have a native way to check whether a single IP (e.g., 192.168.2.55) matches a CIDR range (e.g., 192.168.2.0/24) that’s already in the ruleset. Of course, it’s possible to pipe the output of nft list ruleset into some utility that can parse/expand all CIDRs and check the IP against them all - and that’s exactly what I was hoping that maybe a light-weight utility can be built into geoip-shell for this.

[friendly-bits]

My bad - indeed, turns out that the nft utility doesn't support this sort of lookup. So grepcidr is the way to go, IMO. In general, geoip-shell is entirely written in shell language, and implementing high-performance processing of large volume of data natively in that language is simply impossible. For that reason, geoip-shell employs external utilities for data processing, including awk, sed, grep and others. All those are standard utilities included with almost every linux distribution. Unfortunately, no standard utility can process a list of IP ranges and determine whether given IP address is included in one of them. The best utility for this job I'm aware of is grepcidr. I will consider implementing support for this sort of lookup built into geoip-shell (which would still employ grepcidr for the actual lookup).

[n0obHere]

Thanks @friendly-bits. Just got something together - quick and dirty for my testing.

• Using sipcalc (ipcalc can be used as well) to calculate the subnet and piping the output via grep and awk to catch the start and end IP of each CIDR range.
• Then converting all three IPs into integers to compare later (idea from a colleague)

Never worked with grepcidr before but I'll also take a look into it if it better simplifies the logic here

#!/bin/bash

# Usage: ./check_ip_in_nftables.sh -i <IP>
# Example: ./check_ip_in_nftables.sh -i 123.33.44.55

# Init variables
IP=""
FOUND=0

# Parse arguments
while [[ "$#" -gt 0 ]]; do
    case "$1" in
        -i|--ip) 
            IP="$2"
            shift 2
            ;;
        *) 
            echo "Unknown option: $1"
            echo "Usage: $0 -i <IP>"
            exit 1
            ;;
    esac
done

# Validate IP input
if [[ -z "$IP" ]]; then
    echo "Error: IP address (-i/--ip) is required."
    exit 1
fi

# Convert IP to integer for comparison
ip_to_int() {
    local ip="$1"
    IFS='.' read -r a b c d <<< "$ip"
    echo "$((a * 256 ** 3 + b * 256 ** 2 + c * 256 + d))"
}

# Check if IP is in a CIDR range
check_ip_in_cidr() {
    local cidr="$1"
    local ip="$2"
    local range_start range_end ip_int start_int end_int

    # Extract start and end IPs of the CIDR range
    range_start=$(sipcalc "$cidr" | grep -E "Network range\s*-\s*([0-9.]+)\s*-\s*([0-9.]+)" | awk '{print $4}')
    range_end=$(sipcalc "$cidr" | grep -E "Network range\s*-\s*([0-9.]+)\s*-\s*([0-9.]+)" | awk '{print $6}')

    # Convert to integers
    ip_int=$(ip_to_int "$ip")
    start_int=$(ip_to_int "$range_start")
    end_int=$(ip_to_int "$range_end")

    # Check if IP is within the range
    if (( ip_int >= start_int && ip_int <= end_int )); then
        echo "$ip is in $cidr (range: $range_start - $range_end)"
        return 0  # Success
    else
        echo "$ip is NOT in $cidr"
        return 1  # Failure
    fi
}

# Check all ipv4 CIDR ranges in nftables
echo "Checking if $IP is in any nftables ipv4 CIDR range..."
while read -r cidr; do
    if check_ip_in_cidr "$cidr" "$IP"; then
        FOUND=1
        break  # Currently it exits on first match (this break can be removed to check all other CIDRs that may match the same IP)
    fi
done < <(sudo nft list ruleset | grep -Eo '[0-9]+\.[0-9]+\.[0-9]+\.[0-9]+/[0-9]+' | sort -u) # currently limited to ipv4 address ranges

if [[ "$FOUND" -eq 0 ]]; then
    echo "$IP is NOT in any nftables CIDR range"
    exit 1
fi

[friendly-bits]

This looks like a good and working solution for checking in a relatively small number of IP ranges. However for checking in a few tens of thousands of IP ranges, this will be very slow. P.s. if you want a somewhat faster solution, you could replace some bits of code with POSIX-compliant code and run it in dash, rather than in bash because bash is about 3 times slower. Also try to minimize the number of subshells (including subshells created by pipes), especially in a loop. Creating all those subshells will be a major contributor to the slowness of this script. Another project I created which does certain bits of IP calculation which you could reuse:

https://github.com/friendly-bits/shell-ip-tools

But generally, rather than iterating over the IP ranges one-by-one, it is much better and much faster to use grepcidr. For a working example, you could take a look at the implementation in check-ip-in-source

[n0obHere]

I will take a look into these both. Thank you.

[friendly-bits]

Hi, geoip-shell v0.7.4 implements the lookup action which can now be used to check whether given IP address or addresses are included in any of the IP sets loaded by geoip-shell. Make sure to install grepcidr in order to use this action. For usage, please refer to DETAILS.md or run the command geoip-shell -h.

[n0obHere]

@friendly-bits Just tried to install the update, but it failed with below error. Looks like you forgot to add the lookup script 😃

Checking files... install: Error: missing files: '/adminer/geoip-shell/lib/geoip-shell-lib-lookup.sh'.

[friendly-bits]

Indeed. Sorry about that. I re-released v0.7.4 with lib-lookup included.

[n0obHere]

Thank you @friendly-bits for adding this feature, its working well now.