Environment Setup

These instructions document the process to set up a development environment for running a dark decoy station. The same steps can be used to set up a new production environment as well.

This setup was done on a local router so the ip addresses used are from rfc 1918.

See the image in this repo paper/figures/dev_env.png for a basic layout of this guide

ON ROUTER (openwrt):

Install some non standard openwrt packages that we are going to need

opkg update
opkg install iptables-mod-tee
opkg install ip-full

    # for debugging not absolutely necessary
opkg install tcpdump 

Add new interface for subnet 10.0.1.1/24 with the router at 10.0.1.1

ip addr add 10.0.1.1/24 dev br-lan

Add static ARP route (the server has 2 ethernet connections) connecting address and MAC. This used ... neigh replace... as the address on the router had been co-opted the by another interface on the server (acting as a proxy). If the ARP route hasn't been incorrectly assigned to another interface you would use ... neigh add ....

ip neigh replace 10.0.1.2 lladdr 68:05:ca:17:c4:96 nud permanent dev br-lan
# ip neigh add 10.0.1.2 lladdr 68:05:ca:17:c4:96 nud permanent dev br-lan

NOTE: IF YOU MESS UP THE MAC ADDRESS IN THE STATIC ARP RECORD TRAFFIC WILL GO TO WHATEVER MAC YOU PUT IN. STATIC ADDRESSING ON DEVICES DOES NOT MATTER.

Next port mirror traffic to the tap (server) by adding to file /etc/firewall.user

iptables -t mangle -A PREROUTING 1 -d 192.168.1.0/24 -j TEE --gw 10.0.1.2
iptables -t mangle -A POSTROUTING 1 -s 192.168.1.0/24 -j TEE --gw 10.0.1.2 

Additionally, newer versions of OpenWRT contain a rule that drops packets with WAN destinations that don't have an established conntrack entry from the WAN interface; this is problematic if the client is on the local network. This may manifest itself in the station receiving the initial SYN packet from the client, but not the subsequent ACK. This can be disabled by inserting the following into /etc/config/firewall under the zone beginning with option name 'wan':

option masq_allow_invalid '1'

The full WAN zone may look something like this:

config zone
        option name 'wan'
        list network 'wan'
        list network 'wan6'
        option output 'ACCEPT'
        option masq '1'
        option mtu_fix '1'
        option input 'REJECT'
        option forward 'REJECT'
        option masq_allow_invalid '1'

This option can also be enabled in the WebUI on OpenWRT 19.07+ via Network > Firewall > Zones > WAN Zone > Edit > Conntrack Settings > "Allow "invalid" traffic".

Reload the firewall config to apply the routing rules:

/etc/init.d/firewall reload
    # or
# service firewall restart

ON SERVER:

You should now see traffic on the interface that was designated to be the TAP. test with

sudo tcpdump -i enp2s0  host 52.44.73.6

Once Per machine

Add static address for interfaceso that ubuntu never sends DHCP requests for tap interface by appending the following to /etc/network/interfaces

auto enp2s0
iface enp2s0 inet static
  address 10.0.1.2
  netmask 255.255.255.0

Restart the networking service

sudo service networking reload

Note: if adding routing rules gives and error about non-existent table you need to add the custom routing table using . Add custom ip routing table

echo "200 custom" >> /etc/iproute2/rt_tables

Once Per reboot

Set the global rp_filter to allow packets with falsified ip addresses to pass through

sysctl -w net.ipv4.conf.all.rp_filter=0

Set up the internal tunnels to route packets from rust process_packet to the golang application. Do this for each tunnel being created (one per core being used). Note that the IPtables rules need to point to the interface NOT BEING USED AS A TAP.

    # create tun0 interface
ip tuntap del mode tun tun0
ip tuntap add mode tun tun0
ip rule add iif tun0 lookup custom
ip route add local 0.0.0.0/0 dev tun0 table custom

    # rp_filter 
sysctl -w net.ipv4.conf.tun0.rp_filter=0

    #iptables DNAT routing for packets on the tunX interface
iptables -t nat -I PREROUTING 1 -p tcp -i tun0 -j DNAT --to 192.168.1.101:41245
iptables -I INPUT 1 -i tun0 -j ACCEPT

Troubleshooting:

List iptables rules for inspection

#list rules for specific table
iptables --table nat --list

#shortens to
iptables -t nat -L

# [-v] verbose output (all fields) with [-n] numeric usage information
iptables -t nat -v -n -L

# clear all counters for iptables rules
iptables -Z

# clear counters for specific table
iptables -t nat -Z

# Clear counters for specific chain within table 
iptables -t nat -Z PREROUTING 

# Clear counters for specific rule
iptables -t nat -Z PREROUTING 1

List rules added to ip

ip rule list

list routing information relevant to tuns

ip route show table all | grep tun

Delete all tun0 rules in the ip rule list based on interface (iif) name. If you run the singular version it only deletes one rule.

while sudo ip rule delete iif tun0 2>/dev/null; do true; done