Anonymizing Packet Capture tool for Conjure (#168)

* captool working for pcap from file source

* Remove lifetimes from PacketHandler

* Split interfaces / read pcap dir into functions

* Capture from interfaces

* runs with interface

* limits seem to be working. need a bit more live testing

* termation handler fix and write gzipped

* disable error logging for common errors

* bug fixes and error handling instead of unwrap within threads. panics == bad

* seemss to work for Eth and IP packets, adding better test next

* unit tests working for Eth and IPvX link types

* support raw linktype for tun interfaces

* handling funky link layer types for tun interfaces

* anonymize ipv6 flow label and add cli opt for v4 / v6 only

* flow tracking WIP

* WIP fixing issues and adding flow tracking

* flow tracking passing tests, integration next

* cautously optimistic - integration compiling, tests succeeding

* fixed timestamp issue, flow tracking integration complete, tests passing

* help messages for cli args

* change cli arg name

* default log level in release to warn and err on empty targets

---------

Co-authored-by: Eric Wustrow <[email protected]>

Author: jmwample

util/captool/Cargo.toml

@@ -0,0 +1,28 @@
+[package]
+name = "captool"
+version = "0.1.0"
+edition = "2021"
+
+# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
+
+
+[dependencies]
+pcap = "1.0.0"
+maxminddb = "0.23.0"
+ipnet = "2.7.2"
+threadpool = "1.8.1"
+pnet = "0.33.0"
+hmac = "0.12.1"
+sha2 = "*"
+rand = "0.8.5"
+pcap-file = "2.0.0"
+clap = { version = "4.2.2", features = ["derive"] }
+signal-hook = "0.3.15"
+flate2 = "1.0.25"
+log = "0.4.17"
+simple_logger = "4.1.0"
+
+[dev-dependencies]
+tempfile = "3.5.0"
+byteorder_slice = "3.0.0"
+hex = "0.4.3"

util/captool/README.md

@@ -0,0 +1,27 @@
+# Packet Capture Tool
+
+This packet capture tool allows us to rapidly capture and anonymize packets
+for flow analysis in relation to conjure stations.
+
+This requires that the tool:
+
+- read from multiple sources in parallel
+- has access to GeoIP2 mmdbs to look up Country and ASN info
+- deteministically anonymize client IPs and ports.
+- write pcapng format with supplemental ASN/CC/subnet info in optional comments
+
+Address anonymization is done by first determining what the addresses parent
+subnet allocation is. We then replace the client address with a deterministically
+chosen random addres by filling the host mask with bytes from an HMAC of the
+flow tuple. we do the same for client ports. This allows us to ensure that the
+separate packets within the same flow will still be linked, without
+client-identifying information.
+
+The Key for the HMAC is generated at runtime from the system CSPRNG and never
+written to disk, only being stored in memory during the capture.
+
+---
+
+## Capture Parameters
+
+Flows are captured from first SYN for TCP and from first seen for UDP.

util/captool/rust-toolchain.toml

@@ -0,0 +1,2 @@
+[toolchain]
+channel = "nightly"