update the docs and CHANGELOG about new features

Author: AlyaGomaa

CHANGELOG.md

@@ -1,3 +1,20 @@
+1.1.7 (Feb 28th, 2025)
+- Add global P2P support. Thanks to @d-strat
+- Add new "GRE tunnel scan" detections.
+- Add the option to enable/disable local and online whitelists from slips.yaml.
+- Fix false positive "Connection to a private IP outside of local network" detection. Slips now doesn't alert on DNS servers outside of local network.
+- Fix false positive "Connection to a private IP" detection when the connection is DHCP.
+- Fix false positive "Device changing IP" detection alerting about special IPs.
+- Fix false positive "Invalid DNS answer" detection alerting about .arpa domains.
+- Fix false positive "non-HTTP established connection on port 80".
+- Fix false positive "non-SSL established connection on port 443".
+- Improve "Connection to unknown port" detections. Now the threat level depends on the flow state.
+- Improve "DNS without connection" evidence. Slips now only detects when the query type is A or AAAA.
+- Improve the description of malicious flow by MLflowdetection module.
+- Improve the detections of the MLflowdetection module.
+- Improve the existing "GRE tunnel" detections.
+- Improve whitelists: Slips is now whitelisting CNAME, SNI, related queries, and DNS resolutions of attackers and victims.
+
 1.1.6 (Jan 31st, 2025)
 * 3x speedup of the profiler process responsible for analyzing the given flows.
 * Fix false positive "connection without DNS" detection.

docs/detection_modules.md

@@ -652,14 +652,7 @@ Available detection are:
 - Unencrypted HTTP traffic
 - Non-HTTP connections on port 80.
 
-Slips detects established connections on port 80 that are not using SSL
-using zeek's conn.log flows
 
-if slips finds a flow using destination port 80 and the 'service' field
-in conn.log isn't set to 'http', if means zeek didnt recognize that flow as http.
-Slips makes sure no matching flows were detected as HTTP by zeek
-within 5 mins before or after the given flow. if not, slips sets an evidence saying
-"non http established conn on port 80"
 
 
 ### Multiple empty connections
@@ -729,6 +722,19 @@ When found, slips alerts pastebin download with threat level low because not all
 When slip sees an HTTP unencrypted traffic in zeek's http.log it generates
 an evidence with threat_level low
 
+
+### Non-HTTP connections on port 80
+
+Slips detects established connections on port 80 that are not using HTTP
+using zeek's conn.log flows
+
+if slips finds a flow using destination port 80 and the 'service' field
+in conn.log isn't set to 'http', if means zeek didnt recognize that flow as http.
+Slips makes sure no matching flows were detected as HTTP by zeek
+within 5 mins before or after the given flow. if not, slips sets an evidence saying
+"non http established conn on port 80"
+
+
 ## Leak Detector Module
 
 This module work only when slips is given a PCAP

docs/features.md

@@ -40,6 +40,7 @@ The detection techniques are:
 - GRE tunnels
 - GRE tunnel scan
 - SSH version changing
+- Invalid DNS resolutions
 
 The details of each detection follows.
 
@@ -88,29 +89,35 @@ Then for every source address in conn.log, slips checks if the MAC of it was use
 
 If so, it alerts "Device changing IPs".
 
+
 ## GRE tunnels
 
-Slips uses zeek tunnel.log to alert on GRE tunnels when found. Whenever one is found, slips sets an evidence
+Slips uses zeek tunnel.log to alert on GRE tunnels when found. Whenever one
+any action other than "Tunnel::DISCOVER" is found, slips sets an evidence
 with threat level low
 
 ## GRE tunnel scans
 
-Slips uses zeek tunnel.log to alert on GRE tunnels with DISCOVER actions when found.
+Slips uses zeek tunnel.log to alert on GRE tunnels scan. Slips considers any log with "Tunnel::DISCOVER" action a GRE scan.
+
 The threat level of this evidence is low.
 
 
+
 ### SMTP login bruteforce
 
 Slips alerts when 3+ invalid SMTP login attempts occurs within 10s
 
 
 ### Connection to private IPs
 
+
 Slips detects when a private IP is connected to another private IP with threat level info.
 
-But it skips this alert when it's a DNS connection on port
+But it skips this alert when it's a DNS or a DHCP connection on port
 53, 67 or 68 UDP to the gateway IP.
 
+
 ### Connection to private IPs outside the current local network
 
 Slips detects the currently used local network and alerts if it find a
@@ -144,19 +151,31 @@ When there's a weird HTTP method, slips detects it as well.
 
 ### Non-SSL connections on port 443
 
-Slips detects established connections on port 443 that are not using HTTP
+Slips detects established connections on port 443 that are not using SSL
 using zeek's conn.log flows
 
 if slips finds a flow using destination port 443 and the 'service' field
-in conn.log isn't set to 'ssl', it alerts
+in conn.log isn't set to 'ssl', it alerts.
+
+Sometimes zeek detects a connection from a source to a destination IP on port 443 as SSL, and another connection within
+5 minutes later as non-SSL. Slips detects that and does not set an evidence for any of them.
+
+Here's how it works
+
+
+<img src="https://raw.githubusercontent.com/stratosphereips/StratosphereLinuxIPS/develop/docs/images/how_non_ssl_evidence_works.png.png" >
 
 ## Non-HTTP connections on port 80.
 
 Slips detects established connections on port 80 that are not using SSL
-using zeek's conn.log flows
+using zeek's conn.log flows.
 
 if slips finds a flow using destination port 80 and the 'service' field
-in conn.log isn't set to 'http', it alerts
+in conn.log isn't set to 'http', it sets and evidence.
+
+If a flow without http as a service is found, slips first checks past and future flows from the
+same src ip + dst ip + port to see if there's a flow with http as a service, if there is, slips ignores the alert.
+This is done to avoid FPs coming from zeek.
 
 
 ### Connections without DNS resolution
@@ -197,17 +216,34 @@ Slips detects successful SSH connections using 2 ways
 2. if all bytes sent in a SSH connection is more than 4290 bytes
 
 ### DNS resolutions without a connection
-This will detect DNS resolutions for which no further connection was done. A resolution without a usage is slightly suspicious.
+
+This will detect DNS resolutions for which no further connection was done.
+A resolution without a usage is slightly suspicious.
 
 The domains that are excepted are:
 
 - All reverse DNS resolutions using the in-addr.arpa domain.
 - All .local domains
 - The wild card domain *
 - Subdomains of cymru.com, since it is used by the ipwhois library to get the ASN of an IP and its range.
-- Ignore WPAD domain from Windows
-- Ignore domains without a TLD such as the Chrome test domains.
+- WPAD domain from Windows
+- domains without a TLD such as the Chrome test domains.
+- DNS resolutions of any type other than AAAA and A
+
+Slips doesn't detect 'DNS resolutions without a connection' when running
+on an interface except for when it's done by this instance's own IP and only after 30 minutes has passed to
+avoid false positives (assuming the connection did happen and yet to be logged).
+
+
+When running on interface and files. For each DNS flow found, slips waits 30 mins zeek time
+for the connection to be found before setting an evidence.
 
+This is done by comparing each ts of every new dns flow to the pending detection, once 30 mins difference between the 2
+flows is detected, slips sets the evidence.
+
+To avoid accumulating so many pending DNS flows for 30 mins, slips checks if the connection of the pending DNS flows
+arrived every 10 and 20 mins too, if not found, slips waits extra 10 mins (so that would be 30 mins total) and sets the
+evidence.
 
 ### Connection to unknown ports
 
@@ -232,6 +268,7 @@ Otherwise, Slips triggers and "unknown port" evidence.
 
 For example, even though 5223/TCP isn't a well known port, Apple uses it in Apple Push Notification Service (APNS).
 
+The threat level of this evidence depends on the state of hte flow. established connections have higher threat levels.
 
 ### Data exfiltration
 
@@ -365,6 +402,13 @@ Then whenever slips sees the same IP using another SSH version, it compares the
 
 If they are different, slips generates an alert
 
+## Invalid DNS resolutions
+
+Some DNS resolvers answer the DNS query to adservers with 0.0.0.0 or 127.0.0.1 as the ip of the domain to block the domain.
+Slips detects this and sets an informational evidence.
+
+This detection doesn't apply to queries ending with ".arpa" or ".local"
+
 
 ## Detection modules
 

docs/flowalerts.md

@@ -7,7 +7,7 @@ The detection techniques are:
 - Long connections
 - Successful SSH connections
 - Connections without DNS resolution
-- DNS resolutions to IPs that were never used
+- DNS resolutions without a connection
 - Connections to unknown ports
 - Data exfiltration
 - Malicious JA3 hashes
@@ -32,6 +32,7 @@ The detection techniques are:
 - High entropy DNS TXT answers
 - Devices changing IPs
 - GRE tunnels
+- GRE tunnel scan
 - Invalid DNS answers
 The details of each detection follows.
 
@@ -86,6 +87,7 @@ Slips detects successful SSH connections using 2 ways
 2. If all bytes sent in a SSH connection is more than 4290 bytes
 
 ## DNS resolutions without a connection
+
 This will detect DNS resolutions for which no further connection was done.
 A resolution without a usage is slightly suspicious.
 
@@ -95,8 +97,9 @@ The domains that are excepted are:
 - All .local domains
 - The wild card domain *
 - Subdomains of cymru.com, since it is used by the ipwhois library to get the ASN of an IP and its range.
-- Ignore WPAD domain from Windows
-- Ignore domains without a TLD such as the Chrome test domains.
+- WPAD domain from Windows
+- domains without a TLD such as the Chrome test domains.
+- DNS resolutions of any type other than AAAA and A
 
 Slips doesn't detect 'DNS resolutions without a connection' when running
 on an interface except for when it's done by this instance's own IP and only after 30 minutes has passed to
@@ -113,21 +116,33 @@ To avoid accumulating so many pending DNS flows for 30 mins, slips checks if the
 arrived every 10 and 20 mins too, if not found, slips waits extra 10 mins (so that would be 30 mins total) and sets the
 evidence.
 
+
+
 ## Connection to unknown ports
 
-Slips has a list of known ports located in ```slips_files/ports_info/ports_used_by_specific_orgs.csv```
+Slips has a list of known ports located in ```slips_files/ports_info/services.csv```
 
-It also has a list of ports that belong to a specific organization in ```slips_files/ports_info/ports_used_by_specific_orgs.csv```
+and a list of ports that belong to a specific organization in ```slips_files/ports_info/ports_used_by_specific_orgs.csv```
+
+These are the cases where Slips marks the port as known and doesn't trigger an alert
+
+1. If the port is in the list of well known ports in `services.csv`.
+2. If Slips has that port's info in `ports_used_by_specific_orgs.csv` and the source and destination addresses belong to that organization.
 
-For example, even though 5223/TCP isn't a well known port, Apple uses it in Apple Push Notification Service (APNS).
 
-any port that isn't in the above 2 files is considered unknown to Slips.
+Slips considers an IP belongs to an org if:
 
-Slips will detect established connections only to unknown ports.
+1. Both `saddr` and `daddr` have the organization's name in their MAC vendor (e.g. Apple.)
+2. Both `saddr` and `daddr` belong to the range specified in the`ports_used_by_specific_orgs.csv` for that organization.
+3. If the SNI, hostname, rDNS, ASN of this IP belong to this organization.
+4. If the IP is hardcoded in any of the organizations IPs in `slips_files/organizations_info/`.
+
+Otherwise, Slips triggers and "unknown port" evidence.
+
+For example, even though 5223/TCP isn't a well known port, Apple uses it in Apple Push Notification Service (APNS).
+
+The threat level of this evidence depends on the state of hte flow. established connections have higher threat levels.
 
-Rejected connections (not established) are detected as 'Multiple reconnection attempts'. for more details check
-[Multiple reconnections](https://stratospherelinuxips.readthedocs.io/en/develop/flowalerts.html#multiple-reconnection-attempts)
-below
 
 ## Data Upload
 
@@ -317,7 +332,7 @@ When there's a weird HTTP method, slips detects it as well.
 
 ## Non-SSL connections on port 443
 
-Slips detects established connections on port 443 that are not using HTTP
+Slips detects established connections on port 443 that are not using SSL
 using zeek's conn.log flows
 
 if slips finds a flow using destination port 443 and the 'service' field
@@ -337,7 +352,7 @@ Here's how it works
 
 Slips detects when a private IP is connected to another private IP with threat level info.
 
-But it skips this alert when it's a DNS connection on port
+But it skips this alert when it's a DNS or a DHCP connection on port
 53, 67 or 68 UDP to the gateway IP.
 
 ## Connection to private IPs outside the current local network
@@ -383,16 +398,20 @@ If so, it alerts "Device changing IPs".
 
 ## GRE tunnels
 
-Slips uses zeek tunnel.log to alert on GRE tunnels when found. Whenever one is found, slips sets an evidence
+Slips uses zeek tunnel.log to alert on GRE tunnels when found. Whenever one
+any action other than "Tunnel::DISCOVER" is found, slips sets an evidence
 with threat level low
 
 ## GRE tunnel scans
 
-Slips uses zeek tunnel.log to alert on GRE tunnels with DISCOVER actions when found.
+Slips uses zeek tunnel.log to alert on GRE tunnels scan. Slips considers any log with "Tunnel::DISCOVER" action a GRE scan.
+
 The threat level of this evidence is low.
 
 
 ## Invalid DNS resolutions
 
 Some DNS resolvers answer the DNS query to adservers with 0.0.0.0 or 127.0.0.1 as the ip of the domain to block the domain.
 Slips detects this and sets an informational evidence.
+
+This detection doesn't apply to queries ending with ".arpa" or ".local"

docs/usage.md

@@ -374,6 +374,9 @@ contains that ioc. For example, if you whitelist the flow of the domain slack.co
 request to the DNS server 1.2.3.4 asking for slack.com will still be shown.
 
 
+This whitelist can be enabled or disabled by changing the ```enable_local_whitelist``` key in `config/slips.yaml`.
+
+The attacker and victim of every evidence are checked against the whitelist. In addition to all the related IPs, DNS resolutions, SNI, and CNAMEs of the attacker and teh victim. If any of them are whitelisted, the flow/evidence is discarded.
 
 ### Flows Whitelist
 If you whitelist an IP address, Slips will check all flows and see if you are whitelisting to them or from them.
@@ -426,7 +429,7 @@ Slips still shows the flows to and from these IoC.
 The tranco list is updated daily by default in Slips, but you can change how often to update it using the
 ```online_whitelist_update_period``` key in config/slips.yaml.
 
-
+Tranco whitelist can be enabled or disabled by changing the ```enable_online_whitelist``` key in `config/slips.yaml`.
 
 ### Whitelisting Example
 You can modify the file ```config/whitelist.conf``` file with this content: