You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: README.md
+8-55Lines changed: 8 additions & 55 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,5 +1,5 @@
1
1
<h1align="center">
2
-
Slips v1.1.2
2
+
Slips v1.1.3
3
3
</h1>
4
4
5
5
@@ -33,7 +33,6 @@ Slips v1.1.2
33
33
-[GUI](#graphical-user-interface)
34
34
-[Requirements](#requirements)
35
35
-[Installation](#installation)
36
-
-[Extended Usage](#extended-usage)
37
36
-[Configuration](#configuration)
38
37
-[Features](#features)
39
38
-[Contributing](#contributing)
@@ -60,7 +59,7 @@ Slips is a powerful endpoint behavioral intrusion prevention and detection syste
60
59
Slips is the first free software behavioral machine learning-based IDS/IPS for endpoints. It was created in 2012 by Sebastian Garcia at the Stratosphere Laboratory, AIC, FEE, Czech Technical University in Prague. The goal was to offer a local IDS/IPS that leverages machine learning to detect network attacks using behavioral analysis.
61
60
62
61
63
-
Slips is supported on Linuxand MacOS only. The blocking features of Slips are only supported on Linux
62
+
Slips is supported on Linux, MacOS, and windows dockers only. The blocking features of Slips are only supported on Linux
64
63
65
64
Slips is Python-based and relies on [Zeek network analysis framework](https://zeek.org/get-zeek/) for capturing live traffic and analyzing PCAPs. and relies on
66
65
Redis >= 7.0.4 for interprocess communication.
@@ -70,7 +69,7 @@ Redis >= 7.0.4 for interprocess communication.
##### [Analyse your own traffic without P2P](https://stratospherelinuxips.readthedocs.io/en/develop/installation.html#analyse-your-own-traffic)
180
-
181
-
182
-
##### [Analyse your own traffic with P2P ](https://stratospherelinuxips.readthedocs.io/en/develop/installation.html#for-p2p-support-on-linux)
183
-
184
-
185
-
##### [Analyse a pcap without using P2P](https://stratospherelinuxips.readthedocs.io/en/develop/installation.html#analyze-your-pcap-file)
186
-
187
-
188
-
189
-
### Macos M1
190
-
191
-
#### [Analyse your own traffic without using P2P](https://stratospherelinuxips.readthedocs.io/en/develop/installation.html#id1)
192
-
193
-
194
-
### MacOS Intel processors
195
-
196
-
197
-
##### [Analyse your own traffic without using P2P](https://stratospherelinuxips.readthedocs.io/en/develop/installation.html#id2)
198
-
199
-
200
-
##### [Analyse your own traffic with using P2P](https://stratospherelinuxips.readthedocs.io/en/develop/installation.html#for-p2p-support-on-macos-intel)
201
-
202
-
203
-
##### [Analyse a PCAP without using P2P](https://stratospherelinuxips.readthedocs.io/en/develop/installation.html#id2)
204
-
205
-
206
159
207
160
# Configuration
208
161
Slips has a [config/slips.yaml](https://github.com/stratosphereips/StratosphereLinuxIPS/blob/develop/config/slips.yaml) that contains user configurations for different modules and general execution.
Copy file name to clipboardExpand all lines: docs/architecture.md
+32-21Lines changed: 32 additions & 21 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -4,19 +4,19 @@ The architecture of Slips is basically:
4
4
- To receive some data as input
5
5
- To process it to a common format
6
6
- To enrich it (gather all possible info about the IPs/MAC/User-Agents etc.)
7
-
- To apply detection modules
7
+
- To apply detection modules
8
8
- To output results
9
9
10
10
Slips is heavily based on the Zeek monitoring tool as input tool for packets from the interface and pcap file, due to its excelent recognition of protocols and easiness to identify the content of the traffic.
11
11
12
-
Figure 1 shows how the data is analyzed by Slips.
12
+
Figure 1 shows how the data is analyzed by Slips.
13
13
As we can see, Slips internally uses <ahref="https://zeek.org/">Zeek</a>, an
14
14
open source network security monitoring tool. Slips divides flows into profiles and
15
15
each profile into a timewindows.
16
-
Slips runs detection modules on each flow and stores all evidence,
17
-
alerts and features in an appropriate profile structure.
16
+
Slips runs detection modules on each flow and stores all evidence,
17
+
alerts and features in an appropriate profile structure.
18
18
All profile info, performed detections, profiles and timewindows' data,
19
-
is stored inside a <ahref="https://redis.io/">Redis</a> database.
19
+
is stored inside a <ahref="https://redis.io/">Redis</a> database.
20
20
All flows are read, interpreted by Slips, labeled, and stored in the SQLite database in the output/ dir of each run
21
21
The output of Slips is a folder with logs (output/ directory) that has alert.json, alerts.log, errors.log.
22
22
Kalipso, a terminal graphical user interface. or the Web interface.
@@ -25,7 +25,7 @@ Kalipso, a terminal graphical user interface. or the Web interface.
25
25
.zoom {
26
26
transition: transform .2s; /* Animation */
27
27
margin: 0;
28
-
position: relative;
28
+
position: relative;
29
29
z-index:999;
30
30
}
31
31
@@ -42,7 +42,7 @@ Kalipso, a terminal graphical user interface. or the Web interface.
42
42
43
43
44
44
Below is more explanation on internal representation of data, usage of Zeek and usage of Redis inside Slips.
45
-
### Internal representation of data.
45
+
### Internal representation of data.
46
46
47
47
Slips works at a flow level, instead of a packet level, gaining a high level view of behaviors. Slips creates traffic profiles for each IP that appears in the traffic. A profile contains the complete behavior of an IP address. Each profile is divided into time windows. Each time window is 1 hour long by default and contains dozens of features computed for all connections that start in that time window. Detections are done in each time window, allowing the profile to be marked as uninfected in the next time window.
48
48
@@ -56,20 +56,20 @@ This is what slips stores for each IP/Profile it creates:
56
56
* Used software - list of software used by this profile, for example SSH, Browser, etc.
57
57
* MAC and MAC Vendor - Ether MAC of the IP and the name of the vendor
58
58
* Host-name - the name of the IP
59
-
* first User-agent - First UA seen use dby this profile.
59
+
* first User-agent - First UA seen use dby this profile.
60
60
* OS Type - Type of OS used by this profile as extracted from the user agent
61
61
* OS Name - Name of OS used by this profile as extracted from the user agent
62
62
* Browser - Name of the browser used by this profile as extracted from the user agent
63
-
* User-agents history - history of the all user agents used by this profile
63
+
* User-agents history - history of the all user agents used by this profile
64
64
* DHCP - if the IP is a dhcp or not
65
-
* Starttime - epoch formatted timestamp of when the profile first appeared
65
+
* Starttime - epoch formatted timestamp of when the profile first appeared
66
66
* Duration - the standard duration of every TW in this profile
67
67
* Modules labels - the labels assigned to this profile by each module
68
68
* Gateway - if the IP is the gateway (router) of the network
69
-
* Timewindow count - Amount of timewindows in this profile
69
+
* Timewindow count - Amount of timewindows in this profile
70
70
* ASN - autonomous service number of the IP
71
71
* Asnorg - name of the org that own the ASN of this IP
72
-
* ASN Number
72
+
* ASN Number
73
73
* SNI - Server name indicator
74
74
* Reverse DNS - name of the IP in reverse dns
75
75
* Threat Intelligence - If the IP appeared in any of Slips blacklist
@@ -85,32 +85,32 @@ This is what slips stores for each IP/Profile it creates:
85
85
* Url ratio: The higher the score the more malicious this IP is
86
86
87
87
88
-
### Alerts vs Evidence
88
+
### Alerts vs Evidence
89
89
90
90
When running Slips, the alerts you see in red in the CLI or at the very bottom in kalispo, are a bunch of evidence. Evidence in slips are detections caused by a specific IP in a specific timeframe. Slips doesn't alert on every evidence/detection. it accumulates evidence and only generates and alert when the amount of gathered evidence crosses a threshold. After this threshold Slips generates an alert, marks the timewindow as malicious(displays it in red in kalipso) and blocks the IP causing the alert.
91
-
92
-
### Usage of Zeek.
91
+
92
+
### Usage of Zeek.
93
93
94
94
Slips uses Zeek to generate files for most input types, and this data is used to create the profiles. For example, Slips uses this data to create a visual timeline of activities for each time window. This timeline consists of Zeek generated flows and additional interpretation from other logs like dns log and http log.
95
95
96
96
97
-
### Usage of Redis database.
97
+
### Usage of Redis database.
98
98
99
99
All the data inside Slips is stored in Redis, an in-memory data structure.
100
100
Redis allows all the modules in Slips to access the data in parallel.
101
101
Apart from read and write operations, Slips takes advantage of the Redis messaging system called Redis PUB/SUB.
102
-
Processes may publish data into the channels, while others subscribe to these channels and process the new data when it is published.
102
+
Processes may publish data into the channels, while others subscribe to these channels and process the new data when it is published.
103
103
104
-
### Usage of SQLite database.
104
+
### Usage of SQLite database.
105
105
106
106
Slips uses SQLite database to store all flows in Slips interpreted format.
107
107
The SQLite database is stored in the output/ dir and each flow is labeled to either 'malicious' or 'benign' based on slips detections.
108
108
all the labeled flows in the SQLite database can be exported to tsv or json format.
109
109
110
110
111
-
### Threat Levels
111
+
### Threat Levels
112
112
113
-
Slips has 4 threat levels.
113
+
Slips has 5 threat levels.
114
114
115
115
<styletype="text/css">
116
116
.tg {border-collapse:collapse;border-spacing:0;}
@@ -157,6 +157,17 @@ Slips has 4 threat levels.
157
157
</tr>
158
158
159
159
160
+
### How Slips Stops
161
+
162
+
- When slips is running on an interface or a growing zeek directory, slips keeps running forever until the user presses ctrl+c
163
+
- When Slips is analyzing a PCAP or a zeek directory or any other supported file, It keeps running until no more flows are received.
164
+
- After the modules receive that signal that says "no more new flows are coming", all modules keep processing the existing flows normally until they run out of msgs and stop.
165
+
- Modules stop only if no more msgs are received in their Redis channels, and if they receive the signal that slips is no longer receiving new flows.
166
+
- Slips knows that no more flows are arriving when it reaches the end of the given zeek/suricata/nfdump logs.
167
+
- If some processes are hanging in memory, slips wait by default 1 week before killing them. This can be modified in the config.yaml.
168
+
169
+
For more techincal details about this check https://stratospherelinuxips.readthedocs.io/en/develop/contributing.html#faq
1. slips.py is the entry point, it's responsible for starting all modules, and keeping slips up until the analysis is finished.
8
-
2. slips.py starts the input process, which is the one responsible for reading the flows from the files given to slips using -f
9
-
it detects the type of file, reads it and passes the flows to the profiler process. if slips was given a PCAP or is running on an interface
10
-
, the input process starts a zeek thread that analyzes the pcap/interface using slips' own zeek configuration and sends the generated zeek
11
-
flows to the profiler process.
12
-
3. slips.py also starts the update manager, it updates slips local TI files, like the ones stored in slips_files/organizations_info and slips_files/ports_info.
13
-
later, when slips is starting all the modules, slips also starts the update manager but to update remote TI files in the background in this case.
14
-
4. Once the profiler process receives the flows read by the input process, it starts to convert them to a structure that slips can deal with.
15
-
it creates profiles and time windows for each IP it encounters.
16
-
5. Profiler process gives each flow to the appropriate module to deal with it. for example flows from http.log will be sent to http_analyzer.py
17
-
to analyze them.
18
-
6. Profiler process stores the flows, profiles, etc. in slips databases for later processing. the info stored in the dbs will be used by all modules later.
19
-
Slips has 2 databases, Redis and SQLite. it uses the sqlite db to store all the flows read and labeled. and uses redis for all other operations. the sqlite db is
20
-
created in the output directory, meanwhite the redis database is in-memory.
21
-
7-8. using the flows stored in the db in step 6 and with the help of the timeline module, slips puts the given flows in a human-readable form which is
22
-
then used by the web UI and kalipso UI.
23
-
9. when a module finds a detection, it sends the detection to the evidence process to deal with it (step 10) but first, this evidence is checked by the whitelist to see if it's
24
-
whitelisted in our config/whitelist.conf or not. if the evidence is whitelisted, it will be discarded and won't go through the next steps
25
-
10. now that we're sure that the evidence isn't whitelisted, the evidence process logs it to slips log files and gives the evidence to all modules responsible for exporting
26
-
evidence. so, if CEST, Exporting modules, or CYST is enabled, the evidence process notifies them
27
-
through redis channels that it found an evidence and it's time to share the evidence.
28
-
11. if the blocking module is enabled using -p, the evidence process shares all detected alerts to the blocking module. and the blocking module handles
29
-
the blocking of the attacker IP through the linux firewall (supported in linux only)
30
-
12. if p2p is enabled in config/slips.yaml, the p2p module shares the IP of the attacker, its' score and blocking requests sent by the evidence process
31
-
with other peers in the network so they can block the attackers before they reach them.
32
-
13. The output process is slips custom logging framework. all alerts, warnings and info printed are sent here first for proper formatting and printing.
33
-
34
-
This is a brief explanation of how slips works for new contributors.
35
-
36
-
All modules described above are talked about in more detail in the rest of the documentation.
0 commit comments