Merge pull request #1716 from stratosphereips/sebas-alert-local-visualizer

Add a TAXII web interface for visualising Slips alerts

Author: AlyaGomaa

.gitmodules

@@ -14,3 +14,7 @@
 	path = feel_project
 	url = https://github.com/stratosphereips/feel_project
     branch = main
+
+[submodule "SlipsWeb"]
+	path = SlipsWeb
+	url = https://github.com/stratosphereips/SlipsWeb.git

SlipsWeb

@@ -347,7 +347,7 @@ exporting_alerts:
 
   # Configuer all the methods Slips will export data with
   # Available options are slack or stix
-  # export_to : [stix]
+  # export_to : [stix] (And a TAXII server)
   # export_to : [slack]
   export_to: []
 
@@ -373,12 +373,11 @@ exporting_alerts:
   # For Stix, if Slips should use TLS
   use_https: false
 
-  # TAXII
-  discovery_path: /services/discovery-a
-  inbox_path: /services/inbox-a
+  # TAXII 2 discovery endpoint (relative path or full URL)
+  discovery_path: /taxii2/
 
-  # Collection on the server you want to push stix data to
-  collection_name: collection-a
+  # Collection (ID or title) on the server you want to push STIX data to
+  collection_name: Alerts
 
   # This value is only used when Slips is running non-stop (e.g with -i )
   # push_delay is the time to wait before pushing STIX data to server
@@ -390,13 +389,7 @@ exporting_alerts:
 
   # TAXII server credentials
   taxii_username: admin
-  taxii_password: admin
-
-  # URL used to obtain JWT token. set this to '' if you don't want to use it
-  # is required for JWT based authentication.
-  # (JWT based authentication is Optional)
-  # It's usually /management/auth
-  jwt_auth_path: /management/auth
+  taxii_password: changeme_before_installing_a_medallion_server
 
 #############################
 CESNET:

config/slips.yaml

@@ -64,7 +64,7 @@ RUN apt update && apt install -y --no-install-recommends \
     && curl -fsSL https://download.opensuse.org/repositories/security:zeek/xUbuntu_22.04/Release.key | gpg --dearmor |  tee /etc/apt/trusted.gpg.d/security_zeek.gpg > /dev/null \
     && apt update \
     && apt install -y --no-install-recommends --fix-missing \
-    zeek \
+    zeek-8.0 \
     npm \
     && ln -s /opt/zeek/bin/zeek /usr/local/bin/bro \
     && apt clean \

docker/Dockerfile

@@ -27,7 +27,7 @@ RUN set -eux; \
         | tee /etc/apt/sources.list.d/security:zeek.list \
     && curl -fsSL https://download.opensuse.org/repositories/security:zeek/xUbuntu_22.04/Release.key \
         | gpg --dearmor | tee /etc/apt/trusted.gpg.d/security_zeek.gpg > /dev/null \
-    && apt-get update && apt-get install -y --no-install-recommends --fix-missing zeek \
+    && apt-get update && apt-get install -y --no-install-recommends --fix-missing zeek-8.0  \
     && ln -s /opt/zeek/bin/zeek /usr/local/bin/bro \
     && apt-get clean \
     && rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/*

docker/light/Dockerfile

@@ -16,7 +16,6 @@ scikit_learn
 slackclient
 matplotlib
 stix2
-cabby
 pandas
 setuptools
 numpy

docker/light/excluded_libs.txt

@@ -40,46 +40,44 @@ You can do this by going to the channel, then clicking on the channel's name. Th
 
 ## STIX
 
-If you want to export alerts to your TAXII server using STIX format, change ```export_to``` variable to export to STIX, and Slips will automatically generate a
-```STIX_data.json``` containing all alerts it detects.
+If you want to export alerts to your TAXII 2 server using STIX 2.1 format,
+set ```export_to``` to ```stix``` and Slips will automatically generate a
+```STIX_data.json``` bundle containing the indicators it detects and push it to
+your collection.
 
 
     [ExportingAlerts]
     export_to = [stix]
 
 
-You can add your TAXII server details in the following variables:
+Configure the TAXII client by editing the following variables:
 
-```TAXII_server```: link to your TAXII server
+```TAXII_server```: host name or IP address of the TAXII server.
 
-```port```: port to be used
+```port```: TCP port (optional, defaults to 80/443).
 
-```use_https```: use https or not.
+```use_https```: set to true to connect over HTTPS (be careful that the default TAXII server in SlipsWeb, Medallion, do not support HTTPS yet)
 
-```discovery_path``` and ```inbox_path``` should contain URIs not full urls. For example:
+```discovery_path```: TAXII discovery endpoint path or full URL
+ (for example ```/taxii2/```).
 
-```python
-discovery_path = /services/discovery-a
-inbox_path = /services/inbox-a
-```
-
-```collection_name```: the collection on the server you want to push your STIX data to.
-
-```push_delay```: the time to wait before pushing STIX data to server (in seconds).
-It is used when slips is running non-stop (e.g with -i )
+```collection_name```: ID or title of the TAXII collection that should receive your indicators. Be default `Alerts`.
 
-```taxii_username```: TAXII server user credentials
+```push_delay```: time between automatic pushes (in seconds) when Slips is
+running continuously.
 
-```taxii_password```: TAXII server user password
+```taxii_username``` / ```taxii_password```: credentials used for HTTP Basic authentication.
 
-```jwt_auth_path```: auth path if JWT based authentication is used. It's usually /management/auth. this is what we
-use to get a token.
+**Change the default config password of the TAXII servers you are going to export to in ```config/medallion_config.yaml```**
 
 
-if your TAXII server is a remote server, you can set the ```port``` to 443 or 80.
+Slips stores the generated bundle for each run in the output directory of that
+execution (for example `output/<run_id>/STIX_data.json`), so you can inspect the
+exact STIX objects that were pushed.
 
-If running on a file, Slips will export to server after analysis is done.
-If running on an interface, Slips will export to server every push_delay seconds. by default it's 1h.
+If running on a file, Slips will export once before shutdown.
+If running on an interface, Slips will export to the server every
+```push_delay``` seconds (default 1 hour).
 
 ## JSON format
 

docs/exporting.md

@@ -63,3 +63,4 @@ This documentation gives an overview how Slips works, how to use it and how to h
    contributing
    code_documentation
    related_repos
+   visualisation

docs/index.rst

@@ -203,26 +203,25 @@ You can read more about it [here](https://stratospherelinuxips.readthedocs.io/en
 
 ## Installing Slips natively
 
-Slips is dependent on three major elements:
+Slips depends on three major elements:
 
 - Python 3.10.12
 - Zeek 8.0.0
 - Redis database 7.0.4
 
-To install these elements we will use APT package manager. After that, we will install python packages required for Slips to run and its modules to work. Also, Slips' interface Kalipso depend on Node.JS and several npm packages.
 
+To install these elements, the script will use the APT package manager. After that, it will install python packages required for Slips to run and its modules to work. Also, Slips' interface Kalipso depend on Node JS and several npm packages.
 
 
 **Instructions to download everything for Slips are below.**
 <br>
 
 ### Install Slips using shell script
-You can install it using install.sh
+You can install it using [install.sh](https://github.com/stratosphereips/StratosphereLinuxIPS/blob/master/install/install.sh)
 
 	sudo chmod +x install.sh
 	sudo ./install.sh
 
-
 ### Installing Slips manually
 #### Installing Python, Redis, NodeJs, and required python and npm libraries.
 
@@ -314,6 +313,7 @@ You can kill this redis database by running:
 ```
 then choosing 1.
 
+After these steps, if you need the submodules, you will need to clone them as done in the `install.sh` script.
 
 
 ## Installing Slips on a Raspberry PI

docs/installation.md

@@ -0,0 +1,9 @@
+# Slips Web Visualization
+
+To see the alerts of Slips in a visual way, the methodology is the following
+
+1. Slips must be configured to export the alerts in STIX format to a TAXII server, as explained in [exporting](https://stratospherelinuxips.readthedocs.io/en/develop/exporting.html).
+2. You need to install a TAXII server (available in the SlipsWeb submodule folder). See its README.md
+3. Use the program `SlipsWeb` that is availbale in the StratosphereWeb submodule that reads from the TAXII server.
+
+All the setup does not consume many resources, so you can run this visualization even in small servers like a Raspberry Pi. However, by having many Slips exporting to the same server you can centralize the visualization of many sensors in a unique location, probably with more hardware if needed.