Skip to content

Commit 7d9216e

Browse files
committed
http datasources console command and documentation refactoring
1 parent 0c1aad0 commit 7d9216e

File tree

8 files changed

+480
-73
lines changed

8 files changed

+480
-73
lines changed

README.md

Lines changed: 17 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -1,12 +1,14 @@
11
![Sentinel Kit](docs/img/sentinel-kit_logo.png)
22
# 🛡️ Sentinel Kit: The Simplified Platform for Incident Response (SOC & DFIR)
33

4-
## WARNING: This project is currently in an early stage of development. Not all components have been ported to this repository, and the features are not yet stable enough for production use.
4+
$\color{red}{\textsf{**WARNING**: This project is currently in an early stage of development. Not all components have been ported to this repository, and the features are not yet stable enough for production use.}}$
5+
6+
The features already available online are documented [here](docs/index.md).
57
---
68

79
**Sentinel Kit** is a comprehensive Docker stack designed to provide **Digital Forensics and Incident Response (DFIR)** and **Security Operations Center (SOC)** capabilities with unparalleled deployment simplicity.
810

9-
Ideal for **situational monitoring** or **rapid security incident response**, this integrated platform enables collection, analysis, detection, and immediate response to threats.
11+
Ideal for **situational monitoring** or **small-scale security incident response**, this integrated platform enables collection, analysis, detection, and immediate response to threats.
1012

1113
---
1214

@@ -91,33 +93,34 @@ All of this can be edited in `.env` file
9193

9294
## 🛠️ Technical Architecture (via `docker-compose.yml`)
9395

94-
The architecture is modular and relies on the interconnection of several services via the **sentinel-kit-network** network.
96+
The architecture is modular and relies on the interconnection of several services
9597
![Sentinel-Kit architecture](docs/img/sentinel-kit_network_flow.png)
9698

9799
## ⚙️ Configuration
98100

99101
Main configurations are located in the `config/` folder: (edit these elements only if you know what you are doing 😊)
100102

101-
* `config/caddy_server`: Reverse proxy that serve front and back-end web applications
102-
* `config/certificates`: Contains TLS certification chains for elasticstack, caddy and backend JWT
103+
* `config/caddy_server`: Reverse proxy that serve front and back-end web applications.
103104
* `config/docker-config`: Server stack configuration (dockerfile, entrypoints...).
105+
* `config/elasticsearch`: Configuration of the Elasticsearch certification chain and nodes cluster.
104106
* `config/fluentbit_server`: Fluent Bit configuration files (inputs, filters, outputs to Elasticsearch).
105107
* `config/grafana`: Grafana initial setup (datasources and dashboards).
106-
* `config/prometheus/prometheus.yml`: Prometheus monitoring targets configuration.
108+
* `config/prometheus`: Prometheus monitoring targets configuration.
107109
* `config/sigma_ruleset`: sigma rules used on elasticsearch ingested logs
108110
* `config/yara_ruleset`: yara rules used on `data/yara_triage_data` folder or by *sentinel-kit_datamonitor* agent
109111

110112
## 📖 Data
111113

112114
Persistent data are located in the `data/` folder:
113115

114-
* `data/caddy_logs`: Store the caddy server access & error logs
115-
* `data/ftp_data`: Store file uploaded on the SFTP server
116-
* `data/grafana`: Contains a persistence of your grafana profile if you want to make your own dashboard and customizations
117-
* `data/kibana`: Kibana user customizations
118-
* `data/log_ingest_data`: Is designed to forward logs if you don't want to use fluentbit HTTP forwarder
119-
* `data/mysql_data`: Constains a persistence of the web backend database
120-
* `data/yara_triage_data`: is used to automatically scan any file placed in this folder
116+
* `data/caddy_logs`: Store the caddy server access & error logs.
117+
* `data/fluentbit_db`: fluentbit ingest database (to avoid indexing same data several times).
118+
* `data/ftp_data`: Store file uploaded on the SFTP server.
119+
* `data/grafana`: Contains a persistence of your grafana profile if you want to make your own dashboard and customizations.
120+
* `data/kibana`: Kibana user customizations (dashboard, config...).
121+
* `data/log_ingest_data`: Is designed to forward logs if you don't want to use fluentbit HTTP forwarder.
122+
* `data/mysql_data`: Constains a persistence of the web backend database.
123+
* `data/yara_triage_data`: is used to automatically scan any file placed in this folder.
121124
122125
---
123126
@@ -129,11 +132,7 @@ To stop and remove the containers, networks, and volumes created by Docker Compo
129132
docker-compose down -v
130133
```
131134
132-
If you want to erase all user data:
133-
* remove the __content__ of every folder inside `data/`
134-
* remove the __content__ of `config/certificates/` in caddy_server, elasticsearch and jwt
135-
* remove the __content__ of `config/grafana`
136-
* finally, rebuild the stack with the following command:
135+
If you want to erase all user data, and start from a fresh and clean installation, there is a `clean-user-data` sh or powershell (depending on your OS) to help you erasing all personal data. Then, you can rebuild the whole stack with:
137136
138137
```bash
139138
docker-compose up --build --force-recreate

docs/01-start-sentinel-kit.md

Lines changed: 75 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,75 @@
1+
# Sentinel-Kit quick start
2+
3+
## ⚠️ Security warning
4+
Although the code in this stack is designed to be secure in terms of authentication and client/server exchanges (strong identification, certificates, JWT, etc.), it is your responsibility to limit the server's exposure (flow filtering, whitelisting, etc.). No filtering mechanism is provided in the project, so configure your flows to limit exposure as follows:
5+
6+
![Sentinel Kit architecture and network flow](img/sentinel-kit_network_flow.png)
7+
8+
**Exposed services:**
9+
| **Web API** | Used for clients<->server communications and admin actions over the web interface |
10+
| **SFTP Server** | Secure file/evidence upload |
11+
12+
optional - for interoperability purpose with a forwarder like logstash / winlogbeat / fluentbit etc... It could also be done over Web API (see documentation)
13+
| **Syslog forwarder** | Fluentbit log forwarder to elasticsearch
14+
15+
**Admin access**
16+
| **Web front end** | Access to the admin application (agents control, log analysis...) |
17+
18+
**Admin extended features**
19+
| **PhpMyAdmin** | MySQL database management |
20+
| **Grafana** | Monitoring dashboards (fluentbit forwarder logs / elasticsearch ingestion / stack health)
21+
| **Kibana** | Kibana complete access (for advanced usage only. Standard features are directly implemented in the admin web interface)
22+
23+
## Prerequisites
24+
This project is designed to be deployed in minutes using Docker Compose.
25+
* **Docker**
26+
* **Docker Compose** (or Docker Engine including Compose)
27+
* Minimum **8 GB of RAM** (essential for Elasticsearch)
28+
29+
## 🚀 Let's start Sentinel-Kit
30+
Everything comes as a full docker-compose stack to avoid configuration and depencies and simplify the deployment.
31+
32+
First, clone sentinel kit repository
33+
```bash
34+
git clone
35+
cd sentinel-kit
36+
```
37+
38+
Then, in a local environment, you need to define the following DNS entries:
39+
```bash
40+
# OS `hosts` file
41+
127.0.0.1 sentinel-kit.local
42+
127.0.0.1 backend.sentinel-kit.local
43+
127.0.0.1 phpmyadmin.sentinel-kit.local
44+
127.0.0.1 kibana.sentinel-kit.local
45+
127.0.0.1 grafana.sentinel-kit.local
46+
```
47+
48+
On advanced configuration, you can configure `config/caddy_server/Caddyfile` if you want to set your own nameserver. If so, you will also need to set your hostname, replacing the original ones in `.env` file. You can find details about advanced configuration [here](02-customize-stack.md).
49+
50+
When your DNS configuration is ok. Start the stack with:
51+
```bash
52+
docker-compose up -d
53+
```
54+
Initial startup could be long as elastic configure two nodes and kibana.
55+
56+
Caddy server will generate a complete certification chain for the exposed services (frontend, backend, phpmyadmin, kibana, grafana). If you don't want to always accept untrusted certification chains, add root and intermediate CA to your browser certificates. They are located in ./config/certificates/caddy_server
57+
58+
[https://sentinel-kit.local]https://sentinel-kit.local should return the following page:
59+
![Sentinel Kit Homepage](img/sentinel-kit_homepage.png)
60+
61+
And, that's all, you are good to go!
62+
63+
## 🛑 Stopping and Cleaning the Stack
64+
65+
To stop and remove the containers, networks, and volumes created by Docker Compose:
66+
67+
```bash
68+
docker-compose down -v
69+
```
70+
71+
If you want to erase all user data, and start from a fresh and clean installation, there is a `clean-user-data` sh or powershell (depending on your OS) to help you erasing all personal data. Then, you can rebuild the whole stack with:
72+
73+
```bash
74+
docker-compose up --build --force-recreate
75+
```

docs/02-customize-stack.md

Lines changed: 88 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,88 @@
1+
# 🛠️ Environment Configuration
2+
3+
Most of the parameters for this stack are defined in the **`.env`** file located at the root directory. These settings are loaded **before the stack starts**. If the stack is already running, you must **restart it** for any changes to take effect.
4+
5+
---
6+
7+
## ⚙️ Services and Profiles
8+
9+
The following configuration block defines the active services using Docker Compose profiles and sets the Elasticsearch cluster mode.
10+
11+
```bash
12+
COMPOSE_PROFILES=sftp,es-secondary-node,phpmyadmin,kibana,internal-monitoring
13+
ELASTICSEARCH_CLUSTER_MODE=multi-node
14+
```
15+
## Profile Customization
16+
17+
You can remove certain services if you don't need them. Simply remove the corresponding profile from the `COMPOSE_PROFILES` list:
18+
19+
* Remove internal-monitoring to disable access to Grafana features.
20+
* Remove phpmyadmin to disable direct access to the MySQL database.
21+
* Remove sftp if no external file upload functionality is required.
22+
23+
## Elasticsearch Configuration
24+
25+
By default, Elasticsearch is configured as a two-node cluster (multi-node). For environments with limited resources, you can switch to a single-node setup by removing the `es-secondary-node` profile and updating `ELASTICSEARCH_CLUSTER_MODE` as shown below:
26+
27+
```bash
28+
COMPOSE_PROFILES=sftp,phpmyadmin,kibana,internal-monitoring
29+
ELASTICSEARCH_CLUSTER_MODE=single-node
30+
```
31+
32+
**__Note:__** Switching between single-node and multi-node can be done at any point during the stack's lifecycle without requiring a complete reinstallation or data loss.
33+
34+
### Elasticsearch Memory Limit
35+
To limit the memory allocated to the Elasticsearch cluster, modify the following variable (default is 4GB):
36+
```bash
37+
ELASTICSEARCH_MEMORY_LIMIT=4294967296
38+
```
39+
40+
41+
🌐 Domain Names
42+
The hostnames for the exposed services are customizable here. It is mandatory to map these hostnames to the stack's IP address either in your DNS configuration or in your local hosts file (for isolated local installations).
43+
| Service | Environment Variable | Default Hostname |
44+
| Frontend | SENTINELKIT_FRONTEND_HOSTNAME | sentinel-kit.local |
45+
| Backend API | SENTINELKIT_BACKEND_HOSTNAME | backend.sentinel-kit.local |
46+
| phpMyAdmin | SENTINELKIT_PMA_HOSTNAME | phpmyadmin.sentinel-kit.local |
47+
| Kibana | SENTINELKIT_KIBANA_HOSTNAME | kibana.sentinel-kit.local |
48+
| Grafana | SENTINELKIT_GRAFANA_HOSTNAME | grafana.sentinel-kit.local |
49+
50+
🔒 Secrets and Credentials
51+
52+
For production usage, you can change any of the default credentials below.
53+
54+
⚠️ **__Important:__** Once the stack is initialized, changing certain secrets (like database or Elasticsearch credentials) can destabilize services. It is strongly recommended to modify these values only before the initial launch of the stack.
55+
56+
```bash
57+
SENTINELKIT_DATAMONITOR_SERVER_TOKEN=9561ffd1b6de615286b9e52a9d5bc3226970449700c9461bdbe4225730b47b20
58+
BACKEND_JWT_PASSPHRASE=f164cfc913d2faf65a1b7bc8ccd4aa8b11b5958bce7c20c8cf159a576f8a75f7
59+
MYSQL_ROOT_PASSWORD=sentinel-kit_r00tp4ssw0rd
60+
MYSQL_USER=sentinel-kit_user
61+
MYSQL_PASSWORD=sentinel-kit_passwd
62+
MYSQL_DATABASE=sentinel-kit_db
63+
GF_SECURITY_ADMIN_USER=sentinel-kit_grafana_admin
64+
GF_SECURITY_ADMIN_PASSWORD=sentinel-kit_grafana_password
65+
SFTP_USER=sentinel-kit_sftp_user
66+
SFTP_PASSWORD=sentinel-kit_sftp_passwd
67+
ELASTICSEARCH_CLUSTER_NAME=sentinel-kit-elasticsearch-cluster
68+
ELASTICSEARCH_PASSWORD=sentinelkit_elastic_passwd
69+
```
70+
71+
## 🛑 Stopping and Cleaning the Stack
72+
73+
To only stop the stack, without removing existing data, just do the following command:
74+
```bash
75+
docker compose down
76+
```
77+
78+
To stop __and remove the containers, networks, and volumes__ created by Docker Compose:
79+
80+
```bash
81+
docker-compose down -v
82+
```
83+
84+
If you want to erase all user data, and start from a fresh and clean installation, there is a `clean-user-data` sh or powershell (depending on your OS) to help you erasing all personal data. Then, you can rebuild the whole stack with:
85+
86+
```bash
87+
docker-compose up --build --force-recreate
88+
```

docs/03-create-admin-user.md

Lines changed: 66 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,66 @@
1+
# Create Sentinel Kit admin users
2+
3+
Once installed and operational, the Sentinel Kit administration interface is accessible by default at https://sentinel-kit.local. No web recording functionality is implemented, separating the functions of the platform administrator and the analyst who operates it.
4+
![Sentinel-Kit login page](sentinel-kit_login.png)
5+
6+
## Users management
7+
8+
Every user management operations are done inside the backend container. You can connect to it like this:
9+
```bash
10+
docker exec -it sentinel-kit-app-backend /bin/bash
11+
```
12+
13+
a console utility can be called with `php bin/console`. If you want to create a new user, just type the following command:
14+
15+
```bash
16+
# app:users:create <email> <password>
17+
backend:/var/www/html# php bin/console app:users:create [email protected] MyPassword!
18+
19+
[OK] User successfully created
20+
21+
==============================
22+
23+
OTP authenticator URL: https://backend.sentinel-kit.local/qrcode/b3RwYXV0aDovL3RvdHAvRGF0YU1vbml0b3IlMjBzZXJ2ZXIlM0FkZXYuanAuZ2FybmllciU0MGdtYWlsLmNvbSU0MERhdGFNb25pdG9yP2lzc3Vlcj1EYXRhTW9uaXRvciUyMHNlcnZlciZzZWNyZXQ9SzQzVVlIR1NTQlNETVlHU1hTQVNKN0ZDM0FVSTdFQTM3SExaN1c3QkhZUkZPNzM3VE9WUQ==
24+
==============================
25+
```
26+
OTP is directly available if the admin wants to store it. You don't need to comunicate it to the end user, he will access it on the first successful login on the platform.
27+
![Login OTP form](img/sentinel-kit_login_otp.png)
28+
29+
## Listing users
30+
Existing user accounts can be listed like this
31+
32+
```bash
33+
backend:/var/www/html# php bin/console app:users:list
34+
User ID: 1 | Email: [email protected]
35+
```
36+
37+
## Remove users
38+
You can delete an user providing its ID or email like this
39+
```bash
40+
backend:/var/www/html# php bin/console app:users:delete 1
41+
42+
[OK] User deleted successfully.
43+
44+
```
45+
46+
## Users secrets reset
47+
Password resetting could be done
48+
```bash
49+
backend:/var/www/html# php bin/console app:users:renew-password [email protected] MyNewPa$$w0rd
50+
51+
[OK] User password reset successful.
52+
```
53+
54+
And you can also reset user OTP:
55+
```bash
56+
backend:/var/www/html# php bin/console app:users:renew-otp [email protected]
57+
58+
[OK] User OTP reset successful.
59+
60+
==============================
61+
62+
OTP authenticator URL: https://localhost/qrcode/b3RwYXV0aDovL3RvdHAvRGF0YU1vbml0b3IlMjBzZXJ2ZXIlM0FkZW1vJTQwZXhhbXBsZS5jb20lNDBEYXRhTW9uaXRvcj9pc3N1ZXI9RGF0YU1vbml0b3IlMjBzZXJ2ZXImc2VjcmV0PVgzT1M3QVdDQ1Q3SUZISktCQ0JUUlczVjRWQlgzWDZCN1hLQ1I2NU5JRUZQTDVBNVVPR0E=
63+
```
64+
65+
Your users can now login on the platform
66+
![Sentinel-Kit Homepage](img/sentinel-kit_server_features.png)

0 commit comments

Comments
 (0)