Skip to content

Commit 24fd6bd

Browse files
colin-stubbsanupratharamachandran
colin-stubbs
authored and
anupratharamachandran
committed
Enhancement: Add beelzebub integration, resolve elastic#12910 (elastic#12914)
Add beelzebub integration to support log ingest from Beelzebub LLM enabled honeypots. Ingest is available from file using filestream, as well as http_endpoint.
1 parent 921d0c7 commit 24fd6bd

26 files changed

+2699
-0
lines changed

‎.github/CODEOWNERS‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -126,6 +126,7 @@
126126
/packages/bbot @elastic/security-service-integrations
127127
/packages/beaconing @elastic/ml-ui @elastic/sec-applied-ml
128128
/packages/beat @elastic/stack-monitoring
129+
/packages/beelzebub @elastic/security-service-integrations
129130
/packages/beyondinsight_password_safe @elastic/security-service-integrations
130131
/packages/beyondtrust_pra @elastic/security-service-integrations
131132
/packages/bitdefender @elastic/security-service-integrations

‎packages/beelzebub/_dev/build/build.yml‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
dependencies:
2+
ecs:
3+
reference: [email protected]

‎packages/beelzebub/_dev/build/docs/README.md‎

Lines changed: 73 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,73 @@
1+
# Beelzebub Integration
2+
3+
Beelzebub is an advanced honeypot framework designed to provide a highly secure environment for detecting and analysing cyber attacks. It offers a low code approach for easy implementation and uses AI LLM's to mimic the behaviour of a high-interaction honeypot.
4+
5+
Beelzebub is available on GitHub via [https://github.com/mariocandela/beelzebub](https://github.com/mariocandela/beelzebub) or via [https://beelzebub-honeypot.com](https://beelzebub-honeypot.com)
6+
7+
This integration provides multiple ingest source options including log files and via HTTP POST.
8+
9+
This allows you to search, observe and visualize the Beelzebub logs through Elasticsearch and Kibana.
10+
11+
This integration was last tested with Beelzebub `v3.3.6`.
12+
13+
Please note that Beelzebub only produces NDJSON log files at this time, to ship logs to this integration via any other method you will require another component, such as [Logstash](https://www.elastic.co/logstash), which can perform this by reading the Beelzebub produced log files and transporting the content as it changes to an appropriately configured Elastic Agent input, an ingest location that can be utilised by an appropriately configured Elastic Agent, or directly into Elasticsearch.
14+
15+
For more information, refer to:
16+
1. [GitHub](https://github.com/mariocandela/beelzebub)
17+
2. [Official Beelzebub Project Website](https://beelzebub-honeypot.com)
18+
19+
## Compatability
20+
21+
The package collects log events from file or by receiving HTTP POST requests.
22+
23+
## Configuration
24+
25+
### Enabling the integration in Elastic
26+
27+
1. In Kibana go to **Management > Integrations**
28+
2. Ensure "Display beta integrations" is enabled beneath the category list to the left
29+
3. In "Search for integrations" search bar type **Beelzebub**
30+
4. Click on "Beelzebub" integration from the search results.
31+
5. Click on **Add Beelzebub** button to add the Beelzebub integration.
32+
6. Configure the integration as appropriate
33+
34+
### Configure the Beelzebub integration
35+
36+
1. Choose your ingest method, e.g. file or HTTP. If using HTTP you can enable HTTPS transport by providing an SSL certificate and private key.
37+
2. Choose to store the original event content in `event.original`, or not.
38+
3. Choose to redact passwords, or not.
39+
4. Configure advanced options if desired.
40+
41+
### Example Beelzebub Logging Configuration
42+
43+
Example `beelzebub.yaml` configuration.
44+
```
45+
core:
46+
logging:
47+
debug: false
48+
debugReportCaller: false
49+
logDisableTimestamp: false
50+
logsPath: ./logs/beelzebub.log
51+
tracings:
52+
rabbit-mq:
53+
enabled: false
54+
uri: ""
55+
prometheus:
56+
path: "/metrics"
57+
port: ":2112"
58+
beelzebub-cloud:
59+
enabled: false
60+
uri: ""
61+
auth-token: ""
62+
```
63+
64+
## Logs
65+
66+
The Beelzebub logs dataset provides logs from Beelzebub instances.
67+
68+
All Beelzebub logs are available in the `beelzebub.logs` field group.
69+
70+
{{fields "logs"}}
71+
72+
{{event "logs"}}
73+

‎packages/beelzebub/_dev/deploy/docker/docker-compose.yml‎

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,15 @@
1+
services:
2+
test-filestream:
3+
image: alpine
4+
volumes:
5+
- ./sample_logs:/sample_logs:ro
6+
- ${SERVICE_LOGS_DIR}:/var/log
7+
command: /bin/sh -c "cp /sample_logs/* /var/log/"
8+
test-http_endpoint:
9+
image: docker.elastic.co/observability/stream:v0.17.1
10+
volumes:
11+
- ./sample_logs:/sample_logs:ro
12+
environment:
13+
- STREAM_PROTOCOL=webhook
14+
- STREAM_ADDR=http://elastic-agent:10002/
15+
command: log --start-signal=SIGHUP --delay=5s /sample_logs/logs-ndjson.log

‎packages/beelzebub/_dev/deploy/docker/sample_logs/logs-ndjson.log‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
{"event":{"DateTime":"2025-02-13T01:08:26Z","RemoteAddr":"1.128.0.133:60748","Protocol":"SSH","Command":"","CommandOutput":"","Status":"Stateless","Msg":"New SSH attempt","ID":"1974e109-d6f8-4bb1-934c-180a163e1cb8","Environ":"","User":"root","Password":"test","Client":"SSH-2.0-dropbear","Headers":"","Cookies":"","UserAgent":"","HostHTTPRequest":"","Body":"","HTTPMethod":"","RequestURI":"","Description":"SSH interactive ChatGPT","SourceIp":"1.128.0.133","SourcePort":"60748"},"level":"info","msg":"New Event","status":"Stateless","time":"2025-02-13T01:08:26Z"}
2+
{"event":{"DateTime":"2025-02-13T01:08:34Z","RemoteAddr":"1.128.0.133:60748","Protocol":"SSH","Command":"ps w","CommandOutput":"```\n PID TTY STAT TIME COMMAND\n 2042 pts/0 Ss 0:00 bash\n 2106 pts/0 R+ 0:00 ps w\n```","Status":"Interaction","Msg":"New SSH Terminal Session","ID":"1c18ad80-60bb-48f6-8e47-05e707ac93eb","Environ":"","User":"","Password":"","Client":"","Headers":"","Cookies":"","UserAgent":"","HostHTTPRequest":"","Body":"","HTTPMethod":"","RequestURI":"","Description":"SSH interactive ChatGPT","SourceIp":"1.128.0.133","SourcePort":"60748"},"level":"info","msg":"New Event","status":"Interaction","time":"2025-02-13T01:08:34Z"}
3+
{"event":{"DateTime":"2025-02-27T00:03:02Z","RemoteAddr":"1.128.0.215:55264","Protocol":"HTTP","Command":"","CommandOutput":"","Status":"Stateless","Msg":"HTTP New request","ID":"4df41014-e1e8-45c1-8868-11c750ffd2bc","Environ":"","User":"","Password":"","Client":"","Headers":"[Key: User-Agent, values: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/90.0.4430.85 Safari/537.36 Edg/90.0.818.46],[Key: Accept-Encoding, values: gzip, deflate],[Key: Accept-Encoding, values: gzip, deflate],[Key: Connection, values: keep-alive],[Key: X-Requested-With, values: XMLHttpRequest],[Key: Content-Type, values: application/x-www-form-urlencoded; charset=UTF-8],[Key: Accept-Language, values: en US,en;q=0.9,sv;q=0.8],[Key: Accept, values: */*],","Cookies":"","UserAgent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/90.0.4430.85 Safari/537.36 Edg/90.0.818.46","HostHTTPRequest":"1.128.0.16:80","Body":"","HTTPMethod":"GET","RequestURI":"/","Description":"Wordpress 6.0","SourceIp":"1.128.0.215","SourcePort":"55264"},"level":"info","msg":"New Event","status":"Stateless","time":"2025-02-27T00:03:02Z"}
4+
{"event":{"DateTime":"2025-02-19T07:48:34Z","RemoteAddr":"1.128.0.58:41654","Protocol":"HTTP","Command":"","CommandOutput":"","Status":"Stateless","Msg":"HTTP New request","ID":"1d4373f6-6dde-4b96-8f09-3ffb472de389","Environ":"","User":"","Password":"","Client":"","Headers":"[Key: User-Agent, values: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/78.0.3904.108 Safari/537.36],[Key: Content-Length, values: 314],[Key: Content-Type, values: application/xml],[Key: Accept-Encoding, values: gzip],[Key: Connection, values: close],","Cookies":"","UserAgent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/78.0.3904.108 Safari/537.36","HostHTTPRequest":"1.128.0.16:80","Body":"\u003c!DOCTYPE xxe [\n\u003c!ELEMENT name ANY \u003e\n\u003c!ENTITY xxe SYSTEM \"file:///etc/passwd\"\u003e]\u003e\n\u003cAutodiscover xmlns=\"http://schemas.microsoft.com/exchange/autodiscover/outlook/responseschema/2006a\"\u003e\n\u003cRequest\u003e\n\u003cEMailAddress\u003eaaaaa\u003c/EMailAddress\u003e\n\u003cAcceptableResponseSchema\u003e\u0026xxe;\u003c/AcceptableResponseSchema\u003e\n\u003c/Request\u003e\n\u003c/Autodiscover\u003e","HTTPMethod":"POST","RequestURI":"/Autodiscover/Autodiscover.xml","Description":"Wordpress 6.0","SourceIp":"1.128.0.58","SourcePort":"41654"},"level":"info","msg":"New Event","status":"Stateless","time":"2025-02-19T07:48:34Z"}

‎packages/beelzebub/changelog.yml‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
# newer versions go on top
2+
- version: "0.1.0"
3+
changes:
4+
- description: Initial release of the package post review
5+
type: enhancement
6+
link: https://github.com/elastic/integrations/pull/12914
7+
- version: "0.0.4"
8+
changes:
9+
- description: Initial draft of the package
10+
type: enhancement
11+
link: https://github.com/elastic/integrations/pull/12914

‎packages/beelzebub/data_stream/logs/_dev/test/pipeline/test-common-config.yml‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
fields:
2+
tags:
3+
- preserve_original_event
4+
- redact_passwords

‎packages/beelzebub/data_stream/logs/_dev/test/pipeline/test-logs-ndjson.log‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
{"event":{"DateTime":"2025-02-13T01:08:26Z","RemoteAddr":"1.128.0.133:60748","Protocol":"SSH","Command":"","CommandOutput":"","Status":"Stateless","Msg":"New SSH attempt","ID":"1974e109-d6f8-4bb1-934c-180a163e1cb8","Environ":"","User":"root","Password":"test","Client":"SSH-2.0-dropbear","Headers":"","Cookies":"","UserAgent":"","HostHTTPRequest":"","Body":"","HTTPMethod":"","RequestURI":"","Description":"SSH interactive ChatGPT","SourceIp":"1.128.0.133","SourcePort":"60748"},"level":"info","msg":"New Event","status":"Stateless","time":"2025-02-13T01:08:26Z"}
2+
{"event":{"DateTime":"2025-02-13T01:08:34Z","RemoteAddr":"1.128.0.133:60748","Protocol":"SSH","Command":"ps w","CommandOutput":"```\n PID TTY STAT TIME COMMAND\n 2042 pts/0 Ss 0:00 bash\n 2106 pts/0 R+ 0:00 ps w\n```","Status":"Interaction","Msg":"New SSH Terminal Session","ID":"1c18ad80-60bb-48f6-8e47-05e707ac93eb","Environ":"","User":"","Password":"","Client":"","Headers":"","Cookies":"","UserAgent":"","HostHTTPRequest":"","Body":"","HTTPMethod":"","RequestURI":"","Description":"SSH interactive ChatGPT","SourceIp":"1.128.0.133","SourcePort":"60748"},"level":"info","msg":"New Event","status":"Interaction","time":"2025-02-13T01:08:34Z"}
3+
{"event":{"DateTime":"2025-02-27T00:03:02Z","RemoteAddr":"1.128.0.215:55264","Protocol":"HTTP","Command":"","CommandOutput":"","Status":"Stateless","Msg":"HTTP New request","ID":"4df41014-e1e8-45c1-8868-11c750ffd2bc","Environ":"","User":"","Password":"","Client":"","Headers":"[Key: User-Agent, values: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/90.0.4430.85 Safari/537.36 Edg/90.0.818.46],[Key: Accept-Encoding, values: gzip, deflate],[Key: Accept-Encoding, values: gzip, deflate],[Key: Connection, values: keep-alive],[Key: X-Requested-With, values: XMLHttpRequest],[Key: Content-Type, values: application/x-www-form-urlencoded; charset=UTF-8],[Key: Accept-Language, values: en US,en;q=0.9,sv;q=0.8],[Key: Accept, values: */*],","Cookies":"","UserAgent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/90.0.4430.85 Safari/537.36 Edg/90.0.818.46","HostHTTPRequest":"1.128.0.16:80","Body":"","HTTPMethod":"GET","RequestURI":"/","Description":"Wordpress 6.0","SourceIp":"1.128.0.215","SourcePort":"55264"},"level":"info","msg":"New Event","status":"Stateless","time":"2025-02-27T00:03:02Z"}
4+
{"event":{"DateTime":"2025-02-19T07:48:34Z","RemoteAddr":"1.128.0.58:41654","Protocol":"HTTP","Command":"","CommandOutput":"","Status":"Stateless","Msg":"HTTP New request","ID":"1d4373f6-6dde-4b96-8f09-3ffb472de389","Environ":"","User":"","Password":"","Client":"","Headers":"[Key: User-Agent, values: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/78.0.3904.108 Safari/537.36],[Key: Content-Length, values: 314],[Key: Content-Type, values: application/xml],[Key: Accept-Encoding, values: gzip],[Key: Connection, values: close],","Cookies":"","UserAgent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/78.0.3904.108 Safari/537.36","HostHTTPRequest":"1.128.0.16:80","Body":"\u003c!DOCTYPE xxe [\n\u003c!ELEMENT name ANY \u003e\n\u003c!ENTITY xxe SYSTEM \"file:///etc/passwd\"\u003e]\u003e\n\u003cAutodiscover xmlns=\"http://schemas.microsoft.com/exchange/autodiscover/outlook/responseschema/2006a\"\u003e\n\u003cRequest\u003e\n\u003cEMailAddress\u003eaaaaa\u003c/EMailAddress\u003e\n\u003cAcceptableResponseSchema\u003e\u0026xxe;\u003c/AcceptableResponseSchema\u003e\n\u003c/Request\u003e\n\u003c/Autodiscover\u003e","HTTPMethod":"POST","RequestURI":"/Autodiscover/Autodiscover.xml","Description":"Wordpress 6.0","SourceIp":"1.128.0.58","SourcePort":"41654"},"level":"info","msg":"New Event","status":"Stateless","time":"2025-02-19T07:48:34Z"}

0 commit comments

Comments
 (0)