Skip to content

Commit 0c7ed97

Browse files
taylorcopilotClawdbot
taylorcopilot
and
Clawdbot
authored
Docs: explain CoPilot ↔ Shuffle integration (#733)
* docs: add docs.socfortress.co link to README * docs: add CoPilot-Shuffle integration page --------- Co-authored-by: Clawdbot <clawdbot@Clawdbots-Mac-mini.local>
1 parent 0d3d7df commit 0c7ed97

File tree

2 files changed

+157
-0
lines changed

2 files changed

+157
-0
lines changed

‎docs/user/shuffle-integration.md‎

Lines changed: 156 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,156 @@
1+
# CoPilot ↔ Shuffle Integration (Admin/Operator)
2+
3+
CoPilot integrates with **Shuffle** so you can attach **automation and external notifications** (Teams/Slack/Jira/Email/VT enrichment, etc.) to alerts and cases **without** SOCFortress having to ship a bespoke integration for every downstream tool.
4+
5+
At a high level:
6+
7+
- **Graylog** creates the *SIEM event/alert* (via Event Definitions)
8+
- **CoPilot** polls Graylog for *new, unprocessed* events and turns them into **CoPilot Alerts/Cases**
9+
- **CoPilot** sends a structured payload to **Shuffle**, which runs a workflow you choose
10+
11+
---
12+
13+
## Why we do it this way
14+
15+
### 1) Scale integrations without bloating CoPilot
16+
CoPilot is the “hub” for the stack, but the long tail of integrations (Teams, Jira, PagerDuty, ServiceNow, custom webhooks, etc.) is endless.
17+
18+
Shuffle is purpose-built for this:
19+
20+
- hundreds of apps/connectors
21+
- easy HTTP/webhook support for anything else
22+
- workflows/playbooks can be owned by the operator team (not necessarily product engineering)
23+
24+
### 2) Keep automation close to your environment (minimize exposure)
25+
Shuffle supports an architecture where the **UI/orchestration can be cloud-hosted** while execution happens via an **on‑prem “location/worker”** inside your network.
26+
27+
That’s valuable for SIEM stacks because you often want to:
28+
29+
- reach internal services (Wazuh Manager, Indexer/OpenSearch, AD, internal APIs)
30+
- avoid poking inbound firewall holes
31+
- keep credentials and execution inside your boundary
32+
33+
### 3) Strong separation of concerns
34+
- **Graylog**: detection logic (queries, thresholds, correlation)
35+
- **CoPilot**: multi-tenant alert/case lifecycle + normalization + UI + API
36+
- **Shuffle**: automation/orchestration + outbound integrations
37+
38+
---
39+
40+
## Data flow (what triggers what)
41+
42+
### Alerts (automatic)
43+
1. A Graylog **Event Definition** triggers.
44+
2. Graylog writes an event into its events index (e.g., `gl-events_*`).
45+
3. CoPilot’s scheduler job **polls** for new events where `copilot_alert_id = none`.
46+
4. CoPilot creates a **CoPilot Alert** and then updates the Graylog event so it won’t be reprocessed.
47+
5. If a Shuffle notification workflow is configured for that customer, CoPilot invokes it and includes:
48+
- alert/case metadata
49+
- customer code
50+
- asset identifier
51+
- **alert context** (see below)
52+
53+
### Cases (manual trigger)
54+
Cases can be triggered into Shuffle **manually** from the CoPilot UI. This is intentional so case automation isn’t accidentally run without an analyst decision.
55+
56+
---
57+
58+
## What CoPilot sends to Shuffle (payload model)
59+
60+
CoPilot sends a JSON payload that includes:
61+
62+
- **Alert ID / Case ID** (CoPilot identifiers)
63+
- **Customer code** (multi-tenant routing)
64+
- **Asset name** (e.g., Wazuh agent name, O365 mailbox/user, etc.)
65+
- **Alert Context**: a small, curated set of fields you choose per Source
66+
67+
### Alert Context comes from *Sources*
68+
The most important concept for operators/admins:
69+
70+
> **The fields that appear in Shuffle’s `alert_context` are defined by the CoPilot Source configuration.**
71+
72+
That’s why CoPilot can support wildly different log types:
73+
74+
- Wazuh endpoint events → asset might be `agent.name`
75+
- O365 events → asset might be `o365.recipients` or `user_id`
76+
77+
You choose which fields are “headline context” for that Source, and those same fields become the default “automation payload” for Shuffle.
78+
79+
---
80+
81+
## Setup: enabling CoPilot → Shuffle
82+
83+
### Step 1 — Get a Shuffle API key
84+
In Shuffle:
85+
86+
1. Go to **User Account / Settings**
87+
2. Copy the **API key**
88+
89+
### Step 2 — Add the Shuffle API key to CoPilot
90+
In CoPilot:
91+
92+
1. Navigate to **Connectors**
93+
2. Locate **Shuffle**
94+
3. Paste the API key and **verify**
95+
96+
### Step 3 — Create a Shuffle workflow to receive CoPilot notifications
97+
Create a workflow in Shuffle (example: `SIEM Alert — <customer>`). The workflow will receive the CoPilot payload as input.
98+
99+
### Step 4 — Set the notification workflow per customer
100+
In CoPilot:
101+
102+
1. Open the **Customer**
103+
2. Go to **Notification Workflow**
104+
3. Create/configure the notification and set the **Shuffle Workflow ID**
105+
106+
**Result:**
107+
- New alerts for that customer can invoke the workflow automatically
108+
- Cases can invoke the workflow when manually triggered
109+
110+
---
111+
112+
## How this ties back to Graylog + Indexer/OpenSearch
113+
114+
A quick mental model (this matters when debugging):
115+
116+
- **Indexer/OpenSearch (Wazuh Indexer)** stores the actual event data.
117+
- **Graylog** manages streams/index sets and runs event definitions (alerting) by querying the indexer.
118+
- **CoPilot** reads Graylog’s generated events and then enriches/normalizes them into CoPilot alerts.
119+
120+
So when you want to automate enrichment in Shuffle (e.g., look up related events, run searches, pull agent state):
121+
122+
- Query **Indexer/OpenSearch** for raw events/search
123+
- Use **Wazuh Manager API** for agent lifecycle state
124+
- Use **CoPilot API** when you want CoPilot’s normalized view (alerts/cases/customers)
125+
126+
---
127+
128+
## Practical examples (common workflows)
129+
130+
- Post to **Microsoft Teams** when a High severity alert arrives
131+
- Enrich hashes/domains with **VirusTotal** before an analyst sees the alert
132+
- Create a **Jira** ticket with alert context and links back to CoPilot
133+
- Invoke an internal **FastAPI microservice** (e.g., disable AD user) with an approval step
134+
135+
---
136+
137+
## Troubleshooting checklist
138+
139+
1. **Is the Graylog Event Definition firing?**
140+
- Confirm the event exists in `gl-events_*`.
141+
2. **Is CoPilot creating the alert?**
142+
- If `copilot_alert_id` remains `none`, CoPilot likely can’t map the event to a Source.
143+
3. **Is the Source configured correctly?**
144+
- Ensure the Source matches the event’s `syslog_type` and correct field mapping.
145+
4. **Is the Shuffle connector verified in CoPilot?**
146+
5. **Is the customer’s Shuffle Workflow ID set?**
147+
6. **Check Shuffle runs** for the workflow.
148+
149+
---
150+
151+
## Related docs
152+
153+
- [Incident Sources](ui/incident-sources.md)
154+
- [Alerts (SIEM)](ui/alerts-siem.md)
155+
- [Connectors](ui/connectors.md)
156+
- [External Services](ui/external-services.md)

‎mkdocs.yml‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -77,6 +77,7 @@ nav:
7777
- Quickstart (Admins/Engineers): user/admins-quickstart.md
7878
- Customer Provisioning (Tenancy): user/customer-provisioning.md
7979
- Features by Area: user/features.md
80+
- CoPilot ↔ Shuffle Integration (Admin/Operator): user/shuffle-integration.md
8081
- Navigation Guide (UI): user/navigation.md
8182
- UI Guide (mirrors menu):
8283
- Start here: user/ui/README.md

0 commit comments

Comments
 (0)