You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
A reliable, production-ready Node.js server for processing Unthread.io webhooks with signature verification and smart platform handling. Built with TypeScript, Express.js, and Redis, this webhook server provides secure HMAC-SHA256 signature validation, intelligent event deduplication, and seamless integration with multiple platforms including Discord and Telegram. The server automatically detects event sources, processes various webhook events (conversations, messages, status updates), and efficiently queues them through Redis for downstream consumption by your bot applications, ensuring reliable and scalable webhook processing for your Unthread.io integrations.
5
+
A reliable, production-ready Node.js server for processing Unthread.io webhooks with signature verification and smart platform handling. Built with TypeScript, Express.js, and Redis, this webhook server provides secure HMAC-SHA256 signature validation, intelligent event deduplication, seamless integration with multiple platforms including Discord and Telegram, and **advanced file attachment correlation** that accurately detects source platforms for file uploads. The server automatically detects event sources, processes various webhook events (conversations, messages, status updates), correlates file attachments with their originating platforms, and efficiently queues them through Redis for downstream consumption by your bot applications, ensuring reliable and scalable webhook processing for your Unthread.io integrations.
6
6
7
7
## 🤗 Special Thanks
8
8
@@ -45,6 +45,30 @@ Server runs on `http://localhost:3000` with endpoints:
@@ -116,7 +140,18 @@ For local testing, use [ngrok](https://ngrok.com/): `ngrok http 3000`
116
140
2.**Security**: Validates HMAC-SHA256 signatures using your webhook secret
117
141
3.**Deduplication**: Prevents duplicate event processing with Redis TTL cache
118
142
4.**Platform Detection**: Identifies if events come from dashboard or target platform
119
-
5.**Queue Publishing**: Sends processed events to Redis `unthread-events` queue
143
+
5.**File Attachment Correlation**: Smart correlation system that links file attachments with their source platforms instead of marking them as "unknown"
144
+
6.**Queue Publishing**: Sends processed events to Redis `unthread-events` queue with enhanced attachment metadata
145
+
146
+
### 🎯 File Attachment Intelligence
147
+
148
+
This server features advanced file attachment correlation that:
149
+
150
+
-**Eliminates "Unknown" Sources**: Automatically correlates file upload events with their originating platform (Dashboard, Telegram, Discord, etc.)
151
+
-**Memory-Based Correlation**: Uses intelligent caching to match message events with subsequent file upload events
152
+
-**Rich Metadata Generation**: Provides comprehensive attachment summaries including file count, total size, MIME types, and file names
153
+
-**Multi-Event Buffering**: Handles multiple file attachments in a single conversation with timeout-based processing
154
+
-**Backwards Compatibility**: Existing integrations continue to work without modification
120
155
121
156
## 📊 Event Processing
122
157
@@ -128,7 +163,8 @@ For local testing, use [ngrok](https://ngrok.com/): `ngrok http 3000`
128
163
-`message_created` - New messages
129
164
130
165
### Redis Queue Format
131
-
Events are queued with this structure:
166
+
167
+
Events are queued with this enhanced structure:
132
168
133
169
```json
134
170
{
@@ -140,12 +176,34 @@ Events are queued with this structure:
140
176
"eventId": "evt_123456789",
141
177
"conversationId": "conv_abc123",
142
178
"content": "Hello from support!",
143
-
"eventTimestamp": 1733097600000
179
+
"eventTimestamp": 1733097600000,
180
+
"files": [
181
+
{
182
+
"id": "F123ABC456",
183
+
"name": "document.pdf",
184
+
"size": 524288,
185
+
"mimetype": "application/pdf"
186
+
}
187
+
]
188
+
},
189
+
"attachments": {
190
+
"hasFiles": true,
191
+
"fileCount": 1,
192
+
"totalSize": 524288,
193
+
"types": ["application/pdf"],
194
+
"names": ["document.pdf"]
144
195
},
145
196
"timestamp": 1733097600000
146
197
}
147
198
```
148
199
200
+
**New Enhancement**: Events with file attachments now include an `attachments` metadata object providing:
201
+
-`hasFiles`: Boolean indicating presence of files
202
+
-`fileCount`: Total number of attached files
203
+
-`totalSize`: Combined size of all files in bytes
204
+
-`types`: Array of unique MIME types (deduplicated)
205
+
-`names`: Array of all file names (maintains order)
206
+
149
207
## 💻 Development
150
208
151
209
### Build Commands
@@ -164,11 +222,21 @@ yarn start # Run production build
164
222
src/
165
223
├── app.ts # Main application entry
166
224
├── config/ # Configuration files
225
+
│ ├── env.ts # Environment validation
226
+
│ └── redis.ts # Redis configuration
167
227
├── controllers/ # Request handlers
228
+
│ └── webhookController.ts
168
229
├── middleware/ # Auth & validation
230
+
│ ├── auth.ts # HMAC signature verification
231
+
│ └── validation.ts # Request validation
169
232
├── services/ # Business logic
233
+
│ ├── redisService.ts # Redis operations
234
+
│ └── webhookService.ts # Webhook processing
170
235
├── types/ # TypeScript types
236
+
│ └── index.ts # All type definitions
171
237
└── utils/ # Helper functions
238
+
├── signature.ts # HMAC utilities
239
+
└── fileAttachmentCorrelation.ts # File correlation system
- Events may be classified as "unknown" for edge cases
211
281
282
+
**File Attachment Correlation Issues:**
283
+
284
+
- Verify `TARGET_PLATFORM` is set correctly in your `.env` file
285
+
- Check correlation logs for timing and buffering details
286
+
- File events without correlation data will fall back to "unknown" source
287
+
- Correlation window is 15 seconds - events outside this window may not correlate
288
+
289
+
**Common Solutions:**
290
+
291
+
- Restart the server if platform detection seems inconsistent
292
+
- Clear Redis cache if experiencing correlation issues: `redis-cli FLUSHDB`
293
+
- Enable debug logging by setting `NODE_ENV=development` for detailed correlation logs
294
+
295
+
## 🚀 Integration Benefits
296
+
297
+
### For Bot Developers
298
+
299
+
-**Accurate Source Detection**: No more "unknown" file attachment sources - get precise platform identification
300
+
-**Rich File Metadata**: Access file counts, sizes, types, and names without parsing complex file arrays
301
+
-**Simplified Integration**: Use the `attachments` metadata for quick file handling logic
302
+
-**Backwards Compatibility**: Existing code continues to work unchanged
303
+
304
+
### For Production Systems
305
+
306
+
-**Reliable Correlation**: Memory-based correlation system with 15-second timing windows and automatic fallbacks
307
+
-**Robust Error Handling**: Comprehensive timeout management and duplicate prevention
308
+
-**Scalable Architecture**: Efficient Redis-based queuing with TTL cleanup and deduplication
309
+
-**Production-Ready**: Extensive logging, monitoring, and error recovery mechanisms
310
+
212
311
## 🎯 Contributing
213
312
214
313
Contributions are welcome, create a pull request to this repo and I will review your code. Please consider to submit your pull request to the `dev` branch. Thank you!
215
314
216
315
When contributing, please ensure your code follows the existing TypeScript patterns and includes appropriate error handling.
0 commit comments