[Feature Proposal] New Users Anti Spammer System #39191
Description
Problem Statement
Currently, managing new users and preventing spam requires heavy manual moderation by Moderators and Admin. Standard rule-based spam filters are not good enough to be deployed at scale as they heavily suffer with false positives (penalizing legitimate users like fast typists), immediate ban-evasion by bad actors, and frustrating user experiences when innocent users trigger a blanket mute.
Proposed Architecture Overview
I am proposing an automated, hybrid Node.js service that monitors the first 6 to 10 weeks of user activity to detect spam patterns.
- Event Monitoring & Data Ingestion
To ensure real-time monitoring without degrading chat performance or latency for active users, I propose to implement non-blocking event listeners on core Rocket.Chat lifecycle hooks including afterSaveMessage, afterCreateUser, and channel join events. The proposed system shall compile behavioral metadata such as keystroke frequency, message velocity, URL-to-text ratios, and mass-pinging attempts, crafting the data-rich continuous dataset for the analysis engine to process asynchronously. - Hybrid Detection Engine
This system deploys a two-tiered processing system to optimize server compute resources. The first tier is a lightweight, rule-based filter that catches known malicious IP addresses and blacklisted domains instantly. The second tier is a machine learning behavioral model that analyzes semantic message intent and sequential anomalies, such as a user joining multiple channels and pasting identical text blocks within a narrow time window. This saves heavier algorithmic processing for advanced and difficult spam patterns.
Moreover, as per my previous experience with similar projects I suggest the following to strengthen system integrity and security:
Dynamic Risk Score Profiling
Instead of employing a static scoring profile that permanently penalizes users. We can utilize organic interactions like receiving replies or emoji reactions to dynamically model user profile which will calibrate the score dynamically and reduce threat score over time if genuine and legitimate activities are observed. This way we reduce false positives and improve newcomers who might have received early warnings.
Granular Action Blocking
This system can employ phase-wise restrictions where high-risk actions such as rendering URL previews or broadcasting mass mentions are restricted first before completely blocking plain-text chat. This neutralizes the primary tools used by spammers like Puppeteer, Selenium, or custom API scripts without completely silencing genuine users who trigger a false alarm.
Shadowbanning Protocol
For advanced spam patterns, the system will place the user in a silent drop state. Their messages will appear successful on their local interface but remain completely hidden from the community. This wastes the time and efforts of malicious actors without alerting them to create a new account, effectively containing the threat invisibly.
Automated Administrative Reporting
To eliminate manual exploration through logs, the system will rely on scheduled cron jobs to generate daily reports for human moderators. These reports will outline flagged users, their risk scores, and the automated actions taken by the system, giving moderators a clear, data driven report of system health and prevented attacks.
Additional Context and Some Info About Me
I am Srijan Tripathi, a second-year undergrad pursuing a B.Tech in CSE with Honors in AI/ML. I am well-versed in the MERN stack (MongoDB, Express, React, Node.js) and Next.js, and have previously contributed to open-source React projects like OHC.
Having my focus on AI/ML and my full-stack development experience, I am highly interested in building out this security feature. I would love to have the team's feedback on this architectural approach and if we enhance this further.