Improved moderation tooling #218

Pdzly · 2025-12-28T18:33:25Z

Added warnings, reputation, scam link detection and some more.

- Add spam detection mechanism for message similarity and frequency - Integrate warning escalation and expiration handling - Add modules for raid and mention spam detection - Enhance account analysis with suspicious patterns and actions - Update database models for threat logging and user reputation tracking

…tion actions

Copilot

Pull request overview

This PR adds comprehensive moderation tooling to the Discord bot, including a warnings system, reputation tracking, scam link detection, and automated threat detection capabilities.

Key Changes:

Implemented a formal warnings system with severity levels, expiration, and pardon capabilities
Added a reputation system that tracks user behavior and affects XP gain
Created threat detection modules for scam links, spam, mention spam, toxic content, raid detection, and suspicious accounts
Added database models for warnings, reputation events, threat logs, scam domains, and blocked words

Reviewed changes

Copilot reviewed 32 out of 32 changed files in this pull request and generated 20 comments.

Show a summary per file

File	Description
src/store/storage.ts	Registered new database models and updated parseInt to Number.parseInt
src/store/models/Warning.ts	New model for tracking user warnings with severity, expiration, and pardon support
src/store/models/ThreatLog.ts	New model for logging detected threats and automated moderation actions
src/store/models/ScamDomain.ts	New model for storing known scam domains
src/store/models/ReputationEvent.ts	New model for tracking reputation changes with score calculations
src/store/models/DDUser.ts	Added reputation score and last update timestamp fields
src/store/models/BlockedWord.ts	New model for managing blocked words with categories
src/modules/xp/xpForMessage.util.ts	Integrated reputation system to modify XP gains based on user tier
src/modules/threatDetection/utils/textNormalization.ts	Text normalization utilities for detecting bypass attempts in toxic content
src/modules/threatDetection/threatDetection.module.ts	Module definition registering threat detection listeners
src/modules/threatDetection/logs.ts	Logging system for threat detection events
src/modules/threatDetection/listeners/messageAnalysis.listener.ts	Message listener analyzing for scams, spam, and toxic content
src/modules/threatDetection/listeners/memberJoin.listener.ts	Member join listener for raid and suspicious account detection
src/modules/threatDetection/detectors/toxicContentDetector.ts	Detector for toxic content using normalized text matching
src/modules/threatDetection/detectors/spamDetector.ts	Spam detection using message rate and similarity analysis
src/modules/threatDetection/detectors/scamLinkDetector.ts	Scam link detection using patterns, database, and external API
src/modules/threatDetection/detectors/raidDetector.ts	Raid detection tracking rapid member joins
src/modules/threatDetection/detectors/mentionSpamDetector.ts	Detection of mass mention spam
src/modules/threatDetection/detectors/accountAnalyzer.ts	Analysis of suspicious account patterns
src/modules/starboard/starboard.listener.ts	Grants reputation when messages reach starboard
src/modules/moderation/wordlist.command.ts	Command for managing the blocked words list
src/modules/moderation/warnings.command.ts	Command for viewing user warnings
src/modules/moderation/warningScheduler.listener.ts	Scheduled job for expiring warnings and checking escalation thresholds
src/modules/moderation/warn.command.ts	Command for issuing warnings to users
src/modules/moderation/reputation.service.ts	Service layer for reputation management and tier calculations
src/modules/moderation/reputation.command.ts	Commands for viewing and granting reputation
src/modules/moderation/pardon.command.ts	Command for pardoning (removing) warnings
src/modules/moderation/moderation.module.ts	Updated module to register new commands and listeners
src/modules/moderation/logs.ts	Added log types for warnings, pardons, and reputation
src/index.ts	Registered ThreatDetectionModule
src/config.type.ts	Added configuration interfaces for threat detection and reputation systems
src/Config.ts	Added threat detection configuration for development environment

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-01-06T20:16:18Z

src/modules/threatDetection/detectors/mentionSpamDetector.ts

+	}>;
+}
+
+const userMentionCache = new ExpiryMap<string, UserMentionWindow>(60_000);


The same ExpiryMap issue exists here as in spamDetector - the cache is set to 60 seconds but the configured window could be larger. The userMentionCache expiry time should match or exceed the configured windowSeconds to ensure accurate mention tracking across the entire detection window.

Suggested change

const userMentionCache = new ExpiryMap<string, UserMentionWindow>(60_000);

const userMentionCache = new ExpiryMap<string, UserMentionWindow>(24 * 60 * 60_000);

Copilot · 2026-01-06T20:16:19Z

src/store/storage.ts

 			password,
 			host,
-			port: parseInt(port, 10),
+			port: Number.parseInt(port, 10),


The parseInt function has been changed to Number.parseInt, which is correct. However, this change is a style improvement rather than a functional fix, and similar patterns elsewhere in the codebase should also be updated for consistency. Consider applying this change project-wide if it's a new standard being adopted.

Copilot · 2026-01-06T20:16:19Z

src/modules/threatDetection/utils/textNormalization.ts

+export function normalizeText(text: string): string {
+	let normalized = text.toLowerCase();
+	normalized = removeInvisibleChars(normalized);
+	normalized = removeZalgo(normalized);
+	normalized = normalizeHomoglyphs(normalized);
+	normalized = decodeLeetspeak(normalized);
+	normalized = removeExcessiveRepeats(normalized);
+	return normalized;
+}


The normalizeText function does not return text with separators removed, only the normalized version. This means containsBlockedWord will check both the normalized text and the text without separators, but normalizeText itself doesn't include separator removal. Consider whether normalizeText should include removeSeparators as part of its normalization pipeline, or ensure the function name and documentation accurately reflect what it does.

Copilot · 2026-01-06T20:16:19Z

src/modules/threatDetection/listeners/messageAnalysis.listener.ts

+				config.threatDetection?.mentionSpam?.windowSeconds ?? 60;
+			try {
+				await member.timeout(
+					muteDuration * 1000 * 5,


The muteDuration configuration value is used inconsistently. In the spam detection handler, it's expected to be in milliseconds (line 115), but in the mention spam handler (line 168), it's treated as seconds and multiplied by 1000 * 5. This inconsistency will cause incorrect timeout durations. The configuration should specify the unit clearly, and the usage should be consistent.

Suggested change

config.threatDetection?.mentionSpam?.windowSeconds ?? 60;

try {

await member.timeout(

muteDuration * 1000 * 5,

config.threatDetection?.muteDuration ?? 60_000;

try {

await member.timeout(

muteDuration,

Copilot · 2026-01-06T20:16:20Z

src/modules/threatDetection/detectors/spamDetector.ts

+	}>;
+}
+
+const userMessageCache = new ExpiryMap<string, UserMessageWindow>(60_000);


The ExpiryMap for userMessageCache is set to expire entries after 60 seconds, but the spam detection window could be configured to be larger than 60 seconds via config.threatDetection.spam.windowSeconds. This creates a mismatch where older messages needed for spam detection might be prematurely evicted from the cache, leading to inaccurate spam detection. The cache expiry should be at least as long as the maximum configured window duration.

Copilot · 2026-01-06T20:16:23Z

src/modules/threatDetection/detectors/toxicContentDetector.ts

+
+	if (result.found && result.word) {
+		const matchedBlockedWord = blockedWords.find(
+			(w) => w.word === result.word?.toLowerCase(),


The comparison on line 93 is checking if the result.word (which could be the original blocked word) matches the database record's word field. However, since blocked words are stored in lowercase (normalized in addBlockedWord), the comparison should be case-insensitive or the result.word should be normalized. The current logic may miss matches if result.word contains uppercase letters.

Suggested change

(w) => w.word === result.word?.toLowerCase(),

(w) => w.word.toLowerCase() === result.word.toLowerCase(),

Copilot · 2026-01-06T20:16:23Z

src/modules/threatDetection/detectors/raidDetector.ts

+const RAID_MODE_DURATION_MS = 5 * 60 * 1000;
+


The RAID_MODE_DURATION_MS constant is hardcoded to 5 minutes, but this should be configurable via the config system to allow different servers to adjust raid protection duration based on their needs. Consider adding this to the ThreatDetectionConfig interface and reading it from the configuration.

Suggested change

const RAID_MODE_DURATION_MS = 5 * 60 * 1000;

let RAID_MODE_DURATION_MS = 5 * 60 * 1000;

export function setRaidModeDurationMs(durationMs: number): void {

RAID_MODE_DURATION_MS = durationMs;

}

Copilot · 2026-01-06T20:16:23Z

src/modules/moderation/logs.ts


 	const targetUser = await client.users.fetch(action.target).catch(() => null);
-	let description = `**Offender**: ${targetUser && fakeMention(targetUser)} ${actualMention(action.target)}\n`;
+	const targetLabel = action.kind === "ReputationGranted" ? "Recipient" : "Offender";


There's inconsistent terminology in the moderation logs. Line 194-195 uses "Recipient" for reputation grants but "Offender" for other moderation actions. For warnings and other moderation actions, "Target" or "Subject" might be more neutral terms than "Offender", as some actions (like pardons) aren't punitive.

Suggested change

const targetLabel = action.kind === "ReputationGranted" ? "Recipient" : "Offender";

const targetLabel = action.kind === "ReputationGranted" ? "Recipient" : "Target";

Copilot · 2026-01-06T20:16:24Z

src/modules/threatDetection/detectors/accountAnalyzer.ts

+const DANGEROUS_PATTERN_REGEX = /(\+|\*|\{[0-9]+,\})\s*\1|\(\?[^)]*\+/;
+
+function isPatternSafe(pattern: string): boolean {
+	// Reject overly long patterns
+	if (pattern.length > MAX_PATTERN_LENGTH) {
+		return false;
+	}
+	// Reject patterns with nested quantifiers (e.g., (a+)+ or (a*)*)
+	if (DANGEROUS_PATTERN_REGEX.test(pattern)) {
+		return false;
+	}
+	return true;
+}


There's a potential ReDoS (Regular Expression Denial of Service) vulnerability. While the code attempts to detect dangerous patterns with DANGEROUS_PATTERN_REGEX, the regex itself contains potentially problematic patterns. Additionally, the detection logic may not catch all ReDoS patterns. Consider using a more robust ReDoS detection library or implementing a timeout mechanism for regex execution.

Copilot · 2026-01-06T20:16:24Z

src/modules/moderation/reputation.command.ts

+	getReputationTier,
+	getUserReputation,
+	grantReputation,
+	REPUTATION_TIER_COLORS,
+	REPUTATION_TIER_LABELS,
+	REPUTATION_TIER_THRESHOLDS,
+	ReputationTier,
+	updateReputation,


Unused imports REPUTATION_TIER_COLORS, getReputationTier, updateReputation.

Suggested change

getReputationTier,

getUserReputation,

grantReputation,

REPUTATION_TIER_COLORS,

REPUTATION_TIER_LABELS,

REPUTATION_TIER_THRESHOLDS,

ReputationTier,

updateReputation,

getUserReputation,

grantReputation,

REPUTATION_TIER_LABELS,

REPUTATION_TIER_THRESHOLDS,

ReputationTier,

bristermitten · 2026-01-06T20:25:53Z

this is a very big change so kinda scary to merge all at once but what the hell YOLO

Pdzly added 2 commits December 28, 2025 19:27

- Adjust offender label to dynamically support "Recipient" for reputa…

9384be5

…tion actions

Pdzly requested a review from bristermitten December 28, 2025 18:33

bristermitten requested a review from Copilot January 6, 2026 20:10

Copilot started reviewing on behalf of bristermitten January 6, 2026 20:10 View session

Copilot AI reviewed Jan 6, 2026

View reviewed changes

Merge branch 'master' into feature/improve-moderation

e8face1

fix missing commas

cc72ec6

bristermitten merged commit f23d807 into TheDeveloperDen:master Jan 6, 2026
8 checks passed

	const userMentionCache = new ExpiryMap<string, UserMentionWindow>(60_000);
	const userMentionCache = new ExpiryMap<string, UserMentionWindow>(24 * 60 * 60_000);

	(w) => w.word === result.word?.toLowerCase(),
	(w) => w.word.toLowerCase() === result.word.toLowerCase(),

-const RAID_MODE_DURATION_MS = 5 * 60 * 1000;
+let RAID_MODE_DURATION_MS = 5 * 60 * 1000;
+export function setRaidModeDurationMs(durationMs: number): void {
+	RAID_MODE_DURATION_MS = durationMs;
+}

	const targetLabel = action.kind === "ReputationGranted" ? "Recipient" : "Offender";
	const targetLabel = action.kind === "ReputationGranted" ? "Recipient" : "Target";

Improved moderation tooling #218

Improved moderation tooling #218

Uh oh!

Conversation

Pdzly commented Dec 28, 2025

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI Jan 6, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 6, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 6, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 6, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 6, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 6, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 6, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 6, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 6, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 6, 2026

Choose a reason for hiding this comment

Uh oh!

bristermitten commented Jan 6, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants