Skip to content
This repository was archived by the owner on Jul 22, 2025. It is now read-only.

Conversation

@SamSaffron
Copy link
Member

@SamSaffron SamSaffron commented Dec 5, 2024

This PR introduces an AI-powered spam detection system that helps protect communities from spam by automatically scanning posts from new users.

Key Features:

  • Automatic scanning of the first 3 posts by new users (TL0-TL1)
  • Smart edit detection to catch spam modifications
  • Admin UI for configuration and monitoring
  • Integration with existing spam handling mechanisms
  • Statistics tracking for accuracy monitoring
  • Custom instructions support for site-specific rules

Technical Implementation:

  • New AiSpamLog and AiModerationSetting models for tracking and configuration
  • Post-creation and post-edit hooks for automated scanning
  • Intelligent rate limiting and edit detection
  • Custom LLM prompt engineering for spam detection
  • Integration with Discourse's existing spam handling system

Admin Features:

  • Enable/disable spam detection
  • Select LLM model for scanning
  • Configure custom site-specific instructions
  • Monitor detection statistics (scanned posts, detected spam, false positives/negatives)

Safety Features:

  • Only scans public posts
  • Limited to first 3 posts by new users
  • Minimum edit difference threshold
  • Rate limiting for rescanning edited posts
  • Integration with existing trust system

image

@SamSaffron SamSaffron marked this pull request as ready for review December 9, 2024 06:27
@SamSaffron
Copy link
Member Author

Left to do:

  1. Some way to drill into the 4 lists (false positive / negative / scanned / spam)
  2. Send images through

Except for that, this is pretty much ready

@SamSaffron SamSaffron changed the title WIP: new spam tab for Discourse AI FEATURE: Add AI-powered spam detection for new user posts Dec 9, 2024
Copy link
Member

@keegangeorge keegangeorge left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Follow up changes look good, and overall everything looks good while testing locally. A few 🤏🏽 tiny things

@SamSaffron SamSaffron merged commit 47f5da7 into main Dec 11, 2024
6 checks passed
@SamSaffron SamSaffron deleted the add-spam-tab branch December 11, 2024 22:17
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Development

Successfully merging this pull request may close these issues.

6 participants