Skip to content
This repository was archived by the owner on Jul 22, 2025. It is now read-only.

Commit fe44d78

Browse files
authored
DEV: Expose AI spam scanning metrics (#1077)
This should give us a better idea on how our scanner is faring across sites. ``` # HELP discourse_discourse_ai_spam_detection AI spam scanning statistics # TYPE discourse_discourse_ai_spam_detection counter discourse_discourse_ai_spam_detection{db="default",type="scanned"} 16 discourse_discourse_ai_spam_detection{db="default",type="is_spam"} 7 discourse_discourse_ai_spam_detection{db="default",type="false_positive"} 1 discourse_discourse_ai_spam_detection{db="default",type="false_negative"} 2 ```
1 parent ad7bb9b commit fe44d78

File tree

4 files changed

+42
-0
lines changed

4 files changed

+42
-0
lines changed

.rubocop.yml

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2,3 +2,6 @@ inherit_gem:
22
rubocop-discourse: stree-compat.yml
33
RSpec/NamedSubject:
44
Enabled: false
5+
6+
Style/GlobalVars:
7+
AllowedVariables: [$prometheus_client]

about.json

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
{
2+
"tests": {
3+
"requiredPlugins": [
4+
"https://github.com/discourse/discourse-prometheus"
5+
]
6+
}
7+
}

lib/ai_moderation/spam_metric.rb

Lines changed: 29 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,29 @@
1+
# frozen_string_literal: true
2+
3+
module DiscourseAi
4+
module AiModeration
5+
class SpamMetric
6+
def self.update(new_status, reviewable)
7+
ai_spam_log = AiSpamLog.find_by(reviewable:)
8+
return if ai_spam_log.nil?
9+
10+
increment("scanned")
11+
increment("is_spam") if new_status == :approved && ai_spam_log.is_spam
12+
increment("false_positive") if new_status == :rejected && ai_spam_log.is_spam
13+
increment("false_negative") if new_status == :rejected && !ai_spam_log.is_spam
14+
end
15+
16+
private
17+
18+
def self.increment(type, value = 1)
19+
metric = ::DiscoursePrometheus::InternalMetric::Custom.new
20+
metric.name = "discourse_ai_spam_detection"
21+
metric.type = "Counter"
22+
metric.description = "AI spam scanning statistics"
23+
metric.labels = { db: RailsMultisite::ConnectionManagement.current_db, type: }
24+
metric.value = value
25+
$prometheus_client.send_json(metric.to_h)
26+
end
27+
end
28+
end
29+
end

plugin.rb

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -84,6 +84,9 @@ def self.public_asset_path(name)
8484

8585
on(:reviewable_transitioned_to) do |new_status, reviewable|
8686
ModelAccuracy.adjust_model_accuracy(new_status, reviewable)
87+
if DiscourseAi::AiModeration::SpamScanner.enabled?
88+
DiscourseAi::AiModeration::SpamMetric.update(new_status, reviewable)
89+
end
8790
end
8891

8992
if Rails.env.test?

0 commit comments

Comments
 (0)