Skip to content
This repository was archived by the owner on Jul 22, 2025. It is now read-only.

FEATURE: allow llm triage to automatically hide posts #820

Merged
merged 3 commits into from
Oct 4, 2024
Merged

FEATURE: allow llm triage to automatically hide posts #820

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
da10ac5
FEATURE: allow llm triage to automatically hide posts
SamSaffron Oct 3, 2024
2cfdf07
Shift to using PostActionCreator, this ensures better admin
SamSaffron Oct 4, 2024
3055fdb
lint
SamSaffron Oct 4, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions config/locales/client.en.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -99,6 +99,9 @@ en:
hide_topic:
label: "Hide topic"
description: "Make topic non visible to the public if triggered"
hide_post:
label: "Hide post"
description: "Make post non visible to the public if triggered"
flag_post:
label: "Send to review"
description: "Puts the post into the review queue if triggered, for moderators to triage"
Expand Down
3 changes: 3 additions & 0 deletions discourse_automation/llm_triage.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@
field :category, component: :category
field :tags, component: :tags
field :hide_topic, component: :boolean
field :hide_post, component: :boolean
field :flag_post, component: :boolean
field :canned_reply, component: :message
field :canned_reply_user, component: :user
Expand All @@ -40,6 +41,7 @@
category_id = fields.dig("category", "value")
tags = fields.dig("tags", "value")
hide_topic = fields.dig("hide_topic", "value")
hide_post = fields.dig("hide_post", "value")
flag_post = fields.dig("flag_post", "value")

begin
Expand Down Expand Up @@ -67,6 +69,7 @@
canned_reply: canned_reply,
canned_reply_user: canned_reply_user,
hide_topic: hide_topic,
hide_post: hide_post,
flag_post: flag_post,
automation: self.automation,
)
Expand Down
22 changes: 21 additions & 1 deletion lib/automation/llm_triage.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,11 +13,12 @@ def self.handle(
canned_reply: nil,
canned_reply_user: nil,
hide_topic: nil,
hide_post: nil,
flag_post: nil,
automation: nil
)
if category_id.blank? && tags.blank? && canned_reply.blank? && hide_topic.blank? &&
flag_post.blank?
flag_post.blank? && hide_post.blank?
raise ArgumentError, "llm_triage: no action specified!"
end

Expand Down Expand Up @@ -82,6 +83,25 @@ def self.handle(
force_review: true,
)
end

# note this does not notify user of the action
# as it is mainly used for spam where notification
# is not needed.
# we will need another flag if we want to notify
if hide_post
Copy link
Contributor

@martin-brennan martin-brennan Oct 3, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I feel like you could lean on post.hide! in core here https://github.com/discourse/discourse/blob/c446caa6e9f68be24bdc3d0ff0b33e0ddc2130c6/app/models/post.rb#L604-L604 since it's doing the same thing the code does here, and give it PostActionType.types[:spam] as the post_action_type_id. I think the only thing you would need to do is your additional topic visibility update.

Copy link
Contributor

@martin-brennan martin-brennan Oct 3, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also generally I am not sure this with the flag_post/needs_review part above is quite the correct way to go, seems like going through the PostActionCreator and indicating that it's for spam :

PostActionCreator.new(
  Discourse.system_user,
  post,
  PostActionType.types[:spam],
  message: message,
  queue_for_review: true,
).perform

This auto-hides if necessary and also does the needs_review! on the flagged post. If we did this the hide_topic and hide_post options might not be necessary for the automation.

It feels like we are not relying on the review queue logic to properly mark this as spam, and since this is the case if you mark the reviewable invalid / disagree the post might not get unhidden as you already indicated, and there is no testing for this.

The review queue logic is super hard to follow though so I understand the appeal of doing it in a more basic way here, I just think it may cause other problems doing it this way. This may solve the customer's immediate problem but I am not sure it's maintanable. Maybe we can do some further digging first and try out PostActionCreator before merging this?

Copy link
Contributor

@martin-brennan martin-brennan Oct 3, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Might also be worth having 2 options:

  1. Send to review - Leave this as is, just ask for post approval
  2. Mark as spam - Follow the spam flow with PostActionCreator, which would also do the hiding

And getting rid of hide_post and hide_topic options

Copy link
Member Author

@SamSaffron SamSaffron Oct 3, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

its tricky cause hide_topic is used for automations that do not flag, we can not get rid of it. But I can try to see if I can get PostActionCreator going and allow people to select "send to review/mark as spam/don't involve review" ... its so complicated :(

Copy link
Member

@xfalcox xfalcox Oct 3, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Worth mentioning that back on the old toxicity / nsfw days we used PostActionCreator and had problems because people don't always want the users to be aware that they were flagged / auto-hidden, which happens on both public topics and PMs.

This is the main reason we now only send to the queue without all the usual flag adjacent side effects. But that also means that if people now want it, it needs to be an option much like what @martin-brennan described.

reason ||=
(
if post.hidden_at
Post.hidden_reasons[:flag_threshold_reached_again]
else
Post.hidden_reasons[:flag_threshold_reached]
end
)

post.update!(hidden: true, hidden_at: Time.zone.now, hidden_reason_id: reason)

post.topic.update!(visible: false) if post.post_number == 1
Copy link
Contributor

@martin-brennan martin-brennan Oct 3, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should use topic.update_status for this, example here:

https://github.com/discourse/discourse/blob/c446caa6e9f68be24bdc3d0ff0b33e0ddc2130c6/app/models/post.rb#L628-L633

end
end
end
end
Expand Down
33 changes: 33 additions & 0 deletions spec/lib/modules/automation/llm_triage_spec.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
# frozen_string_literal: true
describe DiscourseAi::Automation::LlmTriage do
fab!(:post)
fab!(:reply) { Fabricate(:post, topic: post.topic, user: Fabricate(:user)) }
fab!(:llm_model)

def triage(**args)
Expand Down Expand Up @@ -37,6 +38,38 @@ def triage(**args)
expect(post.topic.reload.visible).to eq(false)
end

context "when hiding posts on triage" do
it "can hide posts if they are replies" do
DiscourseAi::Completions::Llm.with_prepared_responses(["bad"]) do
triage(
post: reply,
model: "custom:#{llm_model.id}",
hide_post: true,
system_prompt: "test %%POST%%",
search_for_text: "bad",
automation: nil,
)
end

expect(reply.reload.hidden).to eq(true)
end

it "can hide topic if the post is the first post" do
DiscourseAi::Completions::Llm.with_prepared_responses(["bad"]) do
triage(
post: post,
model: "custom:#{llm_model.id}",
hide_post: true,
system_prompt: "test %%POST%%",
search_for_text: "bad",
automation: nil,
)
end

expect(post.reload.topic.visible).to eq(false)
end
end

it "can categorize topics on triage" do
category = Fabricate(:category)

Expand Down