Cascading rules

[cassidyjames]

This is a recommended roadmap item for Coop, which is based on a conversation I had with @julietshen in the coop-internal channel on Discord. It could be useful for Coop to implement "cascading" rules. By that I mean, instead of a rule attempting to match against every media item that stream through the system, the rule could instead be configured to run as follow-up, contingent on an earlier rule being triggered first.

I have the following use-case in mind. Perhaps there exists a more heavy-weight ML service that should only run if an image has first been predicated by a lighter-weight model. Media that trigger the lighter-weight rule would then be put into a queue to be evaluated against the heavy-weight service. This could help avoid filling human moderation queues with many false positives that might arise from a light-weight model, but also avoid performing heavy-weight computation on every media item that streams through the system.

Originally posted by @gbiss in roostorg/awesome-safety-tools#44 (comment)

[cassidyjames]

Yes! This is the model our detections systems use at GitHub... high recall, low precision rules first, then higher precision (more computationally expensive) rules (including bespoke classifiers and LLMs) run to triage the detections and drive down false positives.

Originally posted by @tgthorley in roostorg/awesome-safety-tools#44 (reply in thread)