FEATURE: Make emotion /filter ordering match the dashboard table (#939)

* FEATURE: Make emotion /filter ordering match the dashboard table

This change makes the /filter endpoint use the same criteria we use
in the dashboard table for emotion, so it is not confusing for users.
It means that only posts made in the period with the emotion shall be
shown in the /filter, and the order is simply a count of posts that
match the emotion in the period.

It also uses a trick to extract the filter period, and apply it to
the CTE clause that calculates post emotion count on the period, making
it a bit more efficient. Downside is that /filter filters are evaluated
from left to right, so it will only get the speed-up if the emotion
order is last. As we do this on the dashboard table, it should cover
most uses of the ordering, kicking the need for materialized views
down the road.

* Remove zero score in filter

* add table tooltip

* lint

Author: xfalcox

assets/javascripts/discourse/templates/admin-dashboard-sentiment.hbs

@@ -27,8 +27,16 @@
           <ul class="breadcrumb">
             <li class="item report">
               <LinkTo @route="adminReports" class="report-url">
-                {{i18n "admin.dashboard.emotion"}}
+                {{i18n "admin.dashboard.emotion.title"}}
               </LinkTo>
+              <DTooltip @interactive="true">
+                <:trigger>
+                  {{d-icon "circle-question"}}
+                </:trigger>
+                <:content>
+                  <span>{{i18n "admin.dashboard.emotion.description"}}</span>
+                </:content>
+              </DTooltip>
             </li>
           </ul>
         </div>

config/locales/client.en.yml

@@ -12,7 +12,9 @@ en:
         categories:
           discourse_ai: "Discourse AI"
       dashboard:
-        emotion: "Emotion"
+        emotion:
+          title: "Emotion"
+          description: "The table lists a count of posts classified with a determined emotion. Classified with the model 'SamLowe/roberta-base-go_emotions'."
   js:
     discourse_automation:
       scriptables:

config/locales/server.en.yml

@@ -112,7 +112,7 @@ en:
   reports:
     overall_sentiment:
       title: "Overall sentiment"
-      description: 'The chart compares the number of posts classified as either positive or negative. These are calculated when positive or negative scores > the set threshold score. This means neutral posts are not shown. Private messages (PMs) are also excluded. Classified with "cardiffnlp/twitter-roberta-base-sentiment-latest"'
+      description: 'The chart compares the number of posts classified as either positive or negative. These are calculated when positive or negative scores > the set threshold score. This means neutral posts are not shown. Personal messages (PMs) are also excluded. Classified with "cardiffnlp/twitter-roberta-base-sentiment-latest"'
       xaxis: "Positive(%)"
       yaxis: "Date"
     emotion_admiration:

lib/sentiment/emotion_filter_order.rb

@@ -6,14 +6,25 @@ class EmotionFilterOrder
       def self.register!(plugin)
         Emotions::LIST.each do |emotion|
           filter_order_emotion = ->(scope, order_direction) do
+            scope_period =
+              scope
+                .arel
+                &.constraints
+                &.flat_map(&:children)
+                &.find do |node|
+                  node.is_a?(Arel::Nodes::Grouping) &&
+                    node.expr.to_s.match?(/topics\.bumped_at\s*>=/)
+                end
+                &.expr
+                &.split(">=")
+                &.last if scope.arel.constraints.present? &&
+              scope.arel.constraints.any? { |c| c.is_a?(Arel::Nodes::Grouping) }
+
+            # Fallback in case we can't find the scope period
+            scope_period ||= "CURRENT_DATE - INTERVAL '1 year'"
+
             emotion_clause = <<~SQL
-              SUM(
-                CASE
-                  WHEN (classification_results.classification::jsonb->'#{emotion}')::float > 0.1
-                  THEN 1
-                  ELSE 0
-                END
-               )::float / COUNT(posts.id)
+              COUNT(*) FILTER (WHERE (classification_results.classification::jsonb->'#{emotion}')::float > 0.1)
             SQL
 
             # TODO: This is slow, we will need to materialize this in the future
@@ -35,10 +46,11 @@ def self.register!(plugin)
                     AND topics.deleted_at IS NULL
                     AND posts.deleted_at IS NULL
                     AND posts.post_type = 1
+                    AND posts.created_at >= #{scope_period}
                 GROUP BY
                     1
                 HAVING
-                    #{emotion_clause} > 0.05
+                    #{emotion_clause} > 0
             SQL
 
             scope