MONGOID-5270 Implement .tally method for Mongoid#Criteria (#5344)

* MONGOID-5270 initial implementation of tally

* MONGOID-5270 finish and fully test tally feature

* MONGOID-5270 add typeless field test

* MONGOID-5270 don't change existing behavior

* MONGOID-5270 correct documentation

* MONGOID-5702 docs, release note, renaming

* MONGOID-5270 add none case

* MONGOID-5270 tweaks to pluck and tally logic to make them function correctly and consistently

* MONGOID-5270 update comment

* MONGOID-5270 add tally implementation

* MONGOID-5270 change to not use tally

* MONGOID-5270 add duplicate keys test

* MONGOID-5270 move fetch_and_demongoize and change indifferent access

* MONGOID-5270 use eq instead of be

* Revert "MONGOID-5270 use eq instead of be"

This reverts commit 25d65ef.

Author: neilshweky

docs/reference/queries.txt

@@ -1380,6 +1380,27 @@ Mongoid also has some helpful methods on criteria.
           # expands out to "managers.name" in the query:
           Band.all.pluck('managers.n')
 
+   * - ``Criteria#tally``
+
+       *Get a mapping of values to counts for the provided field.*
+
+       *This method accepts the dot notation, thus permitting referencing
+       fields in embedded associations.*
+
+       *This method respects :ref:`field aliases <field-aliases>`,
+       including those defined in embedded documents.*
+
+     -
+        .. code-block:: ruby
+
+          Band.all.tally(:name)
+
+          Band.all.tally('cities.name')
+
+          # Using the earlier definition of Manager,
+          # expands out to "managers.name" in the query:
+          Band.all.tally('managers.n')
+
 
 Eager Loading
 =============

docs/release-notes/mongoid-8.0.txt

@@ -527,6 +527,7 @@ it was stored and returned as a String.
   # => Mongoid 7: "data!"
   # => Mongoid 8: <BSON::Binary:0x2600 type=generic data=0x6461746121...>
 
+
 ``Decimal128``-backed ``BigDecimal`` fields
 -------------------------------------------
 
@@ -540,3 +541,35 @@ feature flag turned off, values assigned to fields of type ``BigDecimal`` are
 stored as Strings. See the section on :ref:`BigDecimal Fields <bigdecimal-fields>`
 for more details.
 
+
+Implemented ``.tally`` method on ``Mongoid#Criteria``
+-----------------------------------------------------
+
+Mongoid 8 implements the ``.tally`` method on ``Mongoid#Criteria``. ``tally``
+takes a field name as a parameter and returns a mapping from values to their
+counts. For example, take the following model:
+
+.. code::
+
+  class User
+    include Mongoid::Document
+    field :age
+  end
+
+and the following documents in the database:
+
+.. code::
+
+  { _id: 1, age: 21 }
+  { _id: 2, age: 21 }
+  { _id: 3, age: 22 }
+
+Calling ``tally`` on the age field yields the following:
+
+.. code::
+
+  User.tally("age")
+  # => { 21 => 2, 22 => 1 }
+
+The ``tally`` method accepts the dot notation and field aliases. It also
+allows for tallying localized fields.

lib/mongoid/association/macros.rb

@@ -14,7 +14,7 @@ module Macros
         class_attribute :relations
 
         # A hash that maps aliases to their associations. This is used when
-        # associations specify the `as` option, or on a referenced association.
+        # associations specify the `store_as` option, or on a referenced association.
         # On a referenced association, this is used to map the foreign key to
         # the association's name. For example, if we had the following
         # relationship:

lib/mongoid/contextual/memory.rb

@@ -200,6 +200,14 @@ def pluck(*fields)
         documents.pluck(*fields)
       end
 
+      def tally(field)
+        return documents.each_with_object({}) do |d, acc|
+          v = retrieve_value_at_path(d, field)
+          acc[v] ||= 0
+          acc[v] += 1
+        end
+      end
+
       # Skips the provided number of documents.
       #
       # @example Skip the documents.
@@ -415,6 +423,62 @@ def prepare_remove(doc)
       def _session
         @criteria.send(:_session)
       end
+
+      # Retrieve the value for the current document at the given field path.
+      #
+      # For example, if I have the following models:
+      #
+      #   User has_many Accounts
+      #   address is a hash on Account
+      #
+      #   u = User.new(accounts: [ Account.new(address: { street: "W 50th" }) ])
+      #   retrieve_value_at_path(u, "user.accounts.address.street")
+      #   # => [ "W 50th" ]
+      #
+      # Note that the result is in an array since accounts is an array. If it
+      # was nested in two arrays the result would be in a 2D array.
+      #
+      # @param [ Object ] document The object to traverse the field path.
+      # @param [ String ] field_path The dotted string that represents the path
+      #   to the value.
+      #
+      # @return [ Object | nil ] The value at the given field path or nil if it
+      #   doesn't exist.
+      def retrieve_value_at_path(document, field_path)
+        return if field_path.blank? || !document
+        segment, remaining = field_path.to_s.split('.', 2)
+
+        curr = if document.is_a?(Document)
+          # Retrieves field for segment to check localization. Only does one
+          # iteration since there's no dots
+          res = if remaining
+            field = document.class.traverse_association_tree(segment)
+            # If this is a localized field, and there are remaining, get the
+            # _translations hash so that we can get the specified translation in
+            # the remaining
+            if field&.localized?
+              document.send("#{segment}_translations")
+            end
+          end
+          res.nil? ? document.send(segment) : res
+        elsif document.is_a?(Hash)
+          # TODO: Remove the indifferent access when implementing MONGOID-5410.
+          document.key?(segment.to_s) ?
+            document[segment.to_s] :
+            document[segment.to_sym]
+        else
+          nil
+        end
+
+        return curr unless remaining
+
+        if curr.is_a?(Array)
+          # compact is used for consistency with server behavior.
+          curr.map { |d| retrieve_value_at_path(d, remaining) }.compact
+        else
+          retrieve_value_at_path(curr, remaining)
+        end
+      end
     end
   end
 end

lib/mongoid/contextual/mongo.rb

@@ -454,6 +454,75 @@ def pluck(*fields)
         end
       end
 
+      # Get a hash of counts for the values of a single field. For example,
+      # if the following documents were in the database:
+      #
+      #   { _id: 1, age: 21 }
+      #   { _id: 2, age: 21 }
+      #   { _id: 3, age: 22 }
+      #
+      #   Model.tally("age")
+      #
+      # would yield the following result:
+      #
+      #   { 21 => 2, 22 => 1 }
+      #
+      # When tallying a field inside an array or embeds_many association:
+      #
+      #   { _id: 1, array: [ { x: 1 }, { x: 2 } ] }
+      #   { _id: 2, array: [ { x: 1 }, { x: 2 } ] }
+      #   { _id: 3, array: [ { x: 1 }, { x: 3 } ] }
+      #
+      #   Model.tally("array.x")
+      #
+      # The keys of the resulting hash are arrays:
+      #
+      #   { [ 1, 2 ] => 2, [ 1, 3 ] => 1 }
+      #
+      # Note that if tallying an element in an array of hashes, and the key
+      # doesn't exist in some of the hashes, tally will not include those
+      # nil keys in the resulting hash:
+      #
+      #   { _id: 1, array: [ { x: 1 }, { x: 2 }, { y: 3 } ] }
+      #
+      #   Model.tally("array.x")
+      #   # => { [ 1, 2 ] => 1 }
+      #
+      # @param [ String | Symbol ] field The field name.
+      #
+      # @return [ Hash ] The hash of counts.
+      def tally(field)
+        name = klass.cleanse_localized_field_names(field)
+
+        fld = klass.traverse_association_tree(name)
+        pipeline = [ { "$group" => { _id: "$#{name}", counts: { "$sum": 1 } } } ]
+        pipeline.unshift("$match" => view.filter) unless view.filter.blank?
+
+        collection.aggregate(pipeline).reduce({}) do |tallies, doc|
+          is_translation = "#{name}_translations" == field.to_s
+          val = doc["_id"]
+
+          key = if val.is_a?(Array)
+            val.map do |v|
+              demongoize_with_field(fld, v, is_translation)
+            end
+          else
+            demongoize_with_field(fld, val, is_translation)
+          end
+
+          # The only time where a key will already exist in the tallies hash
+          # is when the values are stored differently in the database, but
+          # demongoize to the same value. A good example of when this happens
+          # is when using localized fields. While the server query won't group
+          # together hashes that have other values in different languages, the
+          # demongoized value is just the translation in the current locale,
+          # which can be the same across multiple of those unequal hashes.
+          tallies[key] ||= 0
+          tallies[key] += doc["counts"]
+          tallies
+        end
+      end
+
       # Skips the provided number of documents.
       #
       # @example Skip the documents.
@@ -695,6 +764,26 @@ def acknowledged_write?
         collection.write_concern.nil? || collection.write_concern.acknowledged?
       end
 
+      # Fetch the element from the given hash and demongoize it using the
+      # given field. If the obj is an array, map over it and call this method
+      # on all of its elements.
+      #
+      # @param [ Hash | Array<Hash> ] obj The hash or array of hashes to fetch from.
+      # @param [ String ] meth The key to fetch from the hash.
+      # @param [ Field ] field The field to use for demongoization.
+      #
+      # @return [ Object ] The demongoized value.
+      #
+      # @api private
+      def fetch_and_demongoize(obj, meth, field)
+        if obj.is_a?(Array)
+          obj.map { |doc| fetch_and_demongoize(doc, meth, field) }
+        else
+          res = obj.try(:fetch, meth, nil)
+          field ? field.demongoize(res) : res.class.demongoize(res)
+        end
+      end
+
       # Extracts the value for the given field name from the given attribute
       # hash.
       #
@@ -703,21 +792,13 @@ def acknowledged_write?
       #
       # @param [ Object ] The value for the given field name
       def extract_value(attrs, field_name)
-        def fetch_and_demongoize(d, meth, klass)
-          res = d.try(:fetch, meth, nil)
-          if field = klass.fields[meth]
-            field.demongoize(res)
-          else
-            res.class.demongoize(res)
-          end
-        end
-
         i = 1
         num_meths = field_name.count('.') + 1
         k = klass
         curr = attrs.dup
 
         klass.traverse_association_tree(field_name) do |meth, obj, is_field|
+          field = obj if is_field
           is_translation = false
           # If no association or field was found, check if the meth is an
           # _translations field.
@@ -737,18 +818,18 @@ def fetch_and_demongoize(d, meth, klass)
           #    value so the full hash is returned.
           # 4. Otherwise, fetch and demongoize the value for the key meth.
           curr = if curr.is_a? Array
-            res = curr.map { |x| fetch_and_demongoize(x, meth, k) }
+            res = fetch_and_demongoize(curr, meth, field)
             res.empty? ? nil : res
-          elsif !is_translation && k.fields[meth]&.localized?
+          elsif !is_translation && field&.localized?
             if i < num_meths
               curr.try(:fetch, meth, nil)
             else
-              fetch_and_demongoize(curr, meth, k)
+              fetch_and_demongoize(curr, meth, field)
             end
           elsif is_translation
             curr.try(:fetch, meth, nil)
           else
-            fetch_and_demongoize(curr, meth, k)
+            fetch_and_demongoize(curr, meth, field)
           end
 
           # If it's a relation, update the current klass with the relation klass.
@@ -771,12 +852,28 @@ def fetch_and_demongoize(d, meth, klass)
       # @return [ Object ] The demongoized value.
       def recursive_demongoize(field_name, value, is_translation)
         field = klass.traverse_association_tree(field_name)
+        demongoize_with_field(field, value, is_translation)
+      end
 
+      # Demongoize the value for the given field. If the field is nil or the
+      # field is a translations field, the value is demongoized using its class.
+      #
+      # @param [ Field ] field The field to use to demongoize.
+      # @param [ Object ] value The value to demongoize.
+      # @param [ Boolean ] is_translation The field we are retrieving is an
+      #   _translations field.
+      #
+      # @return [ Object ] The demongoized value.
+      #
+      # @api private
+      def demongoize_with_field(field, value, is_translation)
         if field
           # If it's a localized field that's not a hash, don't demongoize
           # again, we already have the translation. If it's an _translations
           # field, don't demongoize, we want the full hash not just a
           # specific translation.
+          # If it is a hash, and it's not a transaltions field, we need to
+          # demongoize to get the correct translation.
           if field.localized? && (!value.is_a?(Hash) || is_translation)
             value.class.demongoize(value)
           else

lib/mongoid/contextual/none.rb

@@ -95,6 +95,18 @@ def pluck(*args)
         []
       end
 
+      # Tally the field values in null context.
+      #
+      # @example Get the values for null context.
+      #   context.tally(:name)
+      #
+      # @param [ String, Symbol ] arg Field to tally.
+      #
+      # @return [ Hash ] An empty Hash.
+      def tally(arg)
+        {}
+      end
+
       # Create the new null context.
       #
       # @example Create the new context.

lib/mongoid/findable.rb

@@ -43,7 +43,8 @@ module Findable
       :sum,
       :text_search,
       :update,
-      :update_all
+      :update_all,
+      :tally,
 
     # Returns a count of records in the database.
     # If you want to specify conditions use where.