feat: Collect new statistics for the Contact Identification feature

[matheusbsilva137]

Proposed changes (including videos or screenshots)

Added statistics related to the new Contact Identification feature:

• totalContacts: Total number of contacts;
• totalUnknownContacts: Total number of unknown contacts;
• totalMergedContacts: Total number of merged contacts (only contacts that have been deleted on merge are considered merged -- eg if one contact has other 2 similar ones to be merged in it, we'll count 2 merged contacts, NOT 3);
• totalConflicts: Total number of merge conflicts;
• totalResolvedConflicts: Total number of resolved conflicts;
• totalBlockedContacts: Total number of blocked contacts;
• totalPartiallyBlockedContacts: Total number of contacts partially blocked -- only contacts that have at least one blocked contact (but not all of them) are counted;
• totalFullyBlockedContacts: Total number of contacts fully blocked;
• totalVerifiedContacts: Total number of verified contacts;
• avgChannelsPerContact: Average number of channels per contact;
• totalContactsWithoutChannels: Number of contacts without channels;
• totalImportedContacts: Total number of imported contacts;
• totalUpsellViews: Total number of "Advanced Contact Management" Upsell CTA views;
• totalUpsellClicks: Total number of "Advanced Contact Management" Upsell CTA clicks;

Issue(s)

Steps to test or reproduce

Further comments

For load test purposes, I populated my local DB with +2M contacts (2069065 contacts exactly) using this filler repo which I extended in a fork so that the contacts collection could also be populated randomly considering all of its optional fields and arrays of random sizes. Using this DB, I ran each new query 10 times and calculated the average execution time (in milliseconds) and a 95% confidence interval for each of them (of course, with this we're making an assumption that the average execution time for each query follow a normal distribution):

• totalConflicts and avgChannelsPerContact (they're calculated together): 3,444.1 ±75.577 (±2.19%);
• totalUnknownContacts: 324.6 ±6.268 (±1.93%);
• totalBlockedContacts: 589 ±18.433 (±3.13%);
• totalFullyBlockedContacts: 2,743.6 ±41.011 (±1.49%);
• totalPartiallyBlockedContacts: infered from totalBlockedContacts and totalFullyBlockedContacts;
• totalVerifiedContacts: 610.7 ±23.301 (±3.82%);
• totalContactsWithoutChannels: 819.9 ±23.373 (±2.85%);

All new queries used a index, except totalContactsWithoutChannels, totalConflicts and avgChannelsPerContact which all need to read all documents (so I believe they're all essentially a COLSCAN).

SCI-78

[dionisio-bot]

Looks like this PR is ready to merge! 🎉
If you have any trouble, please check the PR guidelines

[changeset-bot]

🦋 Changeset detected

Latest commit: 8ba58ce

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 35 packages

Name	Type
@rocket.chat/meteor	Minor
@rocket.chat/core-typings	Minor
@rocket.chat/model-typings	Minor
@rocket.chat/uikit-playground	Patch
@rocket.chat/api-client	Patch
@rocket.chat/apps	Patch
@rocket.chat/core-services	Patch
@rocket.chat/cron	Patch
@rocket.chat/ddp-client	Patch
@rocket.chat/freeswitch	Patch
@rocket.chat/fuselage-ui-kit	Major
@rocket.chat/gazzodown	Major
@rocket.chat/livechat	Patch
@rocket.chat/rest-typings	Minor
@rocket.chat/ui-contexts	Major
@rocket.chat/account-service	Patch
@rocket.chat/authorization-service	Patch
@rocket.chat/ddp-streamer	Patch
@rocket.chat/omnichannel-transcript	Patch
@rocket.chat/presence-service	Patch
@rocket.chat/queue-worker	Patch
@rocket.chat/stream-hub-service	Patch
@rocket.chat/license	Patch
@rocket.chat/omnichannel-services	Patch
@rocket.chat/pdf-worker	Patch
@rocket.chat/presence	Patch
rocketchat-services	Patch
@rocket.chat/models	Patch
@rocket.chat/network-broker	Patch
@rocket.chat/ui-avatar	Major
@rocket.chat/ui-client	Major
@rocket.chat/ui-video-conf	Major
@rocket.chat/ui-voip	Major
@rocket.chat/web-ui-registration	Major
@rocket.chat/instance-status	Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

[github-actions]

PR Preview Action v1.4.8
🚀 Deployed preview to https://RocketChat.github.io/Rocket.Chat/pr-preview/pr-33895/
on branch gh-pages at 2024-12-19 23:08 UTC

[codecov]

Codecov Report

Attention: Patch coverage is 75.00000% with 4 lines in your changes missing coverage. Please review.

Project coverage is 75.81%. Comparing base (76f6239) to head (8ba58ce).
Report is 1 commits behind head on develop.

Additional details and impacted files

@@             Coverage Diff             @@
##           develop   #33895      +/-   ##
===========================================
- Coverage    75.81%   75.81%   -0.01%     
===========================================
  Files          512      512              
  Lines        22208    22221      +13     
  Branches      5404     5407       +3     
===========================================
+ Hits         16837    16846       +9     
- Misses        4720     4722       +2     
- Partials       651      653       +2     

Flag	Coverage Δ
unit	75.81% <75.00%> (-0.01%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

[dougfabris]

approving on behalf of frontend!

[sampaiodiego]

is it asking too much to create some data, perform some analysis of the queries and attach to the PR? I mean, I wonder if we should start being more cautions adding more metrics, specially when talking about omnichannel metrics as we have some HUGE datasets out there

The base branch was changed.

[matheusbsilva137]

is it asking too much to create some data, perform some analysis of the queries and attach to the PR?

Done! I wrote the results of my tests in the PR description