Fix High CPU Usage in Review Validation System

[xiaomi7732]

CPU Performance Issue in Review Validation

The Azure MCP Profiler has identified several CPU performance bottlenecks in the Review Validation system that need to be addressed:

Primary Issues:

1. Excessive Random Number Generation: System.Random+ThreadSafeRandom.Next is consuming ~7% of CPU time

   • Called from ReviewHelper.DeserializeLocalizedTerm
   • Used to generate random strings for disallowed words
   • This is particularly wasteful when creating 30,000 random strings on startup

2. Inefficient LINQ Operations: System.Linq.Enumerable+WhereSelectEnumerableIterator.ToList() is consuming ~19.7% of CPU time

   • Called from ReviewValidation.StringValidation
   • This is being executed repeatedly for the same culture, which is inefficient even with the caching mechanism

3. Expensive String Replacements: System.String.ReplaceCore is consuming ~16.8% of CPU time

   • Called from ReviewValidation.StringValidation
   • Performing string replacements for all disallowed words (30,000 of them) on each validation call

Memory Issues:

• The string constructor (System.String.Ctor) is allocating significant memory (~26.4%)
• The ToList() operation is causing high memory allocations (~58.6%)

Call Stack Information:

System.Private.CoreLib!System.Random+ThreadSafeRandom.Next(int32,int32)
Store!ReviewHelper.<LoadDisallowedWords>g__DeserializeLocalizedTerm|0_0()
Store!ReviewHelper+<LoadDisallowedWords>d__0.MoveNext()
System.Linq!System.Linq.Enumerable+WhereSelectEnumerableIterator`2[System.__Canon,System.__Canon].ToList()
Store!Store.Reviews.ReviewValidation.StringValidation(class System.String,wchar,class System.Globalization.CultureInfo)
Store!Store.Reviews.BackgroundReviewValidation+<ExecuteAsync>d__0.MoveNext()

Recommendations:

1. Optimize Random Word Generation:

   • Consider pre-generating and caching the disallowed words once at startup instead of generating 30,000 random words
   • Use a more efficient random word generation mechanism if random words are necessary

2. Improve LINQ Operations:

   • Review the caching mechanism for filtered word lists by culture
   • Consider pre-filtering words by culture at initialization time

3. Optimize String Replacement:

   • Consider using a more efficient algorithm for checking and replacing disallowed words
   • Potentially use a trie or other data structure for faster word matching

Affected Files:

• Reviews/ReviewHelper.cs
• Reviews/ReviewValidation.cs
• Reviews/BackgroudReviewValidation.cs

This issue has a significant impact on application performance and should be addressed with high priority.