Skip to content

Chore: Optimize test suite performance and eliminate errors#2148

Merged
caneppelevitor merged 3 commits intostagefrom
chore/optimize-tests
Jan 29, 2026
Merged

Chore: Optimize test suite performance and eliminate errors#2148
caneppelevitor merged 3 commits intostagefrom
chore/optimize-tests

Conversation

@snowmous3
Copy link
Collaborator

@snowmous3 snowmous3 commented Nov 14, 2025 •
edited
Loading

Summary

Comprehensive test suite optimization that improves performance by 36% and eliminates all timeout and duplicate key errors. All 70 tests now pass reliably.

Performance Improvements

Stage Time Improvement
Original (with timeouts) ~197s baseline
Sequential optimization 125.687s 36% faster ✅
Parallel execution 65.429s 67% faster 🎉

📈 Key Achievements

  • ⚡ 67% total performance improvement (from 197s to 65.4s)
  • ✅ 100% test success rate - All 70 tests passing
  • 🔄 Parallel execution enabled - Tests running concurrently on 50% of CPU cores
  • 🚫 Zero duplicate key errors - Fixed with upsert operations

🔧 What Changed

Latest Optimization (Parallel Execution):

  1. Updated SeedTestUser to use updateOne with upsert: true
  2. Updated SeedTestPersonality to use bulkWrite with upsert operations
  3. Enabled parallel execution (maxWorkers: "50%")
  4. Made seed operations idempotent and safe for concurrent execution

✨ Benefits

  • Development: Faster feedback loop (65s vs 197s)
  • CI/CD: Reduced pipeline time and costs
  • Reliability: All tests consistently passing
  • Scalability: Can handle more tests without proportional time increase

The tests are now 3x faster than the original implementation while maintaining 100% reliability!

Test plan

  • Run full test suite locally - all tests passing
  • Verify database cleanup works correctly between suites
  • Confirm no duplicate key errors
  • Validate performance improvements

Related Ticket # (issue)

Type of change

  • Bug fix (non-breaking change which fixes an issue)
  • Existing feature enhancement (non-breaking change which modifies existing functionality)

Developer Checklist

General

  • Code is appropriately commented, particularly in hard-to-understand areas
  • Repository documentation has been updated (Readme.md) with additional steps required for a local environment setup.
  • No console.log or related logging is added.
  • No code is repeated/duplicated in violation of DRY. The exception to this is for new (MVP/Prototype) functionality where the abstraction layer may not be clear (comments should be added to explain the violation of DRY in these scenarios).
  • Documented with TSDoc all library and controller new functions

Backend Changes

  • All endpoints are appropriately secured with Middleware authentication
  • All new endpoints have a interface schema defined

Tests

  • All existing unit and end to end tests pass across all services
  • Unit and end to end tests have been added to ensure backend APIs behave as expected

Merge Request Review Checklist

  • An issue is linked to this PR and these changes meet the requirements outlined in the linked issue(s)
  • High risk and core workflows have been tested and verified in a local environment.
  • Enhancements or opportunities to improve performance, stability, security or code readability have been noted and documented in Project do Github issues if not being addressed.
  • Any dependent changes have been merged and published in downstream modules
  • Changes to multiple services can be deployed in parallel and independently. If not, changes should be broken out into separate merge requests and deployed in order.

@snowmous3 snowmous3 self-assigned this Nov 14, 2025
@sonarqubecloud
Copy link

sonarqubecloud bot commented Nov 14, 2025

Quality Gate Passed Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

@snowmous3
chore: optimize test suite performance and fix errors
36bda30 
Implemented comprehensive test optimization to improve performance and reliability:

**Performance Improvements:**
- Reduced total test execution time from ~197s to ~126s (36% faster)
- All 70 tests now passing with 100% success rate
- Eliminated timeout and duplicate key errors

**Key Changes:**
1. Shared MongoDB Instance
   - Created global setup/teardown for single MongoMemoryServer instance
   - Eliminated per-test-suite MongoDB instances
   - Reduced startup overhead significantly

2. Database Cleanup
   - Added CleanupDatabase utility to clear collections between test suites
   - Prevents duplicate key errors from hardcoded test data IDs
   - Ensures clean state for each test suite

3. Test Configuration
   - Set maxWorkers to 1 for sequential execution (avoids data conflicts)
   - Increased test timeout to 30s for reliability
   - Configured MongoDB binary caching

4. Updated Test Files
   - Removed individual MongoMemoryServer instances
   - Added database cleanup in afterAll hooks
   - Updated all e2e tests to use shared MongoDB instance

Files modified:
- server/tests/globalSetup.ts (new)
- server/tests/globalTeardown.ts (new)
- server/tests/utils/CleanupDatabase.ts (new)
- server/tests/jest-e2e.config.json
- jest-mongodb-config.js (new)
- All e2e test files and parser.service.spec.ts
@snowmous3
perf: enable parallel test execution with upsert operations
c190d2f 
Achieved 67% faster test execution by enabling parallel test runs.

**Performance Results:**
- Original (with timeouts): ~197s
- Sequential optimization: 125.687s (36% improvement)
- Parallel execution: 65.429s (67% improvement, 48% faster than sequential)

**Key Changes:**
1. Enabled parallel execution (maxWorkers: 50% in jest-e2e.config.json)
2. Updated SeedTestUser to use upsert operation instead of insertOne
3. Updated SeedTestPersonality to use bulkWrite with upsert operations
4. Eliminated duplicate key errors during parallel test execution

**Technical Details:**
- Changed seed functions from insertOne/insertMany to updateOne/bulkWrite with upsert: true
- This makes seed operations idempotent and safe for concurrent execution
- Tests can now run in parallel without data conflicts
- All 70 tests passing with 100% success rate

Test Results:
- Test Suites: 7 passed, 7 total
- Tests: 70 passed, 70 total
- Time: 65.429s (vs 125.687s sequential)
@sonarqubecloud
Copy link

sonarqubecloud bot commented Jan 29, 2026

Quality Gate Passed Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

@caneppelevitor caneppelevitor merged commit 12dd69f into stage Jan 29, 2026
7 of 9 checks passed
@LuizFNJ LuizFNJ mentioned this pull request Feb 25, 2026
Merged
46 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@caneppelevitor caneppelevitor caneppelevitor approved these changes

@thesocialdev thesocialdev Awaiting requested review from thesocialdev thesocialdev is a code owner

Assignees

@snowmous3 snowmous3

Labels

None yet

Projects

AletheiaFact Backlog
Status: QA

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@snowmous3 @caneppelevitor