Skip to content

Conversation

nirinchev
Copy link
Collaborator

Proposed changes

The atlas tests would try and cleanup some resources in afterAll/afterEach, but those would run even if beforeAll failed. This was causing some red herring errors due to calling atlas endpoints with an empty projectId, for example. While this doesn't address the underlying problems that caused the setup to fail, at least it reduces the noise.

@Copilot Copilot AI review requested due to automatic review settings August 27, 2025 12:02
@nirinchev nirinchev requested a review from a team as a code owner August 27, 2025 12:02
Copy link
Contributor

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

@coveralls
Copy link
Collaborator

coveralls commented Aug 27, 2025

Pull Request Test Coverage Report for Build 17267759772

Details

  • 0 of 0 changed or added relevant lines in 0 files are covered.
  • No unchanged relevant lines lost coverage.
  • Overall coverage increased (+0.06%) to 81.887%

Totals Coverage Status
Change from base Build 17265344277: 0.06%
Covered Lines: 4422
Relevant Lines: 5381

💛 - Coveralls

@@ -10,6 +10,7 @@ describeAccuracyTests([
parameters: {
database: "mflix",
collection: "movies",
exportTitle: Matcher.string(),
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cc @himanshusinghs - not sure what has happened here - I think I added the titles at some point, then they disappeared 😬 But looking at the results on main, it seems the accuracy tests are failing because of this.


return result;
},
{ timeout: 5000 }
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cc @kmruiz - I expanded this a bit as the test was failing due to the authenticatedUsers collection being empty, not status itself being undefined. I also added it a bit more time since I saw it fail with the 1 second default.

Here's a run where we're only checking status: https://github.com/mongodb-js/mongodb-mcp-server/actions/runs/17266097331/job/48998448665?pr=483

Here's the one that's failing with the 1 second default: https://github.com/mongodb-js/mongodb-mcp-server/actions/runs/17267312116/job/49002419896?pr=483

Copy link
Contributor

📊 Accuracy Test Results

📈 Summary

Metric Value
Commit SHA 0e23c5fc4728faa6baf9d7b460c5f751d66c82a5
Run ID 76f6c853-a2e3-4163-818c-048bbc2b1499
Status done
Total Prompts Evaluated 60
Models Tested 1
Average Accuracy 97.9%
Responses with 0% Accuracy 0
Responses with 75% Accuracy 5
Responses with 100% Accuracy 55

📎 Download Full HTML Report - Look for the accuracy-test-summary artifact for detailed results.

Report generated on: 8/27/2025, 1:21:02 PM

@nirinchev nirinchev merged commit b5e2204 into main Aug 27, 2025
20 checks passed
@nirinchev nirinchev deleted the ni/atlas-tests-guards branch August 27, 2025 13:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants