Skip to content

Conversation

@mcasimir
Copy link
Collaborator

@mcasimir mcasimir commented Sep 4, 2025

NOTE: need to check the content of each eval to make sure it matches what we have in the document

@Copilot Copilot AI review requested due to automatic review settings September 4, 2025 17:13
@mcasimir mcasimir requested a review from a team as a code owner September 4, 2025 17:13
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR adds comprehensive evaluation test cases for the Compass Assistant across multiple MongoDB-related topics. The changes expand test coverage by organizing eval cases into separate, focused modules and introducing new test scenarios for authentication, connection troubleshooting, query optimization, and data modeling guidance.

Key Changes:

  • Reorganized existing eval cases into topic-specific modules with improved naming conventions
  • Added new eval cases covering OIDC authentication, connection errors, explain plans, and data modeling scenarios
  • Expanded Atlas Search test cases with additional query patterns and use cases

Reviewed Changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated no comments.

Show a summary per file
File Description
packages/compass-assistant/test/eval-cases/index.ts Updated imports and exports to use new module names and include additional test case collections
packages/compass-assistant/test/eval-cases/oidc-auth.ts New file with OIDC authentication troubleshooting test cases
packages/compass-assistant/test/eval-cases/model-data.ts Removed (content moved to data-modeling.ts)
packages/compass-assistant/test/eval-cases/explain-plan.ts New file with query performance optimization test cases
packages/compass-assistant/test/eval-cases/data-modeling.ts New file with MongoDB data modeling guidance test cases
packages/compass-assistant/test/eval-cases/connection-errors.ts New file with connection troubleshooting test cases
packages/compass-assistant/test/eval-cases/atlas-search.ts Expanded with additional Atlas Search functionality test cases
packages/compass-assistant/test/eval-cases/aggregation-pipeline.ts Renamed variable and cleaned up formatting

Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.

@mcasimir mcasimir added no release notes Fix or feature not for release notes no-title-validation Skips validation of PR titles (conventional commit adherence + JIRA ticket inclusion) labels Sep 4, 2025
@mcasimir mcasimir changed the title chore: add more eval cases COMPASS-9758 [WIP] chore: add more eval cases COMPASS-9758 Sep 4, 2025
@mcasimir mcasimir marked this pull request as draft September 4, 2025 17:18
@mcasimir mcasimir closed this Sep 4, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

no release notes Fix or feature not for release notes no-title-validation Skips validation of PR titles (conventional commit adherence + JIRA ticket inclusion)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant