Skip to content

chore(tests): accuracy tests for MongoDB tools exposed by MCP server MCP-39 #1285

chore(tests): accuracy tests for MongoDB tools exposed by MCP server MCP-39

chore(tests): accuracy tests for MongoDB tools exposed by MCP server MCP-39 #1285

Triggered via pull request July 18, 2025 11:29
Status Failure
Total duration 6m 25s
Artifacts 3

code_health.yaml

on: pull_request
Matrix: Run MongoDB tests
Run Atlas tests
2m 13s
Run Atlas tests
Report Coverage
36s
Report Coverage
Fit to window
Zoom out
Zoom in

Annotations

13 errors, 19 warnings, and 2 notices
Run MongoDB tests (macos-latest)
Process completed with exit code 1.
tests/integration/tools/mongodb/read/find.test.ts > find tool > should have correct metadata: tests/matchers/toIncludeSameMembers.ts#L4
AssertionError: expected [ { name: 'collection', …(3) }, …(5) ] to deeply equal ArrayContaining{…} - Expected + Received - ArrayContaining [ + [ { - "description": "Database name", - "name": "database", + "description": "Collection name", + "name": "collection", "required": true, "type": "string", }, { - "description": "Collection name", - "name": "collection", + "description": "Database name", + "name": "database", "required": true, "type": "string", }, { "description": "The query filter, matching the syntax of the query argument of db.collection.find()", "name": "filter", - "required": false, - "type": "object", - }, - { - "description": "The projection, matching the syntax of the projection argument of db.collection.find()", - "name": "projection", "required": false, "type": "object", }, { "description": "The maximum number of documents to return", "name": "limit", "required": false, "type": "number", }, { - "description": "A document, describing the sort order, matching the syntax of the sort argument of cursor.sort()", + "description": "The projection, matching the syntax of the projection argument of db.collection.find()", + "name": "projection", + "required": false, + "type": "object", + }, + { + "description": "A document, describing the sort order, matching the syntax of the sort argument of cursor.sort(). The keys of the object are the fields to sort on, while the values are the sort directions (1 for ascending, -1 for descending).", "name": "sort", "required": false, "type": "object", }, ] ❯ Object.toIncludeSameMembers tests/matchers/toIncludeSameMembers.ts:4:20 ❯ tests/integration/helpers.ts:211:32
tests/unit/accuracyScorer.test.ts > calculateToolCallingAccuracy > missing or incorrect parameters > should return 0 when aggregate tool call has incorrect pipeline: tests/unit/accuracyScorer.test.ts#L323
AssertionError: expected 1 to be +0 // Object.is equality - Expected + Received - 0 + 1 ❯ tests/unit/accuracyScorer.test.ts:323:28
Run MongoDB tests (ubuntu-latest)
Process completed with exit code 1.
tests/integration/tools/mongodb/read/find.test.ts > find tool > should have correct metadata: tests/matchers/toIncludeSameMembers.ts#L4
AssertionError: expected [ { name: 'collection', …(3) }, …(5) ] to deeply equal ArrayContaining{…} - Expected + Received - ArrayContaining [ + [ { - "description": "Database name", - "name": "database", + "description": "Collection name", + "name": "collection", "required": true, "type": "string", }, { - "description": "Collection name", - "name": "collection", + "description": "Database name", + "name": "database", "required": true, "type": "string", }, { "description": "The query filter, matching the syntax of the query argument of db.collection.find()", "name": "filter", - "required": false, - "type": "object", - }, - { - "description": "The projection, matching the syntax of the projection argument of db.collection.find()", - "name": "projection", "required": false, "type": "object", }, { "description": "The maximum number of documents to return", "name": "limit", "required": false, "type": "number", }, { - "description": "A document, describing the sort order, matching the syntax of the sort argument of cursor.sort()", + "description": "The projection, matching the syntax of the projection argument of db.collection.find()", + "name": "projection", + "required": false, + "type": "object", + }, + { + "description": "A document, describing the sort order, matching the syntax of the sort argument of cursor.sort(). The keys of the object are the fields to sort on, while the values are the sort directions (1 for ascending, -1 for descending).", "name": "sort", "required": false, "type": "object", }, ] ❯ Object.toIncludeSameMembers tests/matchers/toIncludeSameMembers.ts:4:20 ❯ tests/integration/helpers.ts:211:32
tests/unit/accuracyScorer.test.ts > calculateToolCallingAccuracy > missing or incorrect parameters > should return 0 when aggregate tool call has incorrect pipeline: tests/unit/accuracyScorer.test.ts#L323
AssertionError: expected 1 to be +0 // Object.is equality - Expected + Received - 0 + 1 ❯ tests/unit/accuracyScorer.test.ts:323:28
Run Atlas tests
Process completed with exit code 1.
tests/integration/tools/mongodb/read/find.test.ts > find tool > should have correct metadata: tests/matchers/toIncludeSameMembers.ts#L4
AssertionError: expected [ { name: 'collection', …(3) }, …(5) ] to deeply equal ArrayContaining{…} - Expected + Received - ArrayContaining [ + [ { - "description": "Database name", - "name": "database", + "description": "Collection name", + "name": "collection", "required": true, "type": "string", }, { - "description": "Collection name", - "name": "collection", + "description": "Database name", + "name": "database", "required": true, "type": "string", }, { "description": "The query filter, matching the syntax of the query argument of db.collection.find()", "name": "filter", - "required": false, - "type": "object", - }, - { - "description": "The projection, matching the syntax of the projection argument of db.collection.find()", - "name": "projection", "required": false, "type": "object", }, { "description": "The maximum number of documents to return", "name": "limit", "required": false, "type": "number", }, { - "description": "A document, describing the sort order, matching the syntax of the sort argument of cursor.sort()", + "description": "The projection, matching the syntax of the projection argument of db.collection.find()", + "name": "projection", + "required": false, + "type": "object", + }, + { + "description": "A document, describing the sort order, matching the syntax of the sort argument of cursor.sort(). The keys of the object are the fields to sort on, while the values are the sort directions (1 for ascending, -1 for descending).", "name": "sort", "required": false, "type": "object", }, ] ❯ Object.toIncludeSameMembers tests/matchers/toIncludeSameMembers.ts:4:20 ❯ tests/integration/helpers.ts:211:32
tests/unit/accuracyScorer.test.ts > calculateToolCallingAccuracy > missing or incorrect parameters > should return 0 when aggregate tool call has incorrect pipeline: tests/unit/accuracyScorer.test.ts#L323
AssertionError: expected 1 to be +0 // Object.is equality - Expected + Received - 0 + 1 ❯ tests/unit/accuracyScorer.test.ts:323:28
Run MongoDB tests (windows-latest)
Process completed with exit code 1.
tests/integration/tools/mongodb/read/find.test.ts > find tool > should have correct metadata: tests/matchers/toIncludeSameMembers.ts#L4
AssertionError: expected [ { name: 'collection', …(3) }, …(5) ] to deeply equal ArrayContaining{…} - Expected + Received - ArrayContaining [ + [ { - "description": "Database name", - "name": "database", + "description": "Collection name", + "name": "collection", "required": true, "type": "string", }, { - "description": "Collection name", - "name": "collection", + "description": "Database name", + "name": "database", "required": true, "type": "string", }, { "description": "The query filter, matching the syntax of the query argument of db.collection.find()", "name": "filter", - "required": false, - "type": "object", - }, - { - "description": "The projection, matching the syntax of the projection argument of db.collection.find()", - "name": "projection", "required": false, "type": "object", }, { "description": "The maximum number of documents to return", "name": "limit", "required": false, "type": "number", }, { - "description": "A document, describing the sort order, matching the syntax of the sort argument of cursor.sort()", + "description": "The projection, matching the syntax of the projection argument of db.collection.find()", + "name": "projection", + "required": false, + "type": "object", + }, + { + "description": "A document, describing the sort order, matching the syntax of the sort argument of cursor.sort(). The keys of the object are the fields to sort on, while the values are the sort directions (1 for ascending, -1 for descending).", "name": "sort", "required": false, "type": "object", }, ] ❯ Object.toIncludeSameMembers tests/matchers/toIncludeSameMembers.ts:4:20 ❯ tests/integration/helpers.ts:211:32
tests/unit/accuracyScorer.test.ts > calculateToolCallingAccuracy > missing or incorrect parameters > should return 0 when aggregate tool call has incorrect pipeline: tests/unit/accuracyScorer.test.ts#L323
AssertionError: expected 1 to be +0 // Object.is equality - Expected + Received - 0 + 1 ❯ tests/unit/accuracyScorer.test.ts:323:28
Report Coverage
Unable to download artifact(s): Artifact not found for name: test-results Please ensure that your artifact is not expired and the artifact was uploaded using a compatible version of toolkit/upload-artifact. For more information, visit the GitHub Artifacts FAQ: https://github.com/actions/toolkit/blob/main/packages/artifact/docs/faq.md
Run MongoDB tests (macos-latest)
No ALTQ support in kernel ALTQ related functions disabled pf enabled
Run MongoDB tests (macos-latest)
No ALTQ support in kernel ALTQ related functions disabled
Run MongoDB tests (macos-latest)
pfctl: Use of -f option, could result in flushing of rules present in the main ruleset added by the system at startup. See /etc/pf.conf for further details.
Run MongoDB tests (macos-latest)
YES (0)
Run MongoDB tests (macos-latest)
2025-07-18 11:29:23.063 sysadminctl[1841:10132] Creating home directory at /Users/mitmproxyuser
Run MongoDB tests (macos-latest)
2025-07-18 11:29:23.013 sysadminctl[1841:10132] Assigning UID: 502 GID: 20
Run MongoDB tests (macos-latest)
2025-07-18 11:29:22.903 sysadminctl[1841:10132] Creating user record…
Run MongoDB tests (macos-latest)
IOServiceMatchingfailed for: AppleM2ScalerCSCDriver
Run MongoDB tests (macos-latest)
2025-07-18 11:29:22.223 sysadminctl[1841:10132] ---------------------------- 2025-07-18 11:29:22.224 sysadminctl[1841:10132] No clear text password or interactive option was specified (adduser, change/reset password will not allow user to use FDE) ! 2025-07-18 11:29:22.224 sysadminctl[1841:10132] ----------------------------
Run MongoDB tests (ubuntu-latest)
No files were found with the provided path: coverage/lcov.info. No artifacts will be uploaded.
Run MongoDB tests (ubuntu-latest)
1 added, 0 removed; done.
Run MongoDB tests (ubuntu-latest)
rehash: warning: skipping ca-certificates.crt,it does not contain exactly one certificate or CRL
Run MongoDB tests (ubuntu-latest)
Updating certificates in /etc/ssl/certs...
Run MongoDB tests (ubuntu-latest)
WARNING: apt does not have a stable CLI interface. Use with caution in scripts.
Run Atlas tests
No files were found with the provided path: coverage/lcov.info. No artifacts will be uploaded.
Run Atlas tests
1 added, 0 removed; done.
Run Atlas tests
rehash: warning: skipping ca-certificates.crt,it does not contain exactly one certificate or CRL
Run Atlas tests
Updating certificates in /etc/ssl/certs...
Run Atlas tests
WARNING: apt does not have a stable CLI interface. Use with caution in scripts.
Run MongoDB tests (macos-latest)
The macos-latest label will migrate to macOS 15 beginning August 4, 2025. For more information see https://github.com/actions/runner-images/issues/12520
Run MongoDB tests (macos-latest)
The macos-latest label will migrate to macOS 15 beginning August 4, 2025. For more information see https://github.com/actions/runner-images/issues/12520

Artifacts

Produced during runtime
Name Size Digest
run-atlas-tests-permissions-384e2626390c7b2cbef176da924ed219
140 Bytes
sha256:bf3055cbfd40badb3e023ae284670715d966be647aa197f348aca63a6bfcf393
run-tests-permissions-67517cb514a3baaf049c3dc65e966cdf
140 Bytes
sha256:af118c3d972d644e7266ea2693735aba73b866664cf1d502b340a254bd49c6fc
run-tests-permissions-b61277e25c19c67ed9805f5c82f3a8fe
140 Bytes
sha256:c33616db7d4171d6a43e3cdf225142d6b3d320786d7bbe28b5ab46b144aeb8d1