Skip to content

chore: retry the whole test with fixtures in case of error (#684) #601

chore: retry the whole test with fixtures in case of error (#684)

chore: retry the whole test with fixtures in case of error (#684) #601

Triggered via push October 24, 2025 10:30
Status Failure
Total duration 10m 43s
Artifacts 2
Run Accuracy Tests
10m 30s
Run Accuracy Tests
Fit to window
Zoom out
Zoom in

Annotations

3 errors and 3 warnings
Run Accuracy Tests
Process completed with exit code 1.
tests/accuracy/export.test.ts > 'Azure - gpt-4o' > 'Export all the movies in \'mflix.movi…': tests/accuracy/sdk/agent.ts#L68
AI_RetryError: Failed after 3 attempts. Last error: Your requests to gpt-4o for gpt-4o in South Central US have exceeded the token rate limit for your current OpenAI S0 pricing tier. This request was for ChatCompletions_Create under Azure OpenAI API version 2024-12-01-preview. Please retry after 1 second. To increase your default rate limit, visit: https://aka.ms/oai/quotaincrease. ❯ _retryWithExponentialBackoff node_modules/ai/src/util/retry-with-exponential-backoff.ts:111:13 ❯ fn node_modules/ai/src/generate-text/generate-text.ts:345:34 ❯ node_modules/ai/src/telemetry/record-span.ts:18:22 ❯ generateText node_modules/ai/src/generate-text/generate-text.ts:281:12 ❯ Object.prompt tests/accuracy/sdk/agent.ts:68:44 ❯ tests/accuracy/sdk/describeAccuracyTests.ts:124:28 ⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯ Serialized Error: { reason: 'maxRetriesExceeded', errors: [ { stack: 'AI_APICallError: Your requests to gpt-4o for gpt-4o in South Central US have exceeded the token rate limit for your current OpenAI S0 pricing tier. This request was for ChatCompletions_Create under Azure OpenAI API version 2024-12-01-preview. Please retry after 1 second. To increase your default rate limit, visit: https://aka.ms/oai/quotaincrease.\n at file:///home/runner/work/mongodb-mcp-server/mongodb-mcp-server/node_modules/@ai-sdk/provider-utils/src/response-handler.ts:57:16\n at processTicksAndRejections (node:internal/process/task_queues:105:5)\n at postToApi (file:///home/runner/work/mongodb-mcp-server/mongodb-mcp-server/node_modules/@ai-sdk/provider-utils/src/post-to-api.ts:118:28)\n at OpenAIChatLanguageModel.doGenerate (file:///home/runner/work/mongodb-mcp-server/mongodb-mcp-server/node_modules/@ai-sdk/openai/src/chat/openai-chat-language-model.ts:321:9)\n at fn (file:///home/runner/work/mongodb-mcp-server/mongodb-mcp-server/node_modules/ai/src/generate-text/generate-text.ts:389:32)\n at file:///home/runner/work/mongodb-mcp-server/mongodb-mcp-server/node_modules/ai/src/telemetry/record-span.ts:18:22\n at _retryWithExponentialBackoff (file:///home/runner/work/mongodb-mcp-server/mongodb-mcp-server/node_modules/ai/src/util/retry-with-exponential-backoff.ts:96:12)\n at fn (file:///home/runner/work/mongodb-mcp-server/mongodb-mcp-server/node_modules/ai/src/generate-text/generate-text.ts:345:34)\n at file:///home/runner/work/mongodb-mcp-server/mongodb-mcp-server/node_modules/ai/src/telemetry/record-span.ts:18:22\n at generateText (file:///home/runner/work/mongodb-mcp-server/mongodb-mcp-server/node_modules/ai/src/generate-text/generate-text.ts:281:12)', message: 'Your requests to gpt-4o for gpt-4o in South Central US have exceeded the token rate limit for your current OpenAI S0 pricing tier. This request was for ChatCompletions_Create under Azure OpenAI API version 2024-12-01-preview. Please retry after 1 second. To increase your default rate limit, visit: https://aka.ms/oai/quotaincrease.', name: 'AI_APICallError', cause: undefined, url: 'https://gai-405.openai.azure.com/openai//deployments/gpt-4o/chat/completions?api-version=2024-12-01-preview', requestBodyValues: { model: 'gpt-4o', logit_bias: undefined, logprobs: undefined, top_logprobs: undefined, user: undefined, parallel_tool_calls: undefined, max_tokens: undefined, temperature: undefined, top_p: undefined, frequency_penalty: undefined, presence_penalty: undefined, response_format: undefined, stop: undefined, seed: undefined, verbosity: undefined, max_completion_tokens: undefined, store: undefined, metadata: undefined, prediction: undefined, reasoning_effort: undefined, service_tier: undefined, prompt_cache_key: undefined, safety_identifier: undefined, messages: [ { role: 'system', content: 'The keywords "MUST", "MUST NOT", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119\nYou are an expert AI assistant with access to a set of tools for MongoDB database operations.\nYou MUST use the most relevant tool to answer the user\'s request\nWhen calling a tool, you MUST strictly f
tests/accuracy/dropIndex.test.ts > 'Azure - gpt-4o' > 'First create a text index on field \'…': tests/accuracy/sdk/agent.ts#L68
AI_RetryError: Failed after 3 attempts. Last error: Your requests to gpt-4o for gpt-4o in South Central US have exceeded the token rate limit for your current OpenAI S0 pricing tier. This request was for ChatCompletions_Create under Azure OpenAI API version 2024-12-01-preview. Please retry after 2 seconds. To increase your default rate limit, visit: https://aka.ms/oai/quotaincrease. ❯ _retryWithExponentialBackoff node_modules/ai/src/util/retry-with-exponential-backoff.ts:111:13 ❯ fn node_modules/ai/src/generate-text/generate-text.ts:345:34 ❯ node_modules/ai/src/telemetry/record-span.ts:18:22 ❯ generateText node_modules/ai/src/generate-text/generate-text.ts:281:12 ❯ Object.prompt tests/accuracy/sdk/agent.ts:68:44 ❯ tests/accuracy/sdk/describeAccuracyTests.ts:124:28 ⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯ Serialized Error: { reason: 'maxRetriesExceeded', errors: [ { stack: 'AI_APICallError: Your requests to gpt-4o for gpt-4o in South Central US have exceeded the token rate limit for your current OpenAI S0 pricing tier. This request was for ChatCompletions_Create under Azure OpenAI API version 2024-12-01-preview. Please retry after 1 second. To increase your default rate limit, visit: https://aka.ms/oai/quotaincrease.\n at file:///home/runner/work/mongodb-mcp-server/mongodb-mcp-server/node_modules/@ai-sdk/provider-utils/src/response-handler.ts:57:16\n at processTicksAndRejections (node:internal/process/task_queues:105:5)\n at postToApi (file:///home/runner/work/mongodb-mcp-server/mongodb-mcp-server/node_modules/@ai-sdk/provider-utils/src/post-to-api.ts:118:28)\n at OpenAIChatLanguageModel.doGenerate (file:///home/runner/work/mongodb-mcp-server/mongodb-mcp-server/node_modules/@ai-sdk/openai/src/chat/openai-chat-language-model.ts:321:9)\n at fn (file:///home/runner/work/mongodb-mcp-server/mongodb-mcp-server/node_modules/ai/src/generate-text/generate-text.ts:389:32)\n at file:///home/runner/work/mongodb-mcp-server/mongodb-mcp-server/node_modules/ai/src/telemetry/record-span.ts:18:22\n at _retryWithExponentialBackoff (file:///home/runner/work/mongodb-mcp-server/mongodb-mcp-server/node_modules/ai/src/util/retry-with-exponential-backoff.ts:96:12)\n at fn (file:///home/runner/work/mongodb-mcp-server/mongodb-mcp-server/node_modules/ai/src/generate-text/generate-text.ts:345:34)\n at file:///home/runner/work/mongodb-mcp-server/mongodb-mcp-server/node_modules/ai/src/telemetry/record-span.ts:18:22\n at generateText (file:///home/runner/work/mongodb-mcp-server/mongodb-mcp-server/node_modules/ai/src/generate-text/generate-text.ts:281:12)', message: 'Your requests to gpt-4o for gpt-4o in South Central US have exceeded the token rate limit for your current OpenAI S0 pricing tier. This request was for ChatCompletions_Create under Azure OpenAI API version 2024-12-01-preview. Please retry after 1 second. To increase your default rate limit, visit: https://aka.ms/oai/quotaincrease.', name: 'AI_APICallError', cause: undefined, url: 'https://gai-405.openai.azure.com/openai//deployments/gpt-4o/chat/completions?api-version=2024-12-01-preview', requestBodyValues: { model: 'gpt-4o', logit_bias: undefined, logprobs: undefined, top_logprobs: undefined, user: undefined, parallel_tool_calls: undefined, max_tokens: undefined, temperature: undefined, top_p: undefined, frequency_penalty: undefined, presence_penalty: undefined, response_format: undefined, stop: undefined, seed: undefined, verbosity: undefined, max_completion_tokens: undefined, store: undefined, metadata: undefined, prediction: undefined, reasoning_effort: undefined, service_tier: undefined, prompt_cache_key: undefined, safety_identifier: undefined, messages: [ { role: 'system', content: 'The keywords "MUST", "MUST NOT", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119\nYou are an expert AI assistant with access to a set of tools for MongoDB database operations.\nYou MUST use the most relevant tool to answer the user\'s request\nWhen calling a tool, you MUST strictly
Run Accuracy Tests
1 added, 0 removed; done.
Run Accuracy Tests
rehash: warning: skipping ca-certificates.crt,it does not contain exactly one certificate or CRL
Run Accuracy Tests
Updating certificates in /etc/ssl/certs...

Artifacts

Produced during runtime
Name Size Digest
accuracy-test-summary
46.4 KB
sha256:aef7edbd7e98cc2608ddada972dbaf8b90ea1624c1739e86965bb0fde7b7e486
run-accuracy-tests-permissions-22e58b36b9291daee5ec76021a983f89
140 Bytes
sha256:bb4b5bc93d9c90223a4297a7e4d57096ebe499ccb784abfe40d47b08f850df15