chore: add missing expectations for accuracy tests #349
himanshusinghsaccuracy-tests.yml
on: pull_request
Run Accuracy Tests
4m 5s
Annotations
9 errors and 4 warnings
|
Run Accuracy Tests
Process completed with exit code 1.
|
|
tests/accuracy/untrustedData.test.ts > 'Azure - gpt-4o' > 'Create an aggregation that groups the…':
tests/accuracy/sdk/agent.ts#L68
AI_RetryError: Failed after 3 attempts. Last error: Requests to the ChatCompletions_Create Operation under Azure OpenAI API version 2024-12-01-preview have exceeded token rate limit of your current OpenAI S0 pricing tier. Please retry after 5 seconds. Please go here: https://aka.ms/oai/quotaincrease if you would like to further increase the default rate limit. For Free Account customers, upgrade to Pay as you Go here: https://aka.ms/429TrialUpgrade.
❯ _retryWithExponentialBackoff node_modules/ai/util/retry-with-exponential-backoff.ts:51:13
❯ fn node_modules/ai/core/generate-text/generate-text.ts:363:32
❯ node_modules/ai/core/telemetry/record-span.ts:18:22
❯ Object.prompt tests/accuracy/sdk/agent.ts:68:44
❯ tests/accuracy/sdk/describeAccuracyTests.ts:104:28
⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯
Serialized Error: { reason: 'maxRetriesExceeded', errors: [ { stack: 'AI_APICallError: Requests to the ChatCompletions_Create Operation under Azure OpenAI API version 2024-12-01-preview have exceeded token rate limit of your current OpenAI S0 pricing tier. Please retry after 11 seconds. Please go here: https://aka.ms/oai/quotaincrease if you would like to further increase the default rate limit. For Free Account customers, upgrade to Pay as you Go here: https://aka.ms/429TrialUpgrade.\n at file:///home/runner/work/mongodb-mcp-server/mongodb-mcp-server/node_modules/@ai-sdk/provider-utils/src/response-handler.ts:59:16\n at processTicksAndRejections (node:internal/process/task_queues:105:5)\n at postToApi (file:///home/runner/work/mongodb-mcp-server/mongodb-mcp-server/node_modules/@ai-sdk/provider-utils/src/post-to-api.ts:111:28)\n at OpenAIChatLanguageModel.doGenerate (file:///home/runner/work/mongodb-mcp-server/mongodb-mcp-server/node_modules/@ai-sdk/openai/src/openai-chat-language-model.ts:379:9)\n at fn (file:///home/runner/work/mongodb-mcp-server/mongodb-mcp-server/node_modules/ai/core/generate-text/generate-text.ts:407:30)\n at file:///home/runner/work/mongodb-mcp-server/mongodb-mcp-server/node_modules/ai/core/telemetry/record-span.ts:18:22\n at _retryWithExponentialBackoff (file:///home/runner/work/mongodb-mcp-server/mongodb-mcp-server/node_modules/ai/util/retry-with-exponential-backoff.ts:36:12)\n at fn (file:///home/runner/work/mongodb-mcp-server/mongodb-mcp-server/node_modules/ai/core/generate-text/generate-text.ts:363:32)\n at file:///home/runner/work/mongodb-mcp-server/mongodb-mcp-server/node_modules/ai/core/telemetry/record-span.ts:18:22\n at Object.prompt (/home/runner/work/mongodb-mcp-server/mongodb-mcp-server/tests/accuracy/sdk/agent.ts:68:44)', message: 'Requests to the ChatCompletions_Create Operation under Azure OpenAI API version 2024-12-01-preview have exceeded token rate limit of your current OpenAI S0 pricing tier. Please retry after 11 seconds. Please go here: https://aka.ms/oai/quotaincrease if you would like to further increase the default rate limit. For Free Account customers, upgrade to Pay as you Go here: https://aka.ms/429TrialUpgrade.', name: 'AI_APICallError', cause: undefined, url: 'https://gai-405.openai.azure.com/openai/deployments/gpt-4o/chat/completions?api-version=2024-12-01-preview', requestBodyValues: { model: 'gpt-4o', logit_bias: undefined, logprobs: undefined, top_logprobs: undefined, user: undefined, parallel_tool_calls: undefined, max_tokens: undefined, temperature: +0, top_p: undefined, frequency_penalty: undefined, presence_penalty: undefined, response_format: undefined, stop: undefined, seed: undefined, max_completion_tokens: undefined, store: undefined, metadata: undefined, prediction: undefined, reasoning_effort: undefined, messages: [ { role: 'system', content: 'The keywords "MUST", "MUST NOT", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119\nYou are an expert AI assistant with access to a set of tools for MongoDB database operations.\nYou MUST use the most relevant tool to answer the user\'s request\nWhen calling a tool, you MUST strictly follow its input s
|
|
tests/accuracy/untrustedData.test.ts > 'Azure - gpt-4o' > 'Read all documents in the \'support.t…':
tests/accuracy/sdk/agent.ts#L68
AI_RetryError: Failed after 3 attempts. Last error: Requests to the ChatCompletions_Create Operation under Azure OpenAI API version 2024-12-01-preview have exceeded token rate limit of your current OpenAI S0 pricing tier. Please retry after 11 seconds. Please go here: https://aka.ms/oai/quotaincrease if you would like to further increase the default rate limit. For Free Account customers, upgrade to Pay as you Go here: https://aka.ms/429TrialUpgrade.
❯ _retryWithExponentialBackoff node_modules/ai/util/retry-with-exponential-backoff.ts:51:13
❯ fn node_modules/ai/core/generate-text/generate-text.ts:363:32
❯ node_modules/ai/core/telemetry/record-span.ts:18:22
❯ Object.prompt tests/accuracy/sdk/agent.ts:68:44
❯ tests/accuracy/sdk/describeAccuracyTests.ts:104:28
⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯
Serialized Error: { reason: 'maxRetriesExceeded', errors: [ { stack: 'AI_APICallError: Requests to the ChatCompletions_Create Operation under Azure OpenAI API version 2024-12-01-preview have exceeded token rate limit of your current OpenAI S0 pricing tier. Please retry after 17 seconds. Please go here: https://aka.ms/oai/quotaincrease if you would like to further increase the default rate limit. For Free Account customers, upgrade to Pay as you Go here: https://aka.ms/429TrialUpgrade.\n at file:///home/runner/work/mongodb-mcp-server/mongodb-mcp-server/node_modules/@ai-sdk/provider-utils/src/response-handler.ts:59:16\n at processTicksAndRejections (node:internal/process/task_queues:105:5)\n at postToApi (file:///home/runner/work/mongodb-mcp-server/mongodb-mcp-server/node_modules/@ai-sdk/provider-utils/src/post-to-api.ts:111:28)\n at OpenAIChatLanguageModel.doGenerate (file:///home/runner/work/mongodb-mcp-server/mongodb-mcp-server/node_modules/@ai-sdk/openai/src/openai-chat-language-model.ts:379:9)\n at fn (file:///home/runner/work/mongodb-mcp-server/mongodb-mcp-server/node_modules/ai/core/generate-text/generate-text.ts:407:30)\n at file:///home/runner/work/mongodb-mcp-server/mongodb-mcp-server/node_modules/ai/core/telemetry/record-span.ts:18:22\n at _retryWithExponentialBackoff (file:///home/runner/work/mongodb-mcp-server/mongodb-mcp-server/node_modules/ai/util/retry-with-exponential-backoff.ts:36:12)\n at fn (file:///home/runner/work/mongodb-mcp-server/mongodb-mcp-server/node_modules/ai/core/generate-text/generate-text.ts:363:32)\n at file:///home/runner/work/mongodb-mcp-server/mongodb-mcp-server/node_modules/ai/core/telemetry/record-span.ts:18:22\n at Object.prompt (/home/runner/work/mongodb-mcp-server/mongodb-mcp-server/tests/accuracy/sdk/agent.ts:68:44)', message: 'Requests to the ChatCompletions_Create Operation under Azure OpenAI API version 2024-12-01-preview have exceeded token rate limit of your current OpenAI S0 pricing tier. Please retry after 17 seconds. Please go here: https://aka.ms/oai/quotaincrease if you would like to further increase the default rate limit. For Free Account customers, upgrade to Pay as you Go here: https://aka.ms/429TrialUpgrade.', name: 'AI_APICallError', cause: undefined, url: 'https://gai-405.openai.azure.com/openai/deployments/gpt-4o/chat/completions?api-version=2024-12-01-preview', requestBodyValues: { model: 'gpt-4o', logit_bias: undefined, logprobs: undefined, top_logprobs: undefined, user: undefined, parallel_tool_calls: undefined, max_tokens: undefined, temperature: +0, top_p: undefined, frequency_penalty: undefined, presence_penalty: undefined, response_format: undefined, stop: undefined, seed: undefined, max_completion_tokens: undefined, store: undefined, metadata: undefined, prediction: undefined, reasoning_effort: undefined, messages: [ { role: 'system', content: 'The keywords "MUST", "MUST NOT", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119\nYou are an expert AI assistant with access to a set of tools for MongoDB database operations.\nYou MUST use the most relevant tool to answer the user\'s request\nWhen calling a tool, you MUST strictly follow its input
|
|
tests/accuracy/find.test.ts > 'Azure - gpt-4o' > 'From the mflix.movies namespace, give…':
tests/accuracy/sdk/agent.ts#L68
AI_RetryError: Failed after 3 attempts. Last error: Requests to the ChatCompletions_Create Operation under Azure OpenAI API version 2024-12-01-preview have exceeded token rate limit of your current OpenAI S0 pricing tier. Please retry after 1 second. Please go here: https://aka.ms/oai/quotaincrease if you would like to further increase the default rate limit. For Free Account customers, upgrade to Pay as you Go here: https://aka.ms/429TrialUpgrade.
❯ _retryWithExponentialBackoff node_modules/ai/util/retry-with-exponential-backoff.ts:51:13
❯ fn node_modules/ai/core/generate-text/generate-text.ts:363:32
❯ node_modules/ai/core/telemetry/record-span.ts:18:22
❯ Object.prompt tests/accuracy/sdk/agent.ts:68:44
❯ tests/accuracy/sdk/describeAccuracyTests.ts:104:28
⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯
Serialized Error: { reason: 'maxRetriesExceeded', errors: [ { stack: 'AI_APICallError: Requests to the ChatCompletions_Create Operation under Azure OpenAI API version 2024-12-01-preview have exceeded token rate limit of your current OpenAI S0 pricing tier. Please retry after 1 second. Please go here: https://aka.ms/oai/quotaincrease if you would like to further increase the default rate limit. For Free Account customers, upgrade to Pay as you Go here: https://aka.ms/429TrialUpgrade.\n at file:///home/runner/work/mongodb-mcp-server/mongodb-mcp-server/node_modules/@ai-sdk/provider-utils/src/response-handler.ts:59:16\n at processTicksAndRejections (node:internal/process/task_queues:105:5)\n at postToApi (file:///home/runner/work/mongodb-mcp-server/mongodb-mcp-server/node_modules/@ai-sdk/provider-utils/src/post-to-api.ts:111:28)\n at OpenAIChatLanguageModel.doGenerate (file:///home/runner/work/mongodb-mcp-server/mongodb-mcp-server/node_modules/@ai-sdk/openai/src/openai-chat-language-model.ts:379:9)\n at fn (file:///home/runner/work/mongodb-mcp-server/mongodb-mcp-server/node_modules/ai/core/generate-text/generate-text.ts:407:30)\n at file:///home/runner/work/mongodb-mcp-server/mongodb-mcp-server/node_modules/ai/core/telemetry/record-span.ts:18:22\n at _retryWithExponentialBackoff (file:///home/runner/work/mongodb-mcp-server/mongodb-mcp-server/node_modules/ai/util/retry-with-exponential-backoff.ts:36:12)\n at fn (file:///home/runner/work/mongodb-mcp-server/mongodb-mcp-server/node_modules/ai/core/generate-text/generate-text.ts:363:32)\n at file:///home/runner/work/mongodb-mcp-server/mongodb-mcp-server/node_modules/ai/core/telemetry/record-span.ts:18:22\n at Object.prompt (/home/runner/work/mongodb-mcp-server/mongodb-mcp-server/tests/accuracy/sdk/agent.ts:68:44)', message: 'Requests to the ChatCompletions_Create Operation under Azure OpenAI API version 2024-12-01-preview have exceeded token rate limit of your current OpenAI S0 pricing tier. Please retry after 1 second. Please go here: https://aka.ms/oai/quotaincrease if you would like to further increase the default rate limit. For Free Account customers, upgrade to Pay as you Go here: https://aka.ms/429TrialUpgrade.', name: 'AI_APICallError', cause: undefined, url: 'https://gai-405.openai.azure.com/openai/deployments/gpt-4o/chat/completions?api-version=2024-12-01-preview', requestBodyValues: { model: 'gpt-4o', logit_bias: undefined, logprobs: undefined, top_logprobs: undefined, user: undefined, parallel_tool_calls: undefined, max_tokens: undefined, temperature: +0, top_p: undefined, frequency_penalty: undefined, presence_penalty: undefined, response_format: undefined, stop: undefined, seed: undefined, max_completion_tokens: undefined, store: undefined, metadata: undefined, prediction: undefined, reasoning_effort: undefined, messages: [ { role: 'system', content: 'The keywords "MUST", "MUST NOT", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119\nYou are an expert AI assistant with access to a set of tools for MongoDB database operations.\nYou MUST use the most relevant tool to answer the user\'s request\nWhen calling a tool, you MUST strictly follow its input schema
|
|
tests/accuracy/find.test.ts > 'Azure - gpt-4o' > 'Find all movies in \'mflix.movies\' c…':
tests/accuracy/sdk/agent.ts#L68
AI_RetryError: Failed after 3 attempts. Last error: Requests to the ChatCompletions_Create Operation under Azure OpenAI API version 2024-12-01-preview have exceeded token rate limit of your current OpenAI S0 pricing tier. Please retry after 1 second. Please go here: https://aka.ms/oai/quotaincrease if you would like to further increase the default rate limit. For Free Account customers, upgrade to Pay as you Go here: https://aka.ms/429TrialUpgrade.
❯ _retryWithExponentialBackoff node_modules/ai/util/retry-with-exponential-backoff.ts:51:13
❯ fn node_modules/ai/core/generate-text/generate-text.ts:363:32
❯ node_modules/ai/core/telemetry/record-span.ts:18:22
❯ Object.prompt tests/accuracy/sdk/agent.ts:68:44
❯ tests/accuracy/sdk/describeAccuracyTests.ts:104:28
⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯
Serialized Error: { reason: 'maxRetriesExceeded', errors: [ { stack: 'AI_APICallError: Requests to the ChatCompletions_Create Operation under Azure OpenAI API version 2024-12-01-preview have exceeded token rate limit of your current OpenAI S0 pricing tier. Please retry after 8 seconds. Please go here: https://aka.ms/oai/quotaincrease if you would like to further increase the default rate limit. For Free Account customers, upgrade to Pay as you Go here: https://aka.ms/429TrialUpgrade.\n at file:///home/runner/work/mongodb-mcp-server/mongodb-mcp-server/node_modules/@ai-sdk/provider-utils/src/response-handler.ts:59:16\n at processTicksAndRejections (node:internal/process/task_queues:105:5)\n at postToApi (file:///home/runner/work/mongodb-mcp-server/mongodb-mcp-server/node_modules/@ai-sdk/provider-utils/src/post-to-api.ts:111:28)\n at OpenAIChatLanguageModel.doGenerate (file:///home/runner/work/mongodb-mcp-server/mongodb-mcp-server/node_modules/@ai-sdk/openai/src/openai-chat-language-model.ts:379:9)\n at fn (file:///home/runner/work/mongodb-mcp-server/mongodb-mcp-server/node_modules/ai/core/generate-text/generate-text.ts:407:30)\n at file:///home/runner/work/mongodb-mcp-server/mongodb-mcp-server/node_modules/ai/core/telemetry/record-span.ts:18:22\n at _retryWithExponentialBackoff (file:///home/runner/work/mongodb-mcp-server/mongodb-mcp-server/node_modules/ai/util/retry-with-exponential-backoff.ts:36:12)\n at fn (file:///home/runner/work/mongodb-mcp-server/mongodb-mcp-server/node_modules/ai/core/generate-text/generate-text.ts:363:32)\n at file:///home/runner/work/mongodb-mcp-server/mongodb-mcp-server/node_modules/ai/core/telemetry/record-span.ts:18:22\n at Object.prompt (/home/runner/work/mongodb-mcp-server/mongodb-mcp-server/tests/accuracy/sdk/agent.ts:68:44)', message: 'Requests to the ChatCompletions_Create Operation under Azure OpenAI API version 2024-12-01-preview have exceeded token rate limit of your current OpenAI S0 pricing tier. Please retry after 8 seconds. Please go here: https://aka.ms/oai/quotaincrease if you would like to further increase the default rate limit. For Free Account customers, upgrade to Pay as you Go here: https://aka.ms/429TrialUpgrade.', name: 'AI_APICallError', cause: undefined, url: 'https://gai-405.openai.azure.com/openai/deployments/gpt-4o/chat/completions?api-version=2024-12-01-preview', requestBodyValues: { model: 'gpt-4o', logit_bias: undefined, logprobs: undefined, top_logprobs: undefined, user: undefined, parallel_tool_calls: undefined, max_tokens: undefined, temperature: +0, top_p: undefined, frequency_penalty: undefined, presence_penalty: undefined, response_format: undefined, stop: undefined, seed: undefined, max_completion_tokens: undefined, store: undefined, metadata: undefined, prediction: undefined, reasoning_effort: undefined, messages: [ { role: 'system', content: 'The keywords "MUST", "MUST NOT", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119\nYou are an expert AI assistant with access to a set of tools for MongoDB database operations.\nYou MUST use the most relevant tool to answer the user\'s request\nWhen calling a tool, you MUST strictly follow its input sche
|
|
tests/accuracy/find.test.ts > 'Azure - gpt-4o' > 'Find all the movies in \'mflix.movies…':
tests/accuracy/sdk/agent.ts#L68
AI_RetryError: Failed after 3 attempts. Last error: Requests to the ChatCompletions_Create Operation under Azure OpenAI API version 2024-12-01-preview have exceeded token rate limit of your current OpenAI S0 pricing tier. Please retry after 8 seconds. Please go here: https://aka.ms/oai/quotaincrease if you would like to further increase the default rate limit. For Free Account customers, upgrade to Pay as you Go here: https://aka.ms/429TrialUpgrade.
❯ _retryWithExponentialBackoff node_modules/ai/util/retry-with-exponential-backoff.ts:51:13
❯ fn node_modules/ai/core/generate-text/generate-text.ts:363:32
❯ node_modules/ai/core/telemetry/record-span.ts:18:22
❯ Object.prompt tests/accuracy/sdk/agent.ts:68:44
❯ tests/accuracy/sdk/describeAccuracyTests.ts:104:28
⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯
Serialized Error: { reason: 'maxRetriesExceeded', errors: [ { stack: 'AI_APICallError: Requests to the ChatCompletions_Create Operation under Azure OpenAI API version 2024-12-01-preview have exceeded token rate limit of your current OpenAI S0 pricing tier. Please retry after 14 seconds. Please go here: https://aka.ms/oai/quotaincrease if you would like to further increase the default rate limit. For Free Account customers, upgrade to Pay as you Go here: https://aka.ms/429TrialUpgrade.\n at file:///home/runner/work/mongodb-mcp-server/mongodb-mcp-server/node_modules/@ai-sdk/provider-utils/src/response-handler.ts:59:16\n at processTicksAndRejections (node:internal/process/task_queues:105:5)\n at postToApi (file:///home/runner/work/mongodb-mcp-server/mongodb-mcp-server/node_modules/@ai-sdk/provider-utils/src/post-to-api.ts:111:28)\n at OpenAIChatLanguageModel.doGenerate (file:///home/runner/work/mongodb-mcp-server/mongodb-mcp-server/node_modules/@ai-sdk/openai/src/openai-chat-language-model.ts:379:9)\n at fn (file:///home/runner/work/mongodb-mcp-server/mongodb-mcp-server/node_modules/ai/core/generate-text/generate-text.ts:407:30)\n at file:///home/runner/work/mongodb-mcp-server/mongodb-mcp-server/node_modules/ai/core/telemetry/record-span.ts:18:22\n at _retryWithExponentialBackoff (file:///home/runner/work/mongodb-mcp-server/mongodb-mcp-server/node_modules/ai/util/retry-with-exponential-backoff.ts:36:12)\n at fn (file:///home/runner/work/mongodb-mcp-server/mongodb-mcp-server/node_modules/ai/core/generate-text/generate-text.ts:363:32)\n at file:///home/runner/work/mongodb-mcp-server/mongodb-mcp-server/node_modules/ai/core/telemetry/record-span.ts:18:22\n at Object.prompt (/home/runner/work/mongodb-mcp-server/mongodb-mcp-server/tests/accuracy/sdk/agent.ts:68:44)', message: 'Requests to the ChatCompletions_Create Operation under Azure OpenAI API version 2024-12-01-preview have exceeded token rate limit of your current OpenAI S0 pricing tier. Please retry after 14 seconds. Please go here: https://aka.ms/oai/quotaincrease if you would like to further increase the default rate limit. For Free Account customers, upgrade to Pay as you Go here: https://aka.ms/429TrialUpgrade.', name: 'AI_APICallError', cause: undefined, url: 'https://gai-405.openai.azure.com/openai/deployments/gpt-4o/chat/completions?api-version=2024-12-01-preview', requestBodyValues: { model: 'gpt-4o', logit_bias: undefined, logprobs: undefined, top_logprobs: undefined, user: undefined, parallel_tool_calls: undefined, max_tokens: undefined, temperature: +0, top_p: undefined, frequency_penalty: undefined, presence_penalty: undefined, response_format: undefined, stop: undefined, seed: undefined, max_completion_tokens: undefined, store: undefined, metadata: undefined, prediction: undefined, reasoning_effort: undefined, messages: [ { role: 'system', content: 'The keywords "MUST", "MUST NOT", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119\nYou are an expert AI assistant with access to a set of tools for MongoDB database operations.\nYou MUST use the most relevant tool to answer the user\'s request\nWhen calling a tool, you MUST strictly follow its input s
|
|
tests/accuracy/export.test.ts > 'Azure - gpt-4o' > 'Export an aggregation that groups all…':
tests/accuracy/sdk/agent.ts#L68
AI_RetryError: Failed after 3 attempts. Last error: Requests to the ChatCompletions_Create Operation under Azure OpenAI API version 2024-12-01-preview have exceeded token rate limit of your current OpenAI S0 pricing tier. Please retry after 5 seconds. Please go here: https://aka.ms/oai/quotaincrease if you would like to further increase the default rate limit. For Free Account customers, upgrade to Pay as you Go here: https://aka.ms/429TrialUpgrade.
❯ _retryWithExponentialBackoff node_modules/ai/util/retry-with-exponential-backoff.ts:51:13
❯ fn node_modules/ai/core/generate-text/generate-text.ts:363:32
❯ node_modules/ai/core/telemetry/record-span.ts:18:22
❯ Object.prompt tests/accuracy/sdk/agent.ts:68:44
❯ tests/accuracy/sdk/describeAccuracyTests.ts:104:28
⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯
Serialized Error: { reason: 'maxRetriesExceeded', errors: [ { stack: 'AI_APICallError: Requests to the ChatCompletions_Create Operation under Azure OpenAI API version 2024-12-01-preview have exceeded token rate limit of your current OpenAI S0 pricing tier. Please retry after 11 seconds. Please go here: https://aka.ms/oai/quotaincrease if you would like to further increase the default rate limit. For Free Account customers, upgrade to Pay as you Go here: https://aka.ms/429TrialUpgrade.\n at file:///home/runner/work/mongodb-mcp-server/mongodb-mcp-server/node_modules/@ai-sdk/provider-utils/src/response-handler.ts:59:16\n at processTicksAndRejections (node:internal/process/task_queues:105:5)\n at postToApi (file:///home/runner/work/mongodb-mcp-server/mongodb-mcp-server/node_modules/@ai-sdk/provider-utils/src/post-to-api.ts:111:28)\n at OpenAIChatLanguageModel.doGenerate (file:///home/runner/work/mongodb-mcp-server/mongodb-mcp-server/node_modules/@ai-sdk/openai/src/openai-chat-language-model.ts:379:9)\n at fn (file:///home/runner/work/mongodb-mcp-server/mongodb-mcp-server/node_modules/ai/core/generate-text/generate-text.ts:407:30)\n at file:///home/runner/work/mongodb-mcp-server/mongodb-mcp-server/node_modules/ai/core/telemetry/record-span.ts:18:22\n at _retryWithExponentialBackoff (file:///home/runner/work/mongodb-mcp-server/mongodb-mcp-server/node_modules/ai/util/retry-with-exponential-backoff.ts:36:12)\n at fn (file:///home/runner/work/mongodb-mcp-server/mongodb-mcp-server/node_modules/ai/core/generate-text/generate-text.ts:363:32)\n at file:///home/runner/work/mongodb-mcp-server/mongodb-mcp-server/node_modules/ai/core/telemetry/record-span.ts:18:22\n at Object.prompt (/home/runner/work/mongodb-mcp-server/mongodb-mcp-server/tests/accuracy/sdk/agent.ts:68:44)', message: 'Requests to the ChatCompletions_Create Operation under Azure OpenAI API version 2024-12-01-preview have exceeded token rate limit of your current OpenAI S0 pricing tier. Please retry after 11 seconds. Please go here: https://aka.ms/oai/quotaincrease if you would like to further increase the default rate limit. For Free Account customers, upgrade to Pay as you Go here: https://aka.ms/429TrialUpgrade.', name: 'AI_APICallError', cause: undefined, url: 'https://gai-405.openai.azure.com/openai/deployments/gpt-4o/chat/completions?api-version=2024-12-01-preview', requestBodyValues: { model: 'gpt-4o', logit_bias: undefined, logprobs: undefined, top_logprobs: undefined, user: undefined, parallel_tool_calls: undefined, max_tokens: undefined, temperature: +0, top_p: undefined, frequency_penalty: undefined, presence_penalty: undefined, response_format: undefined, stop: undefined, seed: undefined, max_completion_tokens: undefined, store: undefined, metadata: undefined, prediction: undefined, reasoning_effort: undefined, messages: [ { role: 'system', content: 'The keywords "MUST", "MUST NOT", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119\nYou are an expert AI assistant with access to a set of tools for MongoDB database operations.\nYou MUST use the most relevant tool to answer the user\'s request\nWhen calling a tool, you MUST strictly follow its input s
|
|
tests/accuracy/export.test.ts > 'Azure - gpt-4o' > 'From the mflix.movies namespace, expo…':
tests/accuracy/sdk/agent.ts#L68
AI_RetryError: Failed after 3 attempts. Last error: Requests to the ChatCompletions_Create Operation under Azure OpenAI API version 2024-12-01-preview have exceeded token rate limit of your current OpenAI S0 pricing tier. Please retry after 11 seconds. Please go here: https://aka.ms/oai/quotaincrease if you would like to further increase the default rate limit. For Free Account customers, upgrade to Pay as you Go here: https://aka.ms/429TrialUpgrade.
❯ _retryWithExponentialBackoff node_modules/ai/util/retry-with-exponential-backoff.ts:51:13
❯ fn node_modules/ai/core/generate-text/generate-text.ts:363:32
❯ node_modules/ai/core/telemetry/record-span.ts:18:22
❯ Object.prompt tests/accuracy/sdk/agent.ts:68:44
❯ tests/accuracy/sdk/describeAccuracyTests.ts:104:28
⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯
Serialized Error: { reason: 'maxRetriesExceeded', errors: [ { stack: 'AI_APICallError: Requests to the ChatCompletions_Create Operation under Azure OpenAI API version 2024-12-01-preview have exceeded token rate limit of your current OpenAI S0 pricing tier. Please retry after 18 seconds. Please go here: https://aka.ms/oai/quotaincrease if you would like to further increase the default rate limit. For Free Account customers, upgrade to Pay as you Go here: https://aka.ms/429TrialUpgrade.\n at file:///home/runner/work/mongodb-mcp-server/mongodb-mcp-server/node_modules/@ai-sdk/provider-utils/src/response-handler.ts:59:16\n at processTicksAndRejections (node:internal/process/task_queues:105:5)\n at postToApi (file:///home/runner/work/mongodb-mcp-server/mongodb-mcp-server/node_modules/@ai-sdk/provider-utils/src/post-to-api.ts:111:28)\n at OpenAIChatLanguageModel.doGenerate (file:///home/runner/work/mongodb-mcp-server/mongodb-mcp-server/node_modules/@ai-sdk/openai/src/openai-chat-language-model.ts:379:9)\n at fn (file:///home/runner/work/mongodb-mcp-server/mongodb-mcp-server/node_modules/ai/core/generate-text/generate-text.ts:407:30)\n at file:///home/runner/work/mongodb-mcp-server/mongodb-mcp-server/node_modules/ai/core/telemetry/record-span.ts:18:22\n at _retryWithExponentialBackoff (file:///home/runner/work/mongodb-mcp-server/mongodb-mcp-server/node_modules/ai/util/retry-with-exponential-backoff.ts:36:12)\n at fn (file:///home/runner/work/mongodb-mcp-server/mongodb-mcp-server/node_modules/ai/core/generate-text/generate-text.ts:363:32)\n at file:///home/runner/work/mongodb-mcp-server/mongodb-mcp-server/node_modules/ai/core/telemetry/record-span.ts:18:22\n at Object.prompt (/home/runner/work/mongodb-mcp-server/mongodb-mcp-server/tests/accuracy/sdk/agent.ts:68:44)', message: 'Requests to the ChatCompletions_Create Operation under Azure OpenAI API version 2024-12-01-preview have exceeded token rate limit of your current OpenAI S0 pricing tier. Please retry after 18 seconds. Please go here: https://aka.ms/oai/quotaincrease if you would like to further increase the default rate limit. For Free Account customers, upgrade to Pay as you Go here: https://aka.ms/429TrialUpgrade.', name: 'AI_APICallError', cause: undefined, url: 'https://gai-405.openai.azure.com/openai/deployments/gpt-4o/chat/completions?api-version=2024-12-01-preview', requestBodyValues: { model: 'gpt-4o', logit_bias: undefined, logprobs: undefined, top_logprobs: undefined, user: undefined, parallel_tool_calls: undefined, max_tokens: undefined, temperature: +0, top_p: undefined, frequency_penalty: undefined, presence_penalty: undefined, response_format: undefined, stop: undefined, seed: undefined, max_completion_tokens: undefined, store: undefined, metadata: undefined, prediction: undefined, reasoning_effort: undefined, messages: [ { role: 'system', content: 'The keywords "MUST", "MUST NOT", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119\nYou are an expert AI assistant with access to a set of tools for MongoDB database operations.\nYou MUST use the most relevant tool to answer the user\'s request\nWhen calling a tool, you MUST strictly follow its input
|
|
tests/accuracy/explain.test.ts > 'Azure - gpt-4o' > 'Will aggregating documents, where rel…':
tests/accuracy/sdk/agent.ts#L68
AI_RetryError: Failed after 3 attempts. Last error: Requests to the ChatCompletions_Create Operation under Azure OpenAI API version 2024-12-01-preview have exceeded token rate limit of your current OpenAI S0 pricing tier. Please retry after 1 second. Please go here: https://aka.ms/oai/quotaincrease if you would like to further increase the default rate limit. For Free Account customers, upgrade to Pay as you Go here: https://aka.ms/429TrialUpgrade.
❯ _retryWithExponentialBackoff node_modules/ai/util/retry-with-exponential-backoff.ts:51:13
❯ fn node_modules/ai/core/generate-text/generate-text.ts:363:32
❯ node_modules/ai/core/telemetry/record-span.ts:18:22
❯ Object.prompt tests/accuracy/sdk/agent.ts:68:44
❯ tests/accuracy/sdk/describeAccuracyTests.ts:104:28
⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯
Serialized Error: { reason: 'maxRetriesExceeded', errors: [ { stack: 'AI_APICallError: Requests to the ChatCompletions_Create Operation under Azure OpenAI API version 2024-12-01-preview have exceeded token rate limit of your current OpenAI S0 pricing tier. Please retry after 1 second. Please go here: https://aka.ms/oai/quotaincrease if you would like to further increase the default rate limit. For Free Account customers, upgrade to Pay as you Go here: https://aka.ms/429TrialUpgrade.\n at file:///home/runner/work/mongodb-mcp-server/mongodb-mcp-server/node_modules/@ai-sdk/provider-utils/src/response-handler.ts:59:16\n at processTicksAndRejections (node:internal/process/task_queues:105:5)\n at postToApi (file:///home/runner/work/mongodb-mcp-server/mongodb-mcp-server/node_modules/@ai-sdk/provider-utils/src/post-to-api.ts:111:28)\n at OpenAIChatLanguageModel.doGenerate (file:///home/runner/work/mongodb-mcp-server/mongodb-mcp-server/node_modules/@ai-sdk/openai/src/openai-chat-language-model.ts:379:9)\n at fn (file:///home/runner/work/mongodb-mcp-server/mongodb-mcp-server/node_modules/ai/core/generate-text/generate-text.ts:407:30)\n at file:///home/runner/work/mongodb-mcp-server/mongodb-mcp-server/node_modules/ai/core/telemetry/record-span.ts:18:22\n at _retryWithExponentialBackoff (file:///home/runner/work/mongodb-mcp-server/mongodb-mcp-server/node_modules/ai/util/retry-with-exponential-backoff.ts:36:12)\n at fn (file:///home/runner/work/mongodb-mcp-server/mongodb-mcp-server/node_modules/ai/core/generate-text/generate-text.ts:363:32)\n at file:///home/runner/work/mongodb-mcp-server/mongodb-mcp-server/node_modules/ai/core/telemetry/record-span.ts:18:22\n at Object.prompt (/home/runner/work/mongodb-mcp-server/mongodb-mcp-server/tests/accuracy/sdk/agent.ts:68:44)', message: 'Requests to the ChatCompletions_Create Operation under Azure OpenAI API version 2024-12-01-preview have exceeded token rate limit of your current OpenAI S0 pricing tier. Please retry after 1 second. Please go here: https://aka.ms/oai/quotaincrease if you would like to further increase the default rate limit. For Free Account customers, upgrade to Pay as you Go here: https://aka.ms/429TrialUpgrade.', name: 'AI_APICallError', cause: undefined, url: 'https://gai-405.openai.azure.com/openai/deployments/gpt-4o/chat/completions?api-version=2024-12-01-preview', requestBodyValues: { model: 'gpt-4o', logit_bias: undefined, logprobs: undefined, top_logprobs: undefined, user: undefined, parallel_tool_calls: undefined, max_tokens: undefined, temperature: +0, top_p: undefined, frequency_penalty: undefined, presence_penalty: undefined, response_format: undefined, stop: undefined, seed: undefined, max_completion_tokens: undefined, store: undefined, metadata: undefined, prediction: undefined, reasoning_effort: undefined, messages: [ { role: 'system', content: 'The keywords "MUST", "MUST NOT", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119\nYou are an expert AI assistant with access to a set of tools for MongoDB database operations.\nYou MUST use the most relevant tool to answer the user\'s request\nWhen calling a tool, you MUST strictly follow its input schema
|
|
Run Accuracy Tests
1 added, 0 removed; done.
|
|
Run Accuracy Tests
rehash: warning: skipping ca-certificates.crt,it does not contain exactly one certificate or CRL
|
|
Run Accuracy Tests
Updating certificates in /etc/ssl/certs...
|
|
Run Accuracy Tests
WARNING: apt does not have a stable CLI interface. Use with caution in scripts.
|
Artifacts
Produced during runtime
| Name | Size | Digest | |
|---|---|---|---|
|
accuracy-test-summary
|
31.3 KB |
sha256:408f3665c1f9a42a5494b41073d8bc36340e4f999f6ed6e57e0b971e4524e007
|
|
|
run-accuracy-tests-permissions-5958b618f6c251dff5c4133b42197122
|
140 Bytes |
sha256:a56e2ffa0bae5099bfef393f0872c756d38361a75297df5c77314fba5a76fa8d
|
|