Skip to content

Conversation

@linyuxi-0511
Copy link
Contributor

@linyuxi-0511 linyuxi-0511 commented Feb 6, 2025

Problem

We are mistakenly counting several client-side failures as service faults, which is affecting our availability metrics.

Solution

Exclude frequent client errors from faults, they should be errors.

there are multiple layers of errors, some of those are categorized to toolkit level errors, for example this part:

case MonthlyConversationLimitError.name:
case CodeIterationLimitError.name:
case PromptRefusalException.name:
case NoChangeRequiredException.name:

Others are defaulting to ToolkitError with messages.

Ideally we should add more error models to toolkit, but for now to make a quick fix, I just looked into the error msg, similar to what we do here:

e.message.includes('limit for number of iterations on a code generation'))


  • Treat all work as PUBLIC. Private feature/x branches will not be squash-merged at release time.
  • Your code changes must meet the guidelines in CONTRIBUTING.md.
  • License: I confirm that my contribution is made under the terms of the Apache 2.0 license.

@linyuxi-0511 linyuxi-0511 requested review from a team as code owners February 6, 2025 22:19
@jpinkney-aws jpinkney-aws changed the title Fix: filter out frequent client error from codegeneration telemetry fix(amazonq): filter out frequent client error from codegeneration telemetry Feb 10, 2025
@jpinkney-aws jpinkney-aws reopened this Feb 10, 2025
@github-actions
Copy link

  • This pull request implements a feat or fix, so it must include a changelog entry (unless the fix is for an unreleased feature). Review the changelog guidelines.
    • Note: beta or "experiment" features that have active users should announce fixes in the changelog.
    • If this is not a feature or fix, use an appropriate type from the title guidelines. For example, telemetry-only changes should use the telemetry type.

@jpinkney-aws jpinkney-aws merged commit f5bc8c2 into aws:master Feb 11, 2025
22 of 24 checks passed
s7ab059789 pushed a commit to s7ab059789/aws-toolkit-vscode that referenced this pull request Feb 19, 2025
…lemetry (aws#6521)

## Problem
We are mistakenly counting several client-side failures as service
faults, which is affecting our availability metrics.

## Solution
Exclude frequent client errors from faults, they should be errors.

there are multiple layers of errors, some of those are categorized to
toolkit level errors, for example this part:
```
case MonthlyConversationLimitError.name:
case CodeIterationLimitError.name:
case PromptRefusalException.name:
case NoChangeRequiredException.name:
```
Others are defaulting to ToolkitError with messages.

Ideally we should add more error models to toolkit, but for now to make
a quick fix, I just looked into the error msg, similar to what we do
here:
https://github.com/aws/aws-toolkit-vscode/blob/02d21a2fafc7479fb398ea078aa3e3adf7c62e7a/packages/core/src/amazonqFeatureDev/client/featureDev.ts#L198



---

- Treat all work as PUBLIC. Private `feature/x` branches will not be
squash-merged at release time.
- Your code changes must meet the guidelines in
[CONTRIBUTING.md](https://github.com/aws/aws-toolkit-vscode/blob/master/CONTRIBUTING.md#guidelines).
- License: I confirm that my contribution is made under the terms of the
Apache 2.0 license.

---------

Co-authored-by: Yuxin Lin <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants