refactor(ec2): improve error information in telemetry. #6624
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
justinmk3
reviewed
Feb 19, 2025
justinmk3
approved these changes
Feb 19, 2025
Co-authored-by: Justin M. Keyes <[email protected]>
jpinkney-aws
approved these changes
Feb 19, 2025
jpinkney-aws
pushed a commit
to jpinkney-aws/aws-toolkit-vscode
that referenced
this pull request
Feb 24, 2025
## Problem Within telemetry, there are a large group of fatal errors within EC2 Connect being grouped together under the same general error code. Additionally, there is very little information in the error message to determine the source of this error. Specifically, it appears the the test SSM connection done before attempting to pass the connection to VS Code, implemented [here](https://github.com/aws/aws-toolkit-vscode/blob/b9af56c3097242fb796995479f864d95098bf713/packages/core/src/shared/extensions/ssh.ts#L133-L155), can fail, and gives us a general error code with little information about its root cause. ## Solution - Add a specific error code for this connection type. (reused an old unused one) - Pipe the error from the child process into the error message so that we get it in telemetry. - Add a docstring to test connect function, since its not obvious whats happening as it involves both ssh and ssm. Example Error: <img width="465" alt="image" src="https://github.com/user-attachments/assets/8b27746d-9291-48a4-a125-dcea761a2c13" /> --- - Treat all work as PUBLIC. Private `feature/x` branches will not be squash-merged at release time. - Your code changes must meet the guidelines in [CONTRIBUTING.md](https://github.com/aws/aws-toolkit-vscode/blob/master/CONTRIBUTING.md#guidelines). - License: I confirm that my contribution is made under the terms of the Apache 2.0 license. --------- Co-authored-by: Justin M. Keyes <[email protected]>
laileni-aws
added a commit
to laileni-aws/aws-toolkit-vscode
that referenced
this pull request
Feb 28, 2025
* Release 1.48.0 * Release 3.47.0 * Update version to snapshot version: 3.48.0-SNAPSHOT * Update version to snapshot version: 1.49.0-SNAPSHOT * deps(build): update esbuild-loader to 4.3.0 (aws#6622) ## Problem - we have a couple moderate vulnerabilities due to an old version of esbuild-loader ## Solution - update esbuild-loader + other npm packages that have vulnerabilities ## Notes - In the [3.0.0 esbuild-loader version](https://github.com/privatenumber/esbuild-loader/releases/tag/v3.0.0) ESBuildMinifyPlugin was re-named to EsbuildPlugin, it still does the minification by default though --- - Treat all work as PUBLIC. Private `feature/x` branches will not be squash-merged at release time. - Your code changes must meet the guidelines in [CONTRIBUTING.md](https://github.com/aws/aws-toolkit-vscode/blob/master/CONTRIBUTING.md#guidelines). - License: I confirm that my contribution is made under the terms of the Apache 2.0 license. * fix(amazonq): update capability card text for /test (aws#6646) ## Problem - Capability card text should be updated ## Solution - Update capability card with correct text  --- - Treat all work as PUBLIC. Private `feature/x` branches will not be squash-merged at release time. - Your code changes must meet the guidelines in [CONTRIBUTING.md](https://github.com/aws/aws-toolkit-vscode/blob/master/CONTRIBUTING.md#guidelines). - License: I confirm that my contribution is made under the terms of the Apache 2.0 license. * refactor(util): tryRun() can take a regex aws#6640 ## Problem This is aimed at fixing the following ec2 bug. Attempting to generate the ssh keys via tryRun can log a failure, when there is no failure. Looking at https://github.com/aws/aws-toolkit-vscode/blob/35502be238b1bf7c3ff73e44df8d077a6d32aa85/packages/core/src/awsService/ec2/sshKeyPair.ts#L78-L86 We use `tryRun` to detect if the output contains "unknown key type". This allows us to detect when a certain key type such as RSA or ed25519 is not supported on the local machine. Detecting if the key generated successfully is a harder problem, as the output did not contain a consistent message. However, it results in `tryRun` logging a failure to find "unknown key type" in the output, which means failed to find the expected text in the output, but still generated the key. This is then logged as a failure, but is functionally a success. This is very confusing behavior. The root of this problem is we are trying to use `tryRun` in a way it doesn't naturally support. Rather than use it awkwardly, we can extend it. ## Solution - add support for regex in tryRun New logs: ``` 2025-02-20 12:07:01.624 [info] tryRun: ok: PID 22605: [ssh-keygen -t ed25519 -N -q -f /Users/hkobew/Library/Application Support/Code/User/globalStorage/amazonwebservices.aws-toolkit-vscode/aws-ec2-key] { exitCode: 0, stdout: '/Users/hkobew/Library/Application Support/Code/User/globalStorage/amazonwebservices.aws-toolkit-vscode/aws-ec2-key already exists.\n' + 'Overwrite (y/n)?', stderr: '', error: undefined, signal: undefined } ``` * test(amazon): add more debugging information to inline tests (aws#6644) ## Problem - we have no way to get any information on why an inline test was flaky. It might be because telemetry events are being out of order? - Since fib doesn't have spaces in the contents it's autocompleting on the very first line which may be causing the backend to return no results ## Solution - add more debugging information to understand why there was a failure - add spaces to the test fib function - consider a suggestion shown if the recommendation handler says its visible or if its state has been declared as 'Shown' --- - Treat all work as PUBLIC. Private `feature/x` branches will not be squash-merged at release time. - Your code changes must meet the guidelines in [CONTRIBUTING.md](https://github.com/aws/aws-toolkit-vscode/blob/master/CONTRIBUTING.md#guidelines). - License: I confirm that my contribution is made under the terms of the Apache 2.0 license. * test(amazonq): Add e2e tests for /doc to cover different UX flow (aws#6631) ## Problem ## Solution Add e2e tests to cover: 1. /doc update document flow. 2. Create document in a subfolder. 3. Cancel operation. 4. Discard or accept operation. --- - Treat all work as PUBLIC. Private `feature/x` branches will not be squash-merged at release time. - Your code changes must meet the guidelines in [CONTRIBUTING.md](https://github.com/aws/aws-toolkit-vscode/blob/master/CONTRIBUTING.md#guidelines). - License: I confirm that my contribution is made under the terms of the Apache 2.0 license. --------- Co-authored-by: Avi Alpert <[email protected]> * fix(ec2): increase robustness of checking ssm agent ping status. (aws#6621) ## Problem As part of the EC2 Connect process, we check if the SSM agent is pingable on the target instance. This is done here: https://github.com/aws/aws-toolkit-vscode/blob/b9af56c3097242fb796995479f864d95098bf713/packages/core/src/awsService/ec2/model.ts#L153-L162 This check can fail, and is currently does fail a decent amount of the time in telemetry. ## Solution - wrap the check in a `waitUntil`, retrying every half second up to 10 times. --- - Treat all work as PUBLIC. Private `feature/x` branches will not be squash-merged at release time. - Your code changes must meet the guidelines in [CONTRIBUTING.md](https://github.com/aws/aws-toolkit-vscode/blob/master/CONTRIBUTING.md#guidelines). - License: I confirm that my contribution is made under the terms of the Apache 2.0 license. * refactor(ec2): improve error information in telemetry. (aws#6624) ## Problem Within telemetry, there are a large group of fatal errors within EC2 Connect being grouped together under the same general error code. Additionally, there is very little information in the error message to determine the source of this error. Specifically, it appears the the test SSM connection done before attempting to pass the connection to VS Code, implemented [here](https://github.com/aws/aws-toolkit-vscode/blob/b9af56c3097242fb796995479f864d95098bf713/packages/core/src/shared/extensions/ssh.ts#L133-L155), can fail, and gives us a general error code with little information about its root cause. ## Solution - Add a specific error code for this connection type. (reused an old unused one) - Pipe the error from the child process into the error message so that we get it in telemetry. - Add a docstring to test connect function, since its not obvious whats happening as it involves both ssh and ssm. Example Error: <img width="465" alt="image" src="https://github.com/user-attachments/assets/8b27746d-9291-48a4-a125-dcea761a2c13" /> --- - Treat all work as PUBLIC. Private `feature/x` branches will not be squash-merged at release time. - Your code changes must meet the guidelines in [CONTRIBUTING.md](https://github.com/aws/aws-toolkit-vscode/blob/master/CONTRIBUTING.md#guidelines). - License: I confirm that my contribution is made under the terms of the Apache 2.0 license. --------- Co-authored-by: Justin M. Keyes <[email protected]> * config(amazonq-chat): disable and remove implicit workspace context experiment (aws#6630) ## Problem Remove as experiment has been disable from the service side, no need to collect workspace context if service doesn't actually utilize it. ## Solution --- - Treat all work as PUBLIC. Private `feature/x` branches will not be squash-merged at release time. - Your code changes must meet the guidelines in [CONTRIBUTING.md](https://github.com/aws/aws-toolkit-vscode/blob/master/CONTRIBUTING.md#guidelines). - License: I confirm that my contribution is made under the terms of the Apache 2.0 license. * fix(amazonq): ignored lines should not show up in /review scan issues * Revert "deps(build): update esbuild-loader to 4.3.0 (aws#6622)" (aws#6660) This reverts commit 8d2d335. ## Problem Upgrading esbuild-loader broke loading mynah-ui ## Solution Revert for now --- - Treat all work as PUBLIC. Private `feature/x` branches will not be squash-merged at release time. - Your code changes must meet the guidelines in [CONTRIBUTING.md](https://github.com/aws/aws-toolkit-vscode/blob/master/CONTRIBUTING.md#guidelines). - License: I confirm that my contribution is made under the terms of the Apache 2.0 license. * fix(amazonq): skip tests locally (aws#6658) ## Problem When users choose to skip tests, we were skipping them on our server by passing in their choice to our backend, but we weren't also skipping them locally. ## Solution Tell Maven to use `-DskipTests` --- - Treat all work as PUBLIC. Private `feature/x` branches will not be squash-merged at release time. - Your code changes must meet the guidelines in [CONTRIBUTING.md](https://github.com/aws/aws-toolkit-vscode/blob/master/CONTRIBUTING.md#guidelines). - License: I confirm that my contribution is made under the terms of the Apache 2.0 license. Co-authored-by: David Hasani <[email protected]> * fix(amazonq): fix uploading file method error handling for /doc * "Added Stub to tests to make them work" * chore: update according to the comments * Adding additional patterns for file filtering for test generation. * revamp test to not stub files to better simulate real environment * Sorting workspace folders in ziputils for test generation usecase. * removed debugging comment * correction for workspace folder for usecase. * Revamp test to unaccount for windows bug to solve later * Fixed syntax errors and updated the test generation ignore pattern list. * Correction for files ignore patterns * Fixed file to be the core folder * feat(toolkit): Expand data source and output options for Distributed Map in ASL language schema (aws#6680) ## Problem ## Solution Upgrade `amazon-states-language-service` to 1.16.1. The new version expands data source and output options for Distributed Map in ASL. Feature release post: https://aws.amazon.com/about-aws/whats-new/2025/02/aws-step-functions-data-source-output-option-distributed-map/ ---  - Treat all work as PUBLIC. Private `feature/x` branches will not be squash-merged at release time. - Your code changes must meet the guidelines in [CONTRIBUTING.md](https://github.com/aws/aws-toolkit-vscode/blob/master/CONTRIBUTING.md#guidelines). - License: I confirm that my contribution is made under the terms of the Apache 2.0 license. Co-authored-by: Silvia Chen <[email protected]> * fix(amazonq): Test generation shows incorrect view diff view (aws#6676) ## Problem - When users attempt to generate unit test cases using the /test command for packages other than the primary one in the workspace, they encounter a diff view suggesting the absence of an existing file. However, this is incorrect as there is already an existing test file in the package. ## Solution - Fixed this issue by getting the project path from the target file. --- - Treat all work as PUBLIC. Private `feature/x` branches will not be squash-merged at release time. - Your code changes must meet the guidelines in [CONTRIBUTING.md](https://github.com/aws/aws-toolkit-vscode/blob/master/CONTRIBUTING.md#guidelines). - License: I confirm that my contribution is made under the terms of the Apache 2.0 license. * fix(amazonq): adjust feature-config customization ARN-override logic aws#6674 1. change poll interval from 30mins to 180 mins 2. Only call listAvailableCustomizations for new customization override for update 3. Use a new field customizationArnOverrideV2 for a clean override plan * correction for removing usecase from workspaceUtils. * Fix for imports * fix for useCase trype for lint * Fixed test according to comment * Update packages/amazonq/test/unit/codewhisperer/service/securityScanHandler.test.ts Co-authored-by: Tai Lai <[email protected]> * Fix styles * Release 3.48.0 * Release 1.49.0 * Update version to snapshot version: 3.49.0-SNAPSHOT * Update version to snapshot version: 1.50.0-SNAPSHOT --------- Co-authored-by: aws-toolkit-automation <[email protected]> Co-authored-by: aws-toolkit-automation <> Co-authored-by: Josh Pinkney <[email protected]> Co-authored-by: chungjac <[email protected]> Co-authored-by: Hweinstock <[email protected]> Co-authored-by: Kevin Ding <[email protected]> Co-authored-by: Avi Alpert <[email protected]> Co-authored-by: Justin M. Keyes <[email protected]> Co-authored-by: Will Lo <[email protected]> Co-authored-by: Yaofu Zuo <[email protected]> Co-authored-by: David <[email protected]> Co-authored-by: David Hasani <[email protected]> Co-authored-by: Kevin Ding <[email protected]> Co-authored-by: Ashish Reddy Podduturi <[email protected]> Co-authored-by: Silvia Chen <[email protected]> Co-authored-by: Silvia Chen <[email protected]> Co-authored-by: andrewyuq <[email protected]> Co-authored-by: invictus <[email protected]> Co-authored-by: Tai Lai <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Problem
Within telemetry, there are a large group of fatal errors within EC2 Connect being grouped together under the same general error code. Additionally, there is very little information in the error message to determine the source of this error. Specifically, it appears the the test SSM connection done before attempting to pass the connection to VS Code, implemented here, can fail, and gives us a general error code with little information about its root cause.
Solution
Example Error:
feature/xbranches will not be squash-merged at release time.