-
Notifications
You must be signed in to change notification settings - Fork 227
Add VerifySetup Eval Tests and Tool Mocks #13388
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull request overview
This PR adds evaluation tests for the VerifySetup functionality and introduces tool mocks for SDK generation and building operations. The changes test that the verify_setup tool is correctly invoked both directly and as a prerequisite step before SDK generation.
- Added two new tool mocks: PackageGenerateCode and PackageBuildCode to support testing SDK generation workflows
- Created test to verify that the verify_setup tool is invoked when explicitly requested
- Created test to verify that verify_setup is called before SDK generation operations
Reviewed changes
Copilot reviewed 5 out of 5 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| tools/azsdk-cli/Azure.Sdk.Tools.Cli.Evaluations/ToolMocks/ToolMocks.cs | Registered PackageGenerateCode and PackageBuildCode tool mocks in the mock registry |
| tools/azsdk-cli/Azure.Sdk.Tools.Cli.Evaluations/ToolMocks/PackageGenerateCode.cs | Implemented tool mock for the azsdk_package_generate_code operation with sample responses |
| tools/azsdk-cli/Azure.Sdk.Tools.Cli.Evaluations/ToolMocks/PackageBuildCode.cs | Implemented tool mock for the azsdk_package_build_code operation with sample responses |
| tools/azsdk-cli/Azure.Sdk.Tools.Cli.Evaluations/Scenarios/General/Evaluate_VerifySetup.cs | Added test case to verify that verify_setup tool is invoked for DotNet setup verification |
| tools/azsdk-cli/Azure.Sdk.Tools.Cli.Evaluations/Scenarios/AzureRestApiSpecs/Evaluate_GenerateSdk.cs | Added test case to verify that verify_setup is invoked before SDK generation |
tools/azsdk-cli/Azure.Sdk.Tools.Cli.Evaluations/ToolMocks/PackageGenerateCode.cs
Show resolved
Hide resolved
tools/azsdk-cli/Azure.Sdk.Tools.Cli.Evaluations/ToolMocks/PackageBuildCode.cs
Show resolved
Hide resolved
...zsdk-cli/Azure.Sdk.Tools.Cli.Evaluations/Scenarios/AzureRestApiSpecs/Evaluate_GenerateSdk.cs
Outdated
Show resolved
Hide resolved
...zsdk-cli/Azure.Sdk.Tools.Cli.Evaluations/Scenarios/AzureRestApiSpecs/Evaluate_GenerateSdk.cs
Outdated
Show resolved
Hide resolved
Co-authored-by: Copilot <[email protected]>
…k-tools into env-setup-eval
closes #13132