Skip to content

Conversation

@Abdul-Microsoft
Copy link
Contributor

Purpose

This pull request introduces significant updates to the test automation workflow, quota validation scripts, and error handling mechanisms. The changes include a new GitHub Actions workflow for end-to-end testing, enhanced quota validation for AI model deployments, and improved error handling for rate limit issues in the backend and frontend.

Test Automation Enhancements:

  • Added a new GitHub Actions workflow in .github/workflows/test-automation.yml for end-to-end testing, including retry mechanisms, test report uploads, and email notifications for success or failure.

Quota Validation Improvements:

  • Introduced validate_model_deployment_quota.sh and validate_model_quota.sh scripts to validate Azure quota availability for AI model deployments, with detailed error messages and fallback region recommendations. [1] [2]
  • Added a corresponding PowerShell script (validate_model_deployment_quotas.ps1) for Windows-based quota validation.
  • Updated infra/main.parameters.json to include parameters for AI model deployments, environment configurations, and resource existence checks.
  • Modified azure.yaml to include preprovision hooks for quota validation during deployment.

Backend and Frontend Error Handling:

  • Enhanced error handling in src/backend/app_kernel.py and src/backend/kernel_agents/planner_agent.py to log and handle rate limit issues gracefully. [1] [2]
  • Updated src/frontend/wwwroot/home/home.js to display user-friendly error messages when rate limits are exceeded, preventing further actions until resolved.

Region Configuration Updates:

  • Adjusted the default regions in infra/scripts/quota_check_params.sh to include a refined list of Azure regions for quota checks.
  • ...

Does this introduce a breaking change?

  • Yes
  • No

How to Test

  • Get the code
git clone [repo-address]
cd [repo-name]
git checkout [branch-name]
npm install
  • Test the code

What to Check

Verify that the following are valid

  • ...

Other Information

Prashant-Microsoft and others added 13 commits June 3, 2025 15:14
feat: added rate limit error pop up
feat: quota auto validation before deployment
* Added e2e test automation scripts and pipeline

* Refactor test automation workflow: restore environment variable references and clean up commented paths

* Refactor code for improved readability and maintainability: clean up imports, adjust formatting, and enhance comments across multiple files

* Fix workflow trigger paths in test automation configuration
fix: Add execute permissions for validation scripts in preprovision hook
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

Merges hotfix changes into main, introducing end-to-end test setup, quota validation tooling, and improved rate-limit error handling.

  • Adds a pytest configuration and Playwright page objects for login and BIAB workflows
  • Integrates Azure quota validation scripts (Bash/PowerShell) via preprovision hooks
  • Enhances frontend/backend to catch and surface rate-limit errors

Reviewed Changes

Copilot reviewed 25 out of 25 changed files in this pull request and generated 6 comments.

Show a summary per file
File Description
tests/e2e-test/pytest.ini Configures CLI and file logging for pytest
tests/e2e-test/pages/loginPage.py Implements login flow with Playwright
tests/e2e-test/pages/BIAB.py Adds BIAB page object for task creation and approval
tests/e2e-test/base/base.py Defines base page utilities including API status check
tests/e2e-test/config/constants.py Loads environment and sets test constants
src/frontend/wwwroot/home/home.js Displays user-friendly messages on rate limits
src/backend/app_kernel.py Includes exception details in HTTP 400 responses
src/backend/kernel_agents/planner_agent.py Logs and rethrows rate-limit exceptions
infra/scripts/validate_model_quota.sh Bash script to check Azure AI model quotas
infra/scripts/validate_model_quota.ps1 PowerShell version of quota checker
infra/scripts/validate_model_deployment_* Scripts to iterate model deployments and validate
infra/scripts/quota_check_params.sh Updates default Azure regions list
.github/workflows/test-automation.yml CI workflow for end-to-end test automation
Comments suppressed due to low confidence (1)

tests/e2e-test/pages/loginPage.py:31

  • The method name enter_aditional_info has a typo. Rename it to enter_additional_info to match correct spelling.
def enter_aditional_info(self, text):

@Roopan-Microsoft Roopan-Microsoft merged commit 465a96c into main Jun 5, 2025
8 checks passed
@github-actions
Copy link

github-actions bot commented Jun 5, 2025

🎉 This PR is included in version 1.7.0 🎉

The release is available on GitHub release

Your semantic-release bot 📦🚀

blessing-msft pushed a commit that referenced this pull request Aug 1, 2025
feat: merging the changes from hotfix to main
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants