Skip to content

Conversation

@YunchuWang
Copy link
Member

@YunchuWang YunchuWang commented May 16, 2025

This PR addresses an issue where the VSTest@2 task fails to pick up the latest built test assemblies when running in distributed agent mode.

Without this fix, the test discovery phase uses the most recently built DLLs to detect and assign test cases. However, during execution, the distributed agents may use stale, cached test assemblies from previous successful pipeline runs—potentially missing newly added test cases.

This change ensures that all test agents receive the latest built test DLLs, preventing "test not found" errors and guaranteeing accurate test discovery and execution.

Relevant documentation:
https://learn.microsoft.com/en-us/azure/devops/pipelines/tasks/reference/vstest-v2?view=azure-pipelines

image

Test artifact handling and publishing:

  • Updated testAssembly paths in eng/ci/public-build.yml to use a unified directory structure (tests/**) and added artifactName parameters for each test stage to specify artifact names (CoreTests, AzureStorageTests, EmulatorTests). [1] [2] [3]
  • Added steps in eng/templates/build-steps.yml to publish test artifacts (CoreTests, AzureStorageTests, EmulatorTests) using the 1ES.PublishPipelineArtifact task. [1] [2] [3]

Test execution improvements:

  • Enhanced eng/templates/test.yml to include a new artifactName parameter and steps for downloading test artifacts and listing downloaded test DLLs.
  • Updated the testAssemblyVer2 parameter to use the downloaded artifact path for test execution.
  • Enabled parallel test execution with the runInParallel option for the VSTest task.

@YunchuWang YunchuWang requested a review from bachuv May 17, 2025 14:56
This reverts commit eb11793.
@YunchuWang YunchuWang requested a review from cgillum May 17, 2025 14:57
@cgillum cgillum merged commit a5bee07 into main May 17, 2025
44 checks passed
@cgillum cgillum deleted the wangbill/fixci branch May 17, 2025 16:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants