Use internal repository to manage secondary pipeline dependencies #25535

Abe27342 · 2025-09-24T17:06:49Z

Description

Our secondary pipelines (real service e2e, stress tests, performance benchmarks, etc.) rely on a small amount of infrastructure published internally as npm packages. Before this change, we have been consuming those packages using npx or equivalent bare installations from within the secondary pipeline. This is not great practice as it does not allow repeatability of the generated install tree, as npm and other package managers prefer to use newer versions of packages within semver allowance as they are released.

This PR updates the include-real-service-test template (which is used by several of these pipelines) to run its workload from the context of a very simple internal repository which has pnpm configurations and the packages we need installed. This gives us the benefits of a lock file within source control.

Test run: here

…generator

Copilot

Pull Request Overview

This PR modernizes dependency management for secondary pipelines by introducing an internal repository approach. The change moves from using npx for direct package installations to utilizing the ff_pipeline_host repository with pnpm and lockfile-based dependency management.

Key changes include:

Migrating from npx commands to pnpm exec for better dependency control
Switching from custom workspace directories to $(Build.SourcesDirectory)
Adding proper dependency installation steps using pnpm with lockfiles

Reviewed Changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.

File	Description
tools/pipelines/templates/include-vars-telemetry-generator.yml	Adds new variable for telemetry generator handlers path in the repository-based approach
tools/pipelines/templates/include-upload-stage-telemetry.yml	Replaces telemetry setup template with repository checkout and pnpm-based dependency installation
tools/pipelines/templates/include-test-real-service.yml	Major refactor from workspace-based npm to repository-based pnpm approach with streamlined dependency management
scripts/get-test-pass-rate.mjs	Removes script file (likely moved to the internal repository)

tools/pipelines/templates/include-test-real-service.yml

Abe27342 · 2025-09-24T17:57:06Z

scripts/get-test-pass-rate.mjs

-// This script is used in the "runAfterAll" stage in our E2E test pipeline. It's used
-// to get timeline and metrics data so that the test pass rate can be calculated in a later step.
-
-// The Build ID needed to fetch the desired data.


I just moved this whole script to the internal repository. If we leave it here, we need another checkout step in the telemetry upload job to get at it, which seems kinda wasteful.

Moving it comes with some more compatibility concerns, but I think that's probably ok.

jzaffiro · 2025-09-24T18:02:10Z

tools/pipelines/templates/include-test-real-service.yml

-            workingDirectory: ${{ parameters.testWorkspace }}
            targetType: 'inline'
-            script: 'npm install $(Initialize.testPackageTgz) ${{ parameters.loggerPackage }}'
+            script: 'pnpm install $(Initialize.testPackageTgz)'


Is the logger installed by default when the pipeline host repo is checked out?

jzaffiro

I think the performance benchmarks pipeline will need to be updated separately since it doesn't use the include-test-real-service template, but I'm fine to have this go in first.

Abe27342 · 2025-09-24T18:06:04Z

I think the performance benchmarks pipeline will need to be updated separately since it doesn't use the include-test-real-service template, but I'm fine to have this go in first.

Yup, it definitely needs updates. That's the one I think might be broken as well by this refactor, but we'll see. It's broken now anyway, so... :P

## Description Follow-up to #25535: this extra syntax is no longer required after moving off `npx` and though they are benign for me locally, they cause parameters to not be plumbed through in pipelines (likely due to shell differences) Test run: [here](https://dev.azure.com/fluidframework/internal/_build/results?buildId=357111&view=results) Co-authored-by: Abram Sanderson <[email protected]>

## Description Follow-up to #25535 that allows specifying the primary registry in `include-install-pnpm.yaml` so that the telemetry stage upload template can reference the feed we use to install internal packages. Test run [here](https://dev.azure.com/fluidframework/internal/_build/results?buildId=357059&view=logs&j=7ef2f12f-27d7-5730-657b-d3ad5f630362&t=9da23dd1-2e0f-578e-ea27-ee9162e0f22c). Co-authored-by: Abram Sanderson <[email protected]>

…25543) ## Description Follow-up to #25535: in line with using a "host repository" that has package-lock'd internal dependencies, tighten the "copy devDependency" script that we use to get devDependencies necessary at workload runtime to exclude some common dependencies that are only used at build time. This is motivated by still seeing some DDS fuzz harness pipeline failures due to recently published transitive dependencies of build-tools. Test run: [here](https://dev.azure.com/fluidframework/internal/_build/results?buildId=357194&view=logs&j=bbaafa3d-c204-5f81-4508-786dc29a3642&t=f302f968-fb15-5cb8-d0fb-0d951f384806) --------- Co-authored-by: Abram Sanderson <[email protected]>

…25550) ## Description Follow-up to #25535 which applies the same kind of change to the performance benchmarks pipeline template. Test run: [here](https://dev.azure.com/fluidframework/internal/_build/results?buildId=357522&view=results) --------- Co-authored-by: Abram Sanderson <[email protected]>

) ## Description Follow-up to #25535 which restores the caching of compat dependencies we previously had. Couldn't help myself from making a tweak so the cache key is invalidated on minor version change, which should give us better long-term behavior: the set of versions to install is dynamically computed based on roughly the current package version, so using major+minor here should be a net win as only roughly one build per release should need to install compat dependencies, after that they will be cached. --------- Co-authored-by: Abram Sanderson <[email protected]>

…crosoft#25535) ## Description Our secondary pipelines (real service e2e, stress tests, performance benchmarks, etc.) rely on a small amount of infrastructure published internally as npm packages. Before this change, we have been consuming those packages using `npx` or equivalent bare installations from within the secondary pipeline. This is not great practice as it does not allow repeatability of the generated install tree, as npm and other package managers prefer to use newer versions of packages within semver allowance as they are released. This PR updates the `include-real-service-test` template (which is used by several of these pipelines) to run its workload from the context of a very simple internal repository which has pnpm configurations and the packages we need installed. This gives us the benefits of a lock file within source control. Test run: [here](https://dev.azure.com/fluidframework/internal/_build/results?buildId=356944&view=results) --------- Co-authored-by: Abram Sanderson <[email protected]>

## Description Follow-up to microsoft#25535: this extra syntax is no longer required after moving off `npx` and though they are benign for me locally, they cause parameters to not be plumbed through in pipelines (likely due to shell differences) Test run: [here](https://dev.azure.com/fluidframework/internal/_build/results?buildId=357111&view=results) Co-authored-by: Abram Sanderson <[email protected]>

…soft#25540) ## Description Follow-up to microsoft#25535 that allows specifying the primary registry in `include-install-pnpm.yaml` so that the telemetry stage upload template can reference the feed we use to install internal packages. Test run [here](https://dev.azure.com/fluidframework/internal/_build/results?buildId=357059&view=logs&j=7ef2f12f-27d7-5730-657b-d3ad5f630362&t=9da23dd1-2e0f-578e-ea27-ee9162e0f22c). Co-authored-by: Abram Sanderson <[email protected]>

…icrosoft#25543) ## Description Follow-up to microsoft#25535: in line with using a "host repository" that has package-lock'd internal dependencies, tighten the "copy devDependency" script that we use to get devDependencies necessary at workload runtime to exclude some common dependencies that are only used at build time. This is motivated by still seeing some DDS fuzz harness pipeline failures due to recently published transitive dependencies of build-tools. Test run: [here](https://dev.azure.com/fluidframework/internal/_build/results?buildId=357194&view=logs&j=bbaafa3d-c204-5f81-4508-786dc29a3642&t=f302f968-fb15-5cb8-d0fb-0d951f384806) --------- Co-authored-by: Abram Sanderson <[email protected]>

…icrosoft#25550) ## Description Follow-up to microsoft#25535 which applies the same kind of change to the performance benchmarks pipeline template. Test run: [here](https://dev.azure.com/fluidframework/internal/_build/results?buildId=357522&view=results) --------- Co-authored-by: Abram Sanderson <[email protected]>

…rosoft#25563) ## Description Follow-up to microsoft#25535 which restores the caching of compat dependencies we previously had. Couldn't help myself from making a tweak so the cache key is invalidated on minor version change, which should give us better long-term behavior: the set of versions to install is dynamically computed based on roughly the current package version, so using major+minor here should be a net win as only roughly one build per release should need to install compat dependencies, after that they will be cached. --------- Co-authored-by: Abram Sanderson <[email protected]>

Abram Sanderson added 16 commits September 23, 2025 12:35

First attempt at using setup repo

4b4b1fa

fix syntax

6c76dd1

Install pnpm first

1efc5ef

Specify buildDirectory

4801f7d

testWorkspace -> build.sourcesdirectory

e5e5744

Specify working directory on npm scripts

545eaec

Use correct script names

8552d32

Restore initialize step, remove unnecessary parts

afba3ba

Run initialize from sources directory, also start adapting telemetry-…

f39d745

…generator

Remove --no-frozen-lockfile

6e8828b

Try plumbing reservation id through env

4bacb31

Add inline install

3238fd7

Fix some path manipulation bugs

8b12f72

More path updates

c316cb0

Remove get-test-pass-rate script

b3766b4

Use pnpm exec over indirecting through scripts

e422588

Copilot AI review requested due to automatic review settings September 24, 2025 17:06

github-actions bot added area: build Build related issues base: main PRs targeted against main branch labels Sep 24, 2025

Copilot AI reviewed Sep 24, 2025

View reviewed changes

tools/pipelines/templates/include-test-real-service.yml Show resolved Hide resolved

Abe27342 commented Sep 24, 2025

View reviewed changes

jzaffiro reviewed Sep 24, 2025

View reviewed changes

jzaffiro approved these changes Sep 24, 2025

View reviewed changes

Abe27342 merged commit e049473 into main Sep 24, 2025
81 of 86 checks passed

Abe27342 deleted the test/absander/pipeline-host branch September 24, 2025 18:06

This was referenced Sep 24, 2025

Allow specifying primary registry in include-install-pnpm.yaml #25540

Merged

Remove extra -- from pnpm exec scripts #25541

Merged

Abe27342 mentioned this pull request Sep 25, 2025

Improve 'copy devDependencies' step of include-test-real-service.yml #25543

Merged

This was referenced Sep 25, 2025

Use secondary repository checkout in performance benchmarks pipeline #25550

Merged

Restore compat install caching to include-test-real-service.yaml #25563

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Use internal repository to manage secondary pipeline dependencies #25535

Use internal repository to manage secondary pipeline dependencies #25535

Uh oh!

Abe27342 commented Sep 24, 2025 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Abe27342 Sep 24, 2025

Uh oh!

jzaffiro Sep 24, 2025

Uh oh!

jzaffiro left a comment

Uh oh!

Abe27342 commented Sep 24, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Use internal repository to manage secondary pipeline dependencies #25535

Use internal repository to manage secondary pipeline dependencies #25535

Uh oh!

Conversation

Abe27342 commented Sep 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Uh oh!

Abe27342 Sep 24, 2025

Choose a reason for hiding this comment

Uh oh!

jzaffiro Sep 24, 2025

Choose a reason for hiding this comment

Uh oh!

jzaffiro left a comment

Choose a reason for hiding this comment

Uh oh!

Abe27342 commented Sep 24, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Abe27342 commented Sep 24, 2025 •

edited

Loading