Skip to content

Integrate deployment E2E tests into CI workflow#15468

Open
mitchdenny wants to merge 7 commits intomainfrom
midenn/verify-deployment-test
Open

Integrate deployment E2E tests into CI workflow#15468
mitchdenny wants to merge 7 commits intomainfrom
midenn/verify-deployment-test

Conversation

@mitchdenny
Copy link
Member

@mitchdenny mitchdenny commented Mar 21, 2026

Summary

Integrates deployment E2E tests into the main CI workflow so they run automatically on every non-fork PR. Also migrates the last remaining old-pattern test to the Hex1bAutomator approach, and adds deployment test recording comments.

Changes

CI Integration:

  • New deployment-e2e-tests.yml reusable workflow with enumerate → build → deploy-test matrix fan-out using environment: deployment-testing for OIDC Azure auth
  • Updated ci.yml with deployment_tests job that calls the reusable workflow, with fork detection (head.repo.full_name == repository) and skip logic (docs-only changes)
  • Updated results job to include deployment test results in the aggregation gate
  • Refactored deployment-tests.yml to call the reusable workflow (keeps nightly schedule + create_issue_on_failure)
  • Deleted deployment-test-command.yml — the /deployment-test slash command is replaced by automatic CI integration

Recording Comments:

  • Extended cli-e2e-recording-comment.yml with a second job (add-deployment-recording-comment) that posts deployment test recordings as a separate PR comment with <!-- deployment-e2e-recordings --> marker
  • Includes per-test pass/fail status table with asciinema recording links

Test Migration:

  • Migrated AcaManagedRedisDeploymentTests from Hex1bTerminalInputSequenceBuilder (fluent builder) to Hex1bTerminalAutomator (async/await)
  • Removed dead builder extension methods from DeploymentE2ETestHelpers.cs

Environment Requirement

⚠️ The deployment-testing GitHub environment must allow deployment from PR branches (not just main/release/**). Verify environment protection rules before merging.

Testing

  • Build succeeds with 0 warnings, 0 errors
  • Deployment tests will run automatically on this PR once CI triggers

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings March 21, 2026 03:17
@mitchdenny
Copy link
Member Author

/deployment-test

@github-actions
Copy link
Contributor

github-actions bot commented Mar 21, 2026

🚀 Dogfood this PR with:

⚠️ WARNING: Do not do this without first carefully reviewing the code of this PR to satisfy yourself it is safe.

curl -fsSL https://raw.githubusercontent.com/microsoft/aspire/main/eng/scripts/get-aspire-cli-pr.sh | bash -s -- 15468

Or

  • Run remotely in PowerShell:
iex "& { $(irm https://raw.githubusercontent.com/microsoft/aspire/main/eng/scripts/get-aspire-cli-pr.ps1) } 15468"

@github-actions
Copy link
Contributor

🚀 Deployment tests starting on PR #15468...

This will deploy to real Azure infrastructure. Results will be posted here when complete.

View workflow run

Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR verifies that the /deployment-test slash command still functions after the repository moved from dotnet/aspire to microsoft/aspire by making a whitespace-only change.

Changes:

  • Add a trailing newline at the end of README.md.

@github-actions github-actions bot temporarily deployed to deployment-testing March 21, 2026 03:24 Inactive
@github-actions github-actions bot temporarily deployed to deployment-testing March 21, 2026 03:24 Inactive
@github-actions github-actions bot temporarily deployed to deployment-testing March 21, 2026 03:24 Inactive
@github-actions github-actions bot temporarily deployed to deployment-testing March 21, 2026 03:24 Inactive
@github-actions github-actions bot temporarily deployed to deployment-testing March 21, 2026 03:24 Inactive
@github-actions github-actions bot temporarily deployed to deployment-testing March 21, 2026 03:24 Inactive
@github-actions github-actions bot temporarily deployed to deployment-testing March 21, 2026 03:24 Inactive
@github-actions github-actions bot temporarily deployed to deployment-testing March 21, 2026 03:24 Inactive
@github-actions github-actions bot had a problem deploying to deployment-testing March 21, 2026 03:24 Failure
@github-actions github-actions bot had a problem deploying to deployment-testing March 21, 2026 03:24 Failure
@github-actions github-actions bot temporarily deployed to deployment-testing March 21, 2026 03:24 Inactive
@github-actions github-actions bot temporarily deployed to deployment-testing March 21, 2026 03:24 Inactive
@github-actions github-actions bot temporarily deployed to deployment-testing March 21, 2026 03:24 Inactive
@github-actions github-actions bot temporarily deployed to deployment-testing March 21, 2026 03:24 Inactive
@github-actions github-actions bot temporarily deployed to deployment-testing March 21, 2026 03:24 Inactive
@github-actions github-actions bot temporarily deployed to deployment-testing March 21, 2026 03:24 Inactive
@github-actions github-actions bot temporarily deployed to deployment-testing March 21, 2026 03:24 Inactive
@github-actions github-actions bot temporarily deployed to deployment-testing March 21, 2026 03:24 Inactive
@github-actions github-actions bot temporarily deployed to deployment-testing March 21, 2026 03:24 Inactive
@github-actions github-actions bot temporarily deployed to deployment-testing March 21, 2026 03:24 Inactive
@github-actions github-actions bot temporarily deployed to deployment-testing March 21, 2026 03:24 Inactive
@github-actions github-actions bot temporarily deployed to deployment-testing March 21, 2026 03:24 Inactive
@github-actions github-actions bot temporarily deployed to deployment-testing March 21, 2026 03:24 Inactive
Mitch Denny and others added 2 commits March 21, 2026 16:13
…flow

Reusable workflows called via workflow_call don't automatically
receive the caller's secrets. The deployment-testing environment
secrets (AZURE_DEPLOYMENT_TEST_CLIENT_ID, TENANT_ID, SUBSCRIPTION_ID)
need secrets:inherit to be accessible.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The enumerate-tests action uploads logs-enumerate-tests-Linux, but both
the main test setup and deployment test enumerate jobs use it in the same
workflow run. Add artifactSuffix input to disambiguate artifact names.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@mitchdenny mitchdenny temporarily deployed to deployment-testing March 21, 2026 05:30 — with GitHub Actions Inactive
@mitchdenny mitchdenny temporarily deployed to deployment-testing March 21, 2026 05:30 — with GitHub Actions Inactive
@mitchdenny mitchdenny temporarily deployed to deployment-testing March 21, 2026 05:30 — with GitHub Actions Inactive
@mitchdenny mitchdenny temporarily deployed to deployment-testing March 21, 2026 05:30 — with GitHub Actions Inactive
@mitchdenny mitchdenny temporarily deployed to deployment-testing March 21, 2026 05:30 — with GitHub Actions Inactive
@mitchdenny mitchdenny temporarily deployed to deployment-testing March 21, 2026 05:30 — with GitHub Actions Inactive
@mitchdenny mitchdenny temporarily deployed to deployment-testing March 21, 2026 05:30 — with GitHub Actions Inactive
@mitchdenny mitchdenny temporarily deployed to deployment-testing March 21, 2026 05:30 — with GitHub Actions Inactive
@github-actions
Copy link
Contributor

🎬 CLI E2E Test Recordings — 52 recordings uploaded (commit 7f94382)

View recordings
Test Recording
AddPackageInteractiveWhileAppHostRunningDetached ▶️ View Recording
AddPackageWhileAppHostRunningDetached ▶️ View Recording
AgentCommands_AllHelpOutputs_AreCorrect ▶️ View Recording
AgentInitCommand_DefaultSelection_InstallsSkillOnly ▶️ View Recording
AgentInitCommand_MigratesDeprecatedConfig ▶️ View Recording
AspireAddPackageVersionToDirectoryPackagesProps ▶️ View Recording
AspireUpdateRemovesAppHostPackageVersionFromDirectoryPackagesProps ▶️ View Recording
Banner_DisplayedOnFirstRun ▶️ View Recording
Banner_DisplayedWithExplicitFlag ▶️ View Recording
CertificatesClean_RemovesCertificates ▶️ View Recording
CertificatesTrust_WithNoCert_CreatesAndTrustsCertificate ▶️ View Recording
CertificatesTrust_WithUntrustedCert_TrustsCertificate ▶️ View Recording
ConfigSetGet_CreatesNestedJsonFormat ▶️ View Recording
CreateAndDeployToDockerCompose ▶️ View Recording
CreateAndDeployToDockerComposeInteractive ▶️ View Recording
CreateAndPublishToKubernetes ▶️ View Recording
CreateAndRunAspireStarterProject ▶️ View Recording
CreateAndRunAspireStarterProjectWithBundle ▶️ View Recording
CreateAndRunEmptyAppHostProject ▶️ View Recording
CreateAndRunJsReactProject ▶️ View Recording
CreateAndRunPythonReactProject ▶️ View Recording
CreateAndRunTypeScriptEmptyAppHostProject ▶️ View Recording
CreateAndRunTypeScriptStarterProject ▶️ View Recording
CreateStartAndStopAspireProject ▶️ View Recording
CreateTypeScriptAppHostWithViteApp ▶️ View Recording
DescribeCommandResolvesReplicaNames ▶️ View Recording
DescribeCommandShowsRunningResources ▶️ View Recording
DetachFormatJsonProducesValidJson ▶️ View Recording
DoctorCommand_DetectsDeprecatedAgentConfig ▶️ View Recording
DoctorCommand_WithSslCertDir_ShowsTrusted ▶️ View Recording
DoctorCommand_WithoutSslCertDir_ShowsPartiallyTrusted ▶️ View Recording
GlobalMigration_HandlesCommentsAndTrailingCommas ▶️ View Recording
GlobalMigration_HandlesMalformedLegacyJson ▶️ View Recording
GlobalMigration_PreservesAllValueTypes ▶️ View Recording
GlobalMigration_SkipsWhenNewConfigExists ▶️ View Recording
GlobalSettings_MigratedFromLegacyFormat ▶️ View Recording
InvalidAppHostPathWithComments_IsHealedOnRun ▶️ View Recording
LogsCommandShowsResourceLogs ▶️ View Recording
PsCommandListsRunningAppHost ▶️ View Recording
PsFormatJsonOutputsOnlyJsonToStdout ▶️ View Recording
PublishWithDockerComposeServiceCallbackSucceeds ▶️ View Recording
RestoreGeneratesSdkFiles ▶️ View Recording
RunWithMissingAwaitShowsHelpfulError ▶️ View Recording
SecretCrudOnDotNetAppHost ▶️ View Recording
SecretCrudOnTypeScriptAppHost ▶️ View Recording
StagingChannel_ConfigureAndVerifySettings_ThenSwitchChannels ▶️ View Recording
StopAllAppHostsFromAppHostDirectory ▶️ View Recording
StopAllAppHostsFromUnrelatedDirectory ▶️ View Recording
StopNonInteractiveMultipleAppHostsShowsError ▶️ View Recording
StopNonInteractiveSingleAppHost ▶️ View Recording
StopWithNoRunningAppHostExitsSuccessfully ▶️ View Recording
TypeScriptAppHostWithProjectReferenceIntegration ▶️ View Recording

📹 Recordings uploaded automatically from CI run #23372821660

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants