Skip to content

Long-Running Test spec, plan, and tasks#124

Closed
brooke-hamilton wants to merge 23 commits intomainfrom
001-lrt-modernization-v2
Closed

Long-Running Test spec, plan, and tasks#124
brooke-hamilton wants to merge 23 commits intomainfrom
001-lrt-modernization-v2

Conversation

@brooke-hamilton
Copy link
Copy Markdown
Member

@brooke-hamilton brooke-hamilton commented Nov 12, 2025 •
edited
Loading

Description

This PR adds a feature specification and plan for modernizing the Radius Long-Running Test (LRT) system to improve developer experience and align with contemporary workflow design principles.

The spec was generated using Spec Kit, which is why the folder structure does not align with our existing folder structure.

Overview

The specification defines a modernized LRT system that enables:

  • Developer self-service: Developers can provision dedicated AKS test environments, run tests locally, and tear down infrastructure using simple Make commands
  • Fork-compatible workflows: Contributors can test workflow changes on forks without access to radius-project organization resources
  • Automated release validation: System automatically detects version mismatches and upgrades clusters before running tests
  • Improved observability: Comprehensive health checks, diagnostic artifacts, and actionable error messages

Key User Stories (Priority Order)

Developer Infrastructure Setup

  • Provision dedicated AKS clusters with full isolation per developer
  • Run complete LRT suite locally with simple commands (make lrt-deploy-infra, make lrt-run-all, make lrt-destroy-infra)
  • Infrastructure deployment completes in 30-45 minutes with idempotent operations

Fork-Based Workflow Testing

  • Execute LRT workflows on forks using only fork owner's resources
  • Manual workflow triggering via workflow_dispatch on any branch
  • Scheduled triggers ignored on forks to prevent unexpected resource usage

Run tests on the current release

  • Compare CLI vs cluster Radius versions at workflow start
  • Automatic cluster upgrade using rad upgrade when versions differ
  • Configurable rollback on test failures

Local Test Execution

  • Run individual test suites (make lrt-test-corerp) or full suite (make lrt-run-all)
  • Real-time progress output and fast feedback loops
  • Test execution completes within 120 minutes

brooke-hamilton and others added 9 commits November 6, 2025 10:34
@brooke-hamilton
spec kit init
0188095 
Signed-off-by: Brooke Hamilton <45323234+brooke-hamilton@users.noreply.github.com>
@brooke-hamilton
vs code workspace
0499b89 
Signed-off-by: Brooke Hamilton <45323234+brooke-hamilton@users.noreply.github.com>
@brooke-hamilton
constitution
23375e0 
Signed-off-by: Brooke Hamilton <45323234+brooke-hamilton@users.noreply.github.com>
@DariuszPorowski
feat(spec-kit): add powershell support (#119)
0ee72ce 
* Add spec-kit PowerShell files
* Adjust instructions to use scripts based on OS condition

---------

Signed-off-by: Dariusz Porowski <3431813+DariuszPorowski@users.noreply.github.com>
@DariuszPorowski
feat(spec-kit): constitution - new principles and governance clarific…
bb4c93d 
…ations (#120)

lint scripts with `shfmt`
lint MD with `markdownlint`

---------

Signed-off-by: Dariusz Porowski <3431813+DariuszPorowski@users.noreply.github.com>
@brooke-hamilton
constitution updates
0808866 
Signed-off-by: Brooke Hamilton <45323234+brooke-hamilton@users.noreply.github.com>
@brooke-hamilton
Merge remote-tracking branch 'origin' into spec-kit-constitution
97dd943
@brooke-hamilton
Merge branch 'main' into spec-kit-constitution
3e02060
@brooke-hamilton
spec
3f39cc2 
Signed-off-by: Brooke Hamilton <45323234+brooke-hamilton@users.noreply.github.com>
@DariuszPorowski
fix(spell): add new terms to en-custom.txt
62805b3 
Signed-off-by: GitHub <noreply@github.com>
@DariuszPorowski
fix(spell): add new terms
0809454 
Signed-off-by: GitHub <noreply@github.com>
@DariuszPorowski DariuszPorowski changed the title Long-Running Test spec [WIP] Long-Running Test spec Nov 12, 2025
@brooke-hamilton brooke-hamilton marked this pull request as ready for review November 16, 2025 00:55
@brooke-hamilton brooke-hamilton requested review from a team as code owners November 16, 2025 00:55
@brooke-hamilton
spec updates
a0f1419 
Signed-off-by: Brooke Hamilton <45323234+brooke-hamilton@users.noreply.github.com>
@brooke-hamilton
spelling
f93d95c 
Signed-off-by: Brooke Hamilton <45323234+brooke-hamilton@users.noreply.github.com>
@brooke-hamilton brooke-hamilton changed the title [WIP] Long-Running Test spec Long-Running Test spec Nov 16, 2025
@brooke-hamilton
Merge branch 'spec-kit-constitution' into 001-lrt-modernization
33b5e11
@brooke-hamilton
spec clarify
e8c8954 
Signed-off-by: Brooke Hamilton <45323234+brooke-hamilton@users.noreply.github.com>
@brooke-hamilton
spec clarify
fbb394c 
Signed-off-by: Brooke Hamilton <45323234+brooke-hamilton@users.noreply.github.com>
@brooke-hamilton
plan
315d95d 
Signed-off-by: Brooke Hamilton <45323234+brooke-hamilton@users.noreply.github.com>
@brooke-hamilton
tasks
aed9c62 
Signed-off-by: Brooke Hamilton <45323234+brooke-hamilton@users.noreply.github.com>
@brooke-hamilton
plan
f3d19c0 
Signed-off-by: Brooke Hamilton <45323234+brooke-hamilton@users.noreply.github.com>
@brooke-hamilton
spelling
cb28a3f 
Signed-off-by: Brooke Hamilton <45323234+brooke-hamilton@users.noreply.github.com>
@brooke-hamilton brooke-hamilton changed the title Long-Running Test spec Long-Running Test spec, plan, and tasks Nov 18, 2025
@brooke-hamilton brooke-hamilton marked this pull request as draft November 18, 2025 03:30
@DariuszPorowski
Merge branch 'main' into 001-lrt-modernization-v2
772b190 
Signed-off-by: Dariusz Porowski <3431813+DariuszPorowski@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

@DariuszPorowski DariuszPorowski

@brooke-hamilton brooke-hamilton

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@brooke-hamilton @DariuszPorowski