Remove need to provide client secret when creating workspace and remove Directory.Read.All Dependency from automation admin by marrobi · Pull Request #4775 · microsoft/AzureTRE

marrobi · 2025-11-26T17:55:22Z

Summary

Removes the requirement for users to provide a client secret when creating workspaces and eliminates the Directory.Read.All Microsoft Graph permission from the automation admin identity.

Changes

Base workspace bundle 3.0.0: Terraform now creates, imports, and manages workspace Microsoft Entra ID applications automatically, including secret rotation via azuread_application_password resources
API simplification: Removed extract_workspace_auth_information function - workspace auth info is now handled through Terraform outputs
Removed parameters: client_secret, register_aad_application, scope_id, sp_id, app_role_id_*
New script: add_automation_admin_to_workspace_application.sh for adding automation admin as workspace app owner
Permission reduction: Directory.Read.All no longer required; only Application.ReadWrite.All, Group.Create, Group.Read.All, User.ReadBasic.All, DelegatedPermissionGrant.ReadWrite.All (depending on configuration)

Migration

Existing workspaces continue to work without changes
Upgrading: Ensure Application Admin owns the workspace app, then run upgrade - Terraform will import and manage secrets automatically (needs testing)
New workspaces: No client_secret needed; optionally provide client_id to reuse an existing app

Closes #2247

Fixes microsoft#4772

…robi/issue2247

github-actions · 2025-11-26T17:57:31Z

Unit Test Results

672 tests 672 ✅ 8s ⏱️
1 suites 0 💤
1 files 0 ❌

Results for commit 25e3789.

♻️ This comment has been updated with latest results.

Copilot

Pull request overview

This PR removes the requirement for users to provide a client secret when creating workspaces and eliminates the Directory.Read.All Microsoft Graph permission dependency from the automation admin identity. The changes introduce automatic workspace app provisioning/import via Terraform with built-in password rotation, simplify the API by removing the extract_workspace_auth_information function, and update all related documentation and scripts.

Key changes include:

Terraform now provisions or imports the workspace Entra ID app automatically with dual password rotation using azuread_application_password resources
API no longer requires Directory.Read.All permissions as workspace auth information is handled via Terraform outputs
Major version bump for base workspace bundle (2.8.0 → 3.0.0) due to breaking changes

Reviewed changes

Copilot reviewed 37 out of 38 changed files in this pull request and generated 10 comments.

Show a summary per file

File	Description
templates/workspaces/base/terraform/workspace.tf	Adds import block for existing workspace apps and removes conditional AAD module creation
templates/workspaces/base/terraform/variables.tf	Removes register_aad_application and client_secret variables
templates/workspaces/base/terraform/providers.tf	Adds hashicorp/time provider for password rotation
templates/workspaces/base/terraform/outputs.tf	Simplifies outputs to always reference AAD module directly
templates/workspaces/base/terraform/keyvault.tf	Removes manual client_id and client_secret key vault secret resources
templates/workspaces/base/terraform/aad/variables.tf	Adds client_id variable, changes create_aad_groups type to bool
templates/workspaces/base/terraform/aad/providers.tf	Adds time provider requirement
templates/workspaces/base/terraform/aad/aad.tf	Implements dual password rotation with primary/secondary passwords and intelligent current password selection
templates/workspaces/base/terraform/.terraform.lock.hcl	Adds lock file entry for time provider v0.11.0
templates/workspaces/base/template_schema.json	Removes client_secret from schema and moves create_aad_groups to top level
templates/workspaces/base/porter.yaml	Major version bump to 3.0.0, removes register_aad_application and client_secret parameters
api_app/services/authentication.py	Removes extract_auth_information function
api_app/services/access_service.py	Removes extract_workspace_auth_information abstract method
api_app/services/aad_authentication.py	Removes _get_app_auth_info and extract_workspace_auth_information implementation
api_app/db/repositories/workspaces.py	Removes auth_info parameter from create_workspace_item
api_app/api/routes/workspaces.py	Removes extract_auth_information call and auth_info parameter
api_app/_version.py	Minor version bump to 0.25.5
api_app/tests_ma/test_services/test_aad_access_service.py	Removes tests for extract_workspace_auth_information
api_app/tests_ma/test_db/test_repositories/test_workpaces_repository.py	Updates test calls to remove auth_info parameter
api_app/tests_ma/test_api/test_routes/test_workspaces.py	Removes extract_auth_information mock patches
api_app/tests_ma/test_api/test_routes/test_workspace_users.py	Removes auth_info parameter from sample_workspace
docs/tre-developers/end-to-end-tests.md	Adds instructions for adding automation admin as workspace app owner
docs/tre-admins/setup-instructions/ui-install-base-workspace.md	Simplifies workspace app creation script usage
docs/tre-admins/setup-instructions/installing-base-workspace.md	Removes client_secret from workspace creation example
docs/tre-admins/setup-instructions/cicd-pre-deployment-steps.md	Removes TEST_WORKSPACE_APP_SECRET from required secrets
docs/tre-admins/identities/workspace.md	Removes client secret references and simplifies workspace app creation
docs/tre-admins/identities/application_admin.md	Updates required permissions from Directory.Read.All to Group.Read.All and User.ReadBasic.All
docs/tre-admins/environment-variables.md	Updates permission descriptions for auto workspace features
docs/tre-admins/auth.md	Updates permission descriptions and removes workspace_api_client_secret
devops/scripts/setup_local_debugging.sh	Removes TEST_WORKSPACE_APP_SECRET from environment setup
devops/scripts/create_aad_assets.sh	Removes Directory.Read.All from AUTO_WORKSPACE_APP_REGISTRATION permissions and removes automatic workspace app creation
devops/scripts/aad/wait_for_new_app_registration.sh	Minor cleanup removing echo statement
devops/scripts/aad/create_workspace_application.sh	Significantly simplified to only create minimal app registration without consent/permission setup
devops/scripts/aad/add_automation_admin_to_workspace_application.sh	New script for adding automation admin as workspace app owner
core/terraform/outputs.sh	Removes TEST_WORKSPACE_APP_SECRET from private.env
config_schema.json	Removes workspace_api_client_secret from schema
config.sample.yaml	Updates permission descriptions in comments

Files not reviewed (1)

templates/workspaces/base/terraform/.terraform.lock.hcl: Language not supported

docs/tre-admins/auth.md

config.sample.yaml

templates/workspaces/base/porter.yaml

docs/tre-admins/identities/application_admin.md

docs/tre-admins/environment-variables.md

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Copilot

Pull request overview

Copilot reviewed 37 out of 38 changed files in this pull request and generated 6 comments.

Files not reviewed (1)

templates/workspaces/base/terraform/.terraform.lock.hcl: Language not supported

api_app/_version.py

templates/workspaces/base/terraform/aad/aad.tf

docs/tre-admins/identities/application_admin.md

docs/tre-admins/environment-variables.md

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

github-actions · 2026-01-26T08:58:00Z

🤖 pr-bot 🤖

🏃 Running extended tests: https://github.com/microsoft/AzureTRE/actions/runs/21351779214 (with refid 679d0163)

(in response to this comment from @marrobi)

marrobi · 2026-01-26T10:41:21Z

/test-extended 1cac0eb

github-actions · 2026-01-26T10:41:34Z

🤖 pr-bot 🤖

🏃 Running extended tests: https://github.com/microsoft/AzureTRE/actions/runs/21354756330 (with refid 679d0163)

(in response to this comment from @marrobi)

… into marrobi/issue2247

marrobi · 2026-01-26T23:26:24Z

/test-extended 6769d1a

github-actions · 2026-01-26T23:26:42Z

🤖 pr-bot 🤖

🏃 Running extended tests: https://github.com/microsoft/AzureTRE/actions/runs/21377968136 (with refid 679d0163)

(in response to this comment from @marrobi)

# Conflicts: # .github/linters/.tflint.hcl # .github/linters/.tflint_workspaces.hcl # CHANGELOG.md # api_app/_version.py # core/version.txt # docs/tre-admins/auth.md # docs/tre-admins/environment-variables.md # docs/tre-admins/identities/application_admin.md # docs/tre-admins/identities/workspace.md # docs/tre-admins/setup-instructions/cicd-pre-deployment-steps.md # docs/tre-admins/setup-instructions/workflows.md # docs/tre-developers/end-to-end-tests.md # docs/tre-templates/workspaces/base.md # templates/workspaces/airlock-import-review/porter.yaml # templates/workspaces/base/porter.yaml # templates/workspaces/unrestricted/porter.yaml

…ror handling; remove deprecated redirect URL update script.

…fiers and improve role ID imports

marrobi · 2026-03-11T13:59:30Z

/test-extended ec8d3a9

github-actions · 2026-03-11T13:59:49Z

🤖 pr-bot 🤖

🏃 Running extended tests: https://github.com/microsoft/AzureTRE/actions/runs/22956322878 (with refid 679d0163)

(in response to this comment from @marrobi)

github-actions · 2026-03-11T14:47:07Z

🤖 pr-bot 🤖

🏃 Running extended tests: https://github.com/microsoft/AzureTRE/actions/runs/22956322878 (with refid 679d0163)

(in response to this comment from @marrobi)

github-actions · 2026-03-12T13:03:35Z

🤖 pr-bot 🤖

🏃 Running extended tests: https://github.com/microsoft/AzureTRE/actions/runs/22956322878 (with refid 679d0163)

(in response to this comment from @marrobi)

Copilot

Pull request overview

Copilot reviewed 75 out of 76 changed files in this pull request and generated 5 comments.

Files not reviewed (1)

templates/workspaces/base/terraform/.terraform.lock.hcl: Language not supported

Comments suppressed due to low confidence (1)

api_app/services/aad_authentication.py:604

The comment above _get_app_auth_info says auth_type=Manual "requires Directory.Read.All", but the code path is Graph application/service principal lookup and the PR explicitly removes Directory.Read.All from required permissions. Please update this comment to reflect the actual permission dependency (e.g., Application.Read.All / Application.ReadWrite.All) to avoid misleading operators.

    # DEPRECATED: Remove when workspace base bundles < 3.0.0 are no longer supported.
    # This method is only needed for auth_type=Manual which requires Directory.Read.All.
    def _get_app_auth_info(self, client_id: str) -> dict:
        graph_data = self._get_app_sp_graph_data(client_id)
        if 'value' not in graph_data or len(graph_data['value']) == 0:
            logger.debug(graph_data)
            raise AuthConfigValidationError(f"{strings.ACCESS_UNABLE_TO_GET_INFO_FOR_APP} {client_id}")

e2e_tests/helpers.py

Copilot · 2026-03-12T13:08:19Z

devops/scripts/create_aad_assets.sh

 if [ "${AUTO_WORKSPACE_APP_REGISTRATION:-}" == true ]; then
-  APPLICATION_PERMISSIONS+=("Application.ReadWrite.All" "Directory.Read.All")
+  APPLICATION_PERMISSIONS+=("Application.ReadWrite.All")
 fi

 if [ "${AUTO_WORKSPACE_GROUP_CREATION:-}" == true ]; then
-  APPLICATION_PERMISSIONS+=("Group.Create")
+  APPLICATION_PERMISSIONS+=("Group.Create" "Group.Read.All" "User.ReadBasic.All")
 fi

 if [ "${AUTO_GRANT_WORKSPACE_CONSENT:-}" == true ]; then
-  APPLICATION_PERMISSIONS+=("Application.ReadWrite.All" "DelegatedPermissionGrant.ReadWrite.All")
+  APPLICATION_PERMISSIONS+=("DelegatedPermissionGrant.ReadWrite.All")
 fi


create_aad_assets.sh no longer adds Application.ReadWrite.All when AUTO_GRANT_WORKSPACE_CONSENT=true (it only adds DelegatedPermissionGrant.ReadWrite.All). Several docs/comments in this PR still state that enabling auto-grant consent grants both permissions, and setups with consent enabled but auto workspace app registration disabled may now be missing required permissions. Either update the docs/comments to match the new behavior or add Application.ReadWrite.All in this branch to keep the documented contract.

@copilot update the docs.

devops/scripts/aad/add_automation_admin_to_workspace_application.sh

Copilot · 2026-03-12T13:08:19Z

docs/tre-developers/end-to-end-tests.md

+## Manually created workspace application for targeted tests
+
+Most E2E suites now rely on automatically created workspace applications, so you no longer need to provision a manual app registration for standard runs.
+
+The `test_manually_created_application_owner_token` test (included in the `extended` marker set) exercises the manual-authentication flow. Its fixture automatically runs `devops/scripts/aad/create_workspace_application.sh` to create or reuse a workspace application before deploying the test workspace.
+
+Ensure `az` CLI is installed, you are logged in to the correct tenant (`az login -t <tenant>`), and `APPLICATION_ADMIN_CLIENT_ID` (the application admin app registration) is configured so the script can add the necessary owner.
+
+Run `make test-e2e-custom SELECTOR='manual_app'` to exercise the same flow.


This section references a test_manually_created_application_owner_token test in the extended marker set, but the PR adds e2e_tests/test_manual_workspace.py::test_manually_created_application_workspace marked with @pytest.mark.manual_app (and the new /test-manual-app command). Please update the doc to match the actual test name/marker so contributors can run the right suite.

@copilot open a new pull request to apply changes based on this feedback

Copilot · 2026-03-12T13:08:20Z

templates/workspaces/base/parameters.json

    },
    {
      "name": "client_secret",
      "source": {
        "env": "CLIENT_SECRET"
      }
    },


parameters.json still defines a client_secret parameter sourced from CLIENT_SECRET, but the workspace bundles removed the client_secret parameter from porter.yaml/template schema. This mismatch is likely to cause Porter to fail bundle execution due to an unknown parameter. Remove the client_secret entry from this parameter set (and any related env var wiring) to match the new bundle interface.

@copilot open a new pull request to apply changes based on this feedback

Remove auth_type field entirely from template_schema.json. The new approach makes client_id an always-visible optional top-level field - if provided, Terraform imports the existing AAD app; if empty, Terraform creates one automatically. This eliminates the deprecated API-side Graph validation that blocked workspace creation with bare app registrations. Also moves aad_redirect_uris and create_aad_groups to regular top-level properties (no longer conditional on auth_type). Fixes microsoft#2247

marrobi added 6 commits November 25, 2025 17:14

Use Group.Create instead of Group.ReadWrite.All for group creation

8a391ec

Fixes microsoft#4772

Deployment works, permetations need testing, and docs updating.

481daf8

Update scripts and docs.

6904b72

Remove need for Directory.Read.All

c755f30

Rotating secret

9fd75cf

Merge branch 'main' of https://github.com/microsoft/AzureTRE into mar…

6114d09

…robi/issue2247

marrobi changed the title ~~Simplify Workspace Entra ID Automation and Remove Directory.Read.All Dependency~~ Remove need to provide client secret when creating workspace and remove Directory.Read.All Dependency Nov 26, 2025

marrobi linked an issue Nov 26, 2025 that may be closed by this pull request

Supply Workspace client secret without having to pass it in the API #2247

Open

3 tasks

marrobi changed the title ~~Remove need to provide client secret when creating workspace and remove Directory.Read.All Dependency~~ Remove need to provide client secret when creating workspace and remove Directory.Read.All Dependency from automation admin Nov 26, 2025

marrobi linked an issue Nov 27, 2025 that may be closed by this pull request

Add docs for rotating workspace app registration secrets. #2903

Open

Merge branch 'main' into marrobi/issue2247

e7479bd

marrobi requested a review from Copilot November 27, 2025 10:34

Copilot started reviewing on behalf of marrobi November 27, 2025 10:35 View session

Copilot finished reviewing on behalf of marrobi November 27, 2025 10:38

Copilot AI reviewed Nov 27, 2025

View reviewed changes

marrobi requested review from Copilot and removed request for Copilot November 27, 2025 10:42

Update docs/tre-admins/auth.md

1b9c090

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Copilot started reviewing on behalf of marrobi November 27, 2025 10:43 View session

marrobi and others added 3 commits November 27, 2025 10:44

Update docs/tre-admins/identities/application_admin.md

ccf5aee

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Update docs/tre-admins/environment-variables.md

f93845e

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Update docs/tre-admins/identities/application_admin.md

b79940f

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Copilot finished reviewing on behalf of marrobi November 27, 2025 10:47

Copilot finished reviewing on behalf of marrobi November 27, 2025 10:48

Copilot AI reviewed Nov 27, 2025

View reviewed changes

Update templates/workspaces/base/terraform/aad/aad.tf

173ebd1

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

marrobi assigned Copilot Nov 27, 2025

marrobi added 2 commits January 26, 2026 10:40

Add dependency on ampls_log_anaytics for azure_monitor_private_endpoint

3687a5f

Merge branch 'main' into marrobi/issue2247

1cac0eb

marrobi added 3 commits January 26, 2026 15:54

Update e2e helpers to verify token roles.

d6d4288

Merge branch 'marrobi/issue2247' of https://github.com/marrobi/AzureTRE…

4dc7347

… into marrobi/issue2247

blank line

6769d1a

marrobi closed this Jan 26, 2026

marrobi reopened this Jan 26, 2026

marrobi mentioned this pull request Feb 4, 2026

Make Directory.Read.All optional for non-admin workspace listing (graceful fallback) #4682

Closed

2 tasks

marrobi added 7 commits March 10, 2026 12:49

Refactor AAD authentication service to improve UUID validation and er…

39e7f98

…ror handling; remove deprecated redirect URL update script.

Enhance Azure AD application configuration to support existing identi…

73497dd

…fiers and improve role ID imports

Fix bugs

985dea6

Add additonal imports.

445ef95

Revert tf lint

c5e2bc7

fix linting

ec8d3a9

marrobi marked this pull request as ready for review March 12, 2026 13:03

marrobi requested a review from a team as a code owner March 12, 2026 13:03

marrobi requested a review from Copilot March 12, 2026 13:03

Copilot started reviewing on behalf of marrobi March 12, 2026 13:03 View session

Copilot AI reviewed Mar 12, 2026

View reviewed changes

Conversation

marrobi commented Nov 26, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Changes

Migration

Uh oh!

github-actions bot commented Nov 26, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Unit Test Results

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

github-actions bot commented Jan 26, 2026

Uh oh!

marrobi commented Jan 26, 2026

Uh oh!

github-actions bot commented Jan 26, 2026

Uh oh!

marrobi commented Jan 26, 2026

Uh oh!

github-actions bot commented Jan 26, 2026

Uh oh!

marrobi commented Mar 11, 2026

Uh oh!

github-actions bot commented Mar 11, 2026

Uh oh!

github-actions bot commented Mar 11, 2026

Uh oh!

github-actions bot commented Mar 12, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Copilot AI Mar 12, 2026

Choose a reason for hiding this comment

Uh oh!

marrobi Mar 12, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Copilot AI Mar 12, 2026

Choose a reason for hiding this comment

Uh oh!

marrobi Mar 12, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 12, 2026

Choose a reason for hiding this comment

Uh oh!

marrobi Mar 12, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

marrobi commented Nov 26, 2025 •

edited

Loading

github-actions bot commented Nov 26, 2025 •

edited

Loading