Skip to content

Migrate Azure Firewall and route tables to core configuration #4342

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
marrobi merged 32 commits into from
Jun 18, 2025
Merged

Migrate Azure Firewall and route tables to core configuration #4342

Show file tree
Hide file tree
Changes from 22 commits
Commits
Show all changes
32 commits
Select commit Hold shift + click to select a range
762ca46
Add Terraform configuration for Azure Firewall and route tables to mo…
marrobi Jan 7, 2025
49da7fc
Airlock function storage to use manage identity
tamirkamara Jan 19, 2025
83d3f07
version
tamirkamara Jan 19, 2025
5015aae
changelog
tamirkamara Jan 19, 2025
92d47e7
storage permissions
tamirkamara Jan 19, 2025
fa16b21
Merge branch 'tamirkamara/function-host-storage-mi' of https://github…
marrobi Jan 28, 2025
57a30df
Fix merge issues.
marrobi Jan 29, 2025
3646280
Merge branch 'main' of https://github.com/microsoft/AzureTRE into mar…
marrobi Feb 7, 2025
e517835
remove forced tunneling from shared service
marrobi Feb 7, 2025
f942623
Fix linting
marrobi Feb 7, 2025
dbd3b15
Merge branch 'main' of https://github.com/microsoft/AzureTRE into mar…
marrobi Mar 20, 2025
ff24de2
Update to work with latest changes.
marrobi Mar 21, 2025
9cbf076
Fix linting
marrobi Mar 21, 2025
72fa0da
fix linting
marrobi Mar 21, 2025
917efeb
Update changelog
marrobi Mar 21, 2025
c73a127
Remove Firewall SKU from shared services and RP
marrobi Mar 21, 2025
07ebda0
fix linting
marrobi Mar 21, 2025
c61ee60
fix linting
marrobi Mar 21, 2025
0918606
Fix PR comments.
marrobi Apr 28, 2025
ee4ef36
Merge branch 'main' of https://github.com/microsoft/AzureTRE into mar…
marrobi Apr 28, 2025
e2ab94e
Fix migrate script issues
marrobi Apr 30, 2025
4c8dcf1
Merge branch 'main' of https://github.com/microsoft/AzureTRE into mar…
marrobi Apr 30, 2025
9db999a
Merge branch 'main' of https://github.com/microsoft/AzureTRE into mar…
marrobi Jun 11, 2025
9c43f75
Fix merge issues.
marrobi Jun 11, 2025
65ef756
Update policy name and other merge issues.
marrobi Jun 11, 2025
fde34db
Check for core resouce gorup before migration.
marrobi Jun 11, 2025
438ad94
Fix linting.
marrobi Jun 11, 2025
3761015
fix linting
marrobi Jun 11, 2025
c87a6ac
Set default firewall sku
marrobi Jun 16, 2025
6696d1c
Set script to handle if FIREWALL_SKU is not set.
marrobi Jun 16, 2025
0d7e6a4
Fix script existing if resource does not exist.
marrobi Jun 16, 2025
cc14315
Update comment
marrobi Jun 18, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
209 changes: 107 additions & 102 deletions .github/workflows/deploy_tre_reusable.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
name: Deploy Azure TRE Reusable

on: # yamllint disable-line rule:truthy
on: # yamllint disable-line rule:truthy
workflow_call:
inputs:
prRef:
Expand Down Expand Up @@ -286,7 +286,12 @@ jobs:
strategy:
fail-fast: true
matrix:
target: [build-and-push-api, build-and-push-resource-processor, build-and-push-airlock-processor]
target:
[
build-and-push-api,
build-and-push-resource-processor,
build-and-push-airlock-processor,
]

steps:
- name: Checkout
Expand Down Expand Up @@ -397,38 +402,38 @@ jobs:
strategy:
matrix:
include:
- {BUNDLE_TYPE: "workspace",
BUNDLE_DIR: "./templates/workspaces/base"}
- {BUNDLE_TYPE: "workspace",
BUNDLE_DIR: "./templates/workspaces/unrestricted"}
- {BUNDLE_TYPE: "workspace",
BUNDLE_DIR: "./templates/workspaces/airlock-import-review"}
- {BUNDLE_TYPE: "workspace_service",
BUNDLE_DIR: "./templates/workspace_services/guacamole"}
- {BUNDLE_TYPE: "workspace_service",
BUNDLE_DIR: "./templates/workspace_services/azureml"}
- {BUNDLE_TYPE: "workspace_service",
BUNDLE_DIR: "./templates/workspace_services/gitea"}
- {BUNDLE_TYPE: "workspace_service",
BUNDLE_DIR: "./templates/workspace_services/mysql"}
- {BUNDLE_TYPE: "workspace_service",
BUNDLE_DIR: "./templates/workspace_services/health-services"}
- {BUNDLE_TYPE: "workspace_service",
BUNDLE_DIR: "./templates/workspace_services/databricks"}
- {BUNDLE_TYPE: "workspace_service",
BUNDLE_DIR: "./templates/workspace_services/ohdsi"}
- {BUNDLE_TYPE: "workspace_service",
BUNDLE_DIR: "./templates/workspace_services/azuresql"}
- {BUNDLE_TYPE: "workspace_service",
BUNDLE_DIR: "./templates/workspace_services/openai"}
- {BUNDLE_TYPE: "user_resource",
BUNDLE_DIR: "./templates/workspace_services/guacamole/user_resources/guacamole-azure-windowsvm"}
- {BUNDLE_TYPE: "user_resource",
BUNDLE_DIR: "./templates/workspace_services/guacamole/user_resources/guacamole-azure-linuxvm"}
- {BUNDLE_TYPE: "user_resource",
BUNDLE_DIR: "./templates/workspace_services/guacamole/user_resources/guacamole-azure-export-reviewvm"}
- {BUNDLE_TYPE: "user_resource",
BUNDLE_DIR: "./templates/workspace_services/guacamole/user_resources/guacamole-azure-import-reviewvm"}
- BUNDLE_TYPE: "workspace"
BUNDLE_DIR: "./templates/workspaces/base"
- BUNDLE_TYPE: "workspace"
BUNDLE_DIR: "./templates/workspaces/unrestricted"
- BUNDLE_TYPE: "workspace"
BUNDLE_DIR: "./templates/workspaces/airlock-import-review"
- BUNDLE_TYPE: "workspace_service"
BUNDLE_DIR: "./templates/workspace_services/guacamole"
- BUNDLE_TYPE: "workspace_service"
BUNDLE_DIR: "./templates/workspace_services/azureml"
- BUNDLE_TYPE: "workspace_service"
BUNDLE_DIR: "./templates/workspace_services/gitea"
- BUNDLE_TYPE: "workspace_service"
BUNDLE_DIR: "./templates/workspace_services/mysql"
- BUNDLE_TYPE: "workspace_service"
BUNDLE_DIR: "./templates/workspace_services/health-services"
- BUNDLE_TYPE: "workspace_service"
BUNDLE_DIR: "./templates/workspace_services/databricks"
- BUNDLE_TYPE: "workspace_service"
BUNDLE_DIR: "./templates/workspace_services/ohdsi"
- BUNDLE_TYPE: "workspace_service"
BUNDLE_DIR: "./templates/workspace_services/azuresql"
- BUNDLE_TYPE: "workspace_service"
BUNDLE_DIR: "./templates/workspace_services/openai"
- BUNDLE_TYPE: "user_resource"
BUNDLE_DIR: "./templates/workspace_services/guacamole/user_resources/guacamole-azure-windowsvm"
- BUNDLE_TYPE: "user_resource"
BUNDLE_DIR: "./templates/workspace_services/guacamole/user_resources/guacamole-azure-linuxvm"
- BUNDLE_TYPE: "user_resource"
BUNDLE_DIR: "./templates/workspace_services/guacamole/user_resources/guacamole-azure-export-reviewvm"
- BUNDLE_TYPE: "user_resource"
BUNDLE_DIR: "./templates/workspace_services/guacamole/user_resources/guacamole-azure-import-reviewvm"
environment: ${{ inputs.environmentName }}
steps:
- name: Checkout
Expand Down Expand Up @@ -464,22 +469,22 @@ jobs:
strategy:
matrix:
include:
- {BUNDLE_TYPE: "shared_service",
BUNDLE_DIR: "./templates/shared_services/firewall/"}
- {BUNDLE_TYPE: "shared_service",
BUNDLE_DIR: "./templates/shared_services/gitea/"}
- {BUNDLE_TYPE: "shared_service",
BUNDLE_DIR: "./templates/shared_services/admin-vm/"}
- {BUNDLE_TYPE: "shared_service",
BUNDLE_DIR: "./templates/shared_services/airlock_notifier/"}
- {BUNDLE_TYPE: "shared_service",
BUNDLE_DIR: "./templates/shared_services/certs/"}
- {BUNDLE_TYPE: "shared_service",
BUNDLE_DIR: "./templates/shared_services/cyclecloud/"}
- {BUNDLE_TYPE: "shared_service",
BUNDLE_DIR: "./templates/shared_services/sonatype-nexus-vm/"}
- {BUNDLE_TYPE: "shared_service",
BUNDLE_DIR: "./templates/shared_services/databricks-auth/"}
- BUNDLE_TYPE: "shared_service"
BUNDLE_DIR: "./templates/shared_services/firewall/"
- BUNDLE_TYPE: "shared_service"
BUNDLE_DIR: "./templates/shared_services/gitea/"
- BUNDLE_TYPE: "shared_service"
BUNDLE_DIR: "./templates/shared_services/admin-vm/"
- BUNDLE_TYPE: "shared_service"
BUNDLE_DIR: "./templates/shared_services/airlock_notifier/"
- BUNDLE_TYPE: "shared_service"
BUNDLE_DIR: "./templates/shared_services/certs/"
- BUNDLE_TYPE: "shared_service"
BUNDLE_DIR: "./templates/shared_services/cyclecloud/"
- BUNDLE_TYPE: "shared_service"
BUNDLE_DIR: "./templates/shared_services/sonatype-nexus-vm/"
- BUNDLE_TYPE: "shared_service"
BUNDLE_DIR: "./templates/shared_services/databricks-auth/"
environment: ${{ inputs.environmentName }}
steps:
- name: Checkout
Expand Down Expand Up @@ -515,22 +520,22 @@ jobs:
strategy:
matrix:
include:
- {BUNDLE_TYPE: "shared_service",
BUNDLE_DIR: "./templates/shared_services/firewall"}
- {BUNDLE_TYPE: "shared_service",
BUNDLE_DIR: "./templates/shared_services/gitea"}
- {BUNDLE_TYPE: "shared_service",
BUNDLE_DIR: "./templates/shared_services/admin-vm/"}
- {BUNDLE_TYPE: "shared_service",
BUNDLE_DIR: "./templates/shared_services/airlock_notifier/"}
- {BUNDLE_TYPE: "shared_service",
BUNDLE_DIR: "./templates/shared_services/certs/"}
- {BUNDLE_TYPE: "shared_service",
BUNDLE_DIR: "./templates/shared_services/cyclecloud/"}
- {BUNDLE_TYPE: "shared_service",
BUNDLE_DIR: "./templates/shared_services/sonatype-nexus-vm/"}
- {BUNDLE_TYPE: "shared_service",
BUNDLE_DIR: "./templates/shared_services/databricks-auth/"}
- BUNDLE_TYPE: "shared_service"
BUNDLE_DIR: "./templates/shared_services/firewall"
- BUNDLE_TYPE: "shared_service"
BUNDLE_DIR: "./templates/shared_services/gitea"
- BUNDLE_TYPE: "shared_service"
BUNDLE_DIR: "./templates/shared_services/admin-vm/"
- BUNDLE_TYPE: "shared_service"
BUNDLE_DIR: "./templates/shared_services/airlock_notifier/"
- BUNDLE_TYPE: "shared_service"
BUNDLE_DIR: "./templates/shared_services/certs/"
- BUNDLE_TYPE: "shared_service"
BUNDLE_DIR: "./templates/shared_services/cyclecloud/"
- BUNDLE_TYPE: "shared_service"
BUNDLE_DIR: "./templates/shared_services/sonatype-nexus-vm/"
- BUNDLE_TYPE: "shared_service"
BUNDLE_DIR: "./templates/shared_services/databricks-auth/"
environment: ${{ inputs.environmentName }}
steps:
- name: Checkout
Expand Down Expand Up @@ -573,28 +578,28 @@ jobs:
matrix:
include:
# bundles type can be inferred from the bundle dir (but this is more explicit)
- {BUNDLE_TYPE: "workspace",
BUNDLE_DIR: "./templates/workspaces/base"}
- {BUNDLE_TYPE: "workspace",
BUNDLE_DIR: "./templates/workspaces/unrestricted"}
- {BUNDLE_TYPE: "workspace",
BUNDLE_DIR: "./templates/workspaces/airlock-import-review"}
- {BUNDLE_TYPE: "workspace_service",
BUNDLE_DIR: "./templates/workspace_services/guacamole"}
- {BUNDLE_TYPE: "workspace_service",
BUNDLE_DIR: "./templates/workspace_services/azureml"}
- {BUNDLE_TYPE: "workspace_service",
BUNDLE_DIR: "./templates/workspace_services/gitea"}
- {BUNDLE_TYPE: "workspace_service",
BUNDLE_DIR: "./templates/workspace_services/mysql"}
- {BUNDLE_TYPE: "workspace_service",
BUNDLE_DIR: "./templates/workspace_services/health-services"}
- {BUNDLE_TYPE: "workspace_service",
BUNDLE_DIR: "./templates/workspace_services/databricks"}
- {BUNDLE_TYPE: "workspace_service",
BUNDLE_DIR: "./templates/workspace_services/ohdsi"}
- {BUNDLE_TYPE: "workspace_service",
BUNDLE_DIR: "./templates/workspace_services/azuresql"}
- BUNDLE_TYPE: "workspace"
BUNDLE_DIR: "./templates/workspaces/base"
- BUNDLE_TYPE: "workspace"
BUNDLE_DIR: "./templates/workspaces/unrestricted"
- BUNDLE_TYPE: "workspace"
BUNDLE_DIR: "./templates/workspaces/airlock-import-review"
- BUNDLE_TYPE: "workspace_service"
BUNDLE_DIR: "./templates/workspace_services/guacamole"
- BUNDLE_TYPE: "workspace_service"
BUNDLE_DIR: "./templates/workspace_services/azureml"
- BUNDLE_TYPE: "workspace_service"
BUNDLE_DIR: "./templates/workspace_services/gitea"
- BUNDLE_TYPE: "workspace_service"
BUNDLE_DIR: "./templates/workspace_services/mysql"
- BUNDLE_TYPE: "workspace_service"
BUNDLE_DIR: "./templates/workspace_services/health-services"
- BUNDLE_TYPE: "workspace_service"
BUNDLE_DIR: "./templates/workspace_services/databricks"
- BUNDLE_TYPE: "workspace_service"
BUNDLE_DIR: "./templates/workspace_services/ohdsi"
- BUNDLE_TYPE: "workspace_service"
BUNDLE_DIR: "./templates/workspace_services/azuresql"

environment: ${{ inputs.environmentName }}
steps:
Expand Down Expand Up @@ -638,18 +643,18 @@ jobs:
strategy:
matrix:
include:
- {BUNDLE_TYPE: "user_resource",
BUNDLE_DIR: "./templates/workspace_services/guacamole/user_resources/guacamole-azure-windowsvm",
WORKSPACE_SERVICE_NAME: "tre-service-guacamole"}
- {BUNDLE_TYPE: "user_resource",
BUNDLE_DIR: "./templates/workspace_services/guacamole/user_resources/guacamole-azure-linuxvm",
WORKSPACE_SERVICE_NAME: "tre-service-guacamole"}
- {BUNDLE_TYPE: "user_resource",
BUNDLE_DIR: "./templates/workspace_services/guacamole/user_resources/guacamole-azure-export-reviewvm",
WORKSPACE_SERVICE_NAME: "tre-service-guacamole"}
- {BUNDLE_TYPE: "user_resource",
BUNDLE_DIR: "./templates/workspace_services/guacamole/user_resources/guacamole-azure-import-reviewvm",
WORKSPACE_SERVICE_NAME: "tre-service-guacamole"}
- BUNDLE_TYPE: "user_resource"
BUNDLE_DIR: "./templates/workspace_services/guacamole/user_resources/guacamole-azure-windowsvm"
WORKSPACE_SERVICE_NAME: "tre-service-guacamole"
- BUNDLE_TYPE: "user_resource"
BUNDLE_DIR: "./templates/workspace_services/guacamole/user_resources/guacamole-azure-linuxvm"
WORKSPACE_SERVICE_NAME: "tre-service-guacamole"
- BUNDLE_TYPE: "user_resource"
BUNDLE_DIR: "./templates/workspace_services/guacamole/user_resources/guacamole-azure-export-reviewvm"
WORKSPACE_SERVICE_NAME: "tre-service-guacamole"
- BUNDLE_TYPE: "user_resource"
BUNDLE_DIR: "./templates/workspace_services/guacamole/user_resources/guacamole-azure-import-reviewvm"
WORKSPACE_SERVICE_NAME: "tre-service-guacamole"
environment: ${{ inputs.environmentName }}
steps:
- name: Checkout
Expand Down Expand Up @@ -714,7 +719,6 @@ jobs:
TEST_ACCOUNT_CLIENT_SECRET: "${{ secrets.TEST_ACCOUNT_CLIENT_SECRET }}"
TRE_ID: ${{ secrets.TRE_ID }}
LOCATION: ${{ vars.LOCATION }}
FIREWALL_SKU: ${{ vars.FIREWALL_SKU}}

- name: State Store Migrations
uses: ./.github/actions/devcontainer_run_command
Expand Down Expand Up @@ -812,7 +816,8 @@ jobs:
if: ${{ inputs.e2eTestsCustomSelector != '' }}
runs-on: ubuntu-latest
environment: ${{ inputs.environmentName }}
needs: [deploy_shared_services, register_bundles, register_user_resource_bundles]
needs:
[deploy_shared_services, register_bundles, register_user_resource_bundles]
timeout-minutes: 300
steps:
- name: Checkout
Expand Down
1 change: 1 addition & 0 deletions CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@
ENHANCEMENTS:
* Add ability to pass values to install stage on pipleine [#4451](https://github.com/microsoft/AzureTRE/pull/4451)
* Format the error message in the Operations panel for enhanced readability ([#4493](https://github.com/microsoft/AzureTRE/issues/4493))
* Migrate Azure Firewall and Route Tables to Core Terraform ([#4342](https://github.com/microsoft/AzureTRE/pull/4342))
* Added ability to assign VMs to other users at creation time ([#1179](https://github.com/microsoft/AzureTRE/issues/1179))

BUG FIXES:
Expand Down
21 changes: 1 addition & 20 deletions Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -43,12 +43,6 @@ build-and-push-airlock-processor: build-airlock-processor push-airlock-processor
help: ## 💬 This help message :)
@grep -E '[a-zA-Z_-]+:.*?## .*$$' $(firstword $(MAKEFILE_LIST)) | awk 'BEGIN {FS = ":.*?## "}; {printf "\033[36m%-25s\033[0m %s\n", $$1, $$2}'

# Description: Migrate the firewall state from the core deployment to the shared services deployment.
# This is a one-time operation and should only be run when you are moving from the core deployment to the shared services deployment.
# This command will remove the firewall state from the core deployment and import it into the shared services deployment.
# Example: make migrate-firewall-state
migrate-firewall-state: prepare-tf-state

# Description: Bootstrap Terraform
# Example: make bootstrap
bootstrap:
Expand Down Expand Up @@ -133,18 +127,6 @@ push-resource-processor-vm-porter-image:
push-airlock-processor:
$(call push_image,"airlock-processor","${MAKEFILE_DIR}/airlock_processor/_version.py")

# Description: Prepare terraform state for migration
# # These targets are for a graceful migration of Firewall
# # from terraform state in Core to a Shared Service.
# # See https://github.com/microsoft/AzureTRE/issues/1177
# Example: make prepare-tf-state
prepare-tf-state:
$(call target_title, "Preparing terraform state") \
&& . ${MAKEFILE_DIR}/devops/scripts/check_dependencies.sh nodocker,env \
&& pushd ${MAKEFILE_DIR}/core/terraform > /dev/null && ../../shared_services/firewall/terraform/remove_state.sh && popd > /dev/null \
&& pushd ${MAKEFILE_DIR}/templates/shared_services/firewall/terraform > /dev/null && ./import_state.sh && popd > /dev/null
# / End migration targets

# Description: Deploy the core infrastructure of TRE.
# This will create the core resource group (named rg-<TRE_ID>) with the necessary resources.
# Example: make deploy-core
Expand Down Expand Up @@ -436,8 +418,7 @@ deploy-shared-service:
firewall-install:
. ${MAKEFILE_DIR}/devops/scripts/check_dependencies.sh env \
&& $(MAKE) bundle-build bundle-publish bundle-register deploy-shared-service \
DIR=${MAKEFILE_DIR}/templates/shared_services/firewall/ BUNDLE_TYPE=shared_service \
PROPS="$${FIREWALL_SKU+--firewall_sku $${FIREWALL_SKU} }$${FIREWALL_FORCE_TUNNEL_IP+--firewall_force_tunnel_ip $${FIREWALL_FORCE_TUNNEL_IP} }"
DIR=${MAKEFILE_DIR}/templates/shared_services/firewall/ BUNDLE_TYPE=shared_service

# Description: Upload the static website to the storage account
# Example: make static-web-upload
Expand Down
26 changes: 13 additions & 13 deletions core/terraform/.terraform.lock.hcl
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading
Loading