Skip to content

Commit ef712f3

Browse files
wanghanchengchnleokondrashov
wanghanchengchn
authored and
leokondrashov
committed
Integrate Azure Functions deployment
Signed-off-by: hancheng wang <wanghanchengchn@gmail.com> Integrate Azure Functions deployment Signed-off-by: hancheng wang <wanghanchengchn@gmail.com> Integrate Azure Functions deployment Signed-off-by: hancheng wang <wanghanchengchn@gmail.com> Integrate Azure Functions deployment Signed-off-by: hancheng wang <wanghanchengchn@gmail.com> Integrate Azure Functions deployment Signed-off-by: hancheng wang <wanghanchengchn@gmail.com> Integrate Azure Functions deployment Signed-off-by: hancheng wang <wanghanchengchn@gmail.com> Integrate Azure Functions deployment Signed-off-by: hancheng wang <wanghanchengchn@gmail.com> Integrate Azure Functions deployment Signed-off-by: hancheng wang <wanghanchengchn@gmail.com> Integrate Azure Functions deployment Signed-off-by: hancheng wang <wanghanchengchn@gmail.com>
1 parent a6ab45c commit ef712f3

File tree

22 files changed

+1291
-35
lines changed

22 files changed

+1291
-35
lines changed

‎.github/workflows/azure_integration_test.yaml‎

Lines changed: 65 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,65 @@
1+
name: Azure Integration Tests
2+
3+
on:
4+
push:
5+
branches:
6+
- main # Trigger the workflow when code is pushed to the main branch
7+
pull_request:
8+
branches:
9+
- main # Trigger the workflow when the PR targets the main branch
10+
workflow_dispatch: # Allows manual triggering of the workflow
11+
12+
env:
13+
GOOS: linux
14+
GO111MODULE: on
15+
16+
jobs:
17+
test-azure:
18+
name: Run Azure Integration Tests
19+
runs-on: ubuntu-24.04
20+
env:
21+
AZURE_APP_ID: ${{ secrets.AZURE_APP_ID }}
22+
AZURE_PASSWORD: ${{ secrets.AZURE_PASSWORD }}
23+
AZURE_TENANT: ${{ secrets.AZURE_TENANT }}
24+
25+
steps:
26+
- name: Check if environment variables are set # Validate secrets are passed
27+
run: |
28+
if [[ -z "$AZURE_APP_ID" ]]; then
29+
echo "AZURE_APP_ID is not set. Please check if secrets.AZURE_APP_ID is in the repository."
30+
exit 1
31+
fi
32+
if [[ -z "$AZURE_PASSWORD" ]]; then
33+
echo "AZURE_PASSWORD is not set. Please check if secrets.AZURE_PASSWORD is in the repository."
34+
exit 1
35+
fi
36+
if [[ -z "$AZURE_TENANT" ]]; then
37+
echo "AZURE_TENANT is not set. Please check if secrets.AZURE_TENANT is in the repository."
38+
exit 1
39+
fi
40+
41+
- name: Checkout GitHub Repository
42+
uses: actions/checkout@v4
43+
with:
44+
lfs: true
45+
46+
- name: Install Azure CLI
47+
run: |
48+
curl -sL https://aka.ms/InstallAzureCLIDeb | sudo bash
49+
az --version
50+
51+
- name: Install Golang
52+
uses: actions/setup-go@v5
53+
with:
54+
go-version: 1.22
55+
56+
- name: Set up Python 3.10
57+
uses: actions/setup-python@v4
58+
with:
59+
python-version: '3.10'
60+
61+
- name: Azure CLI Login Using Service Principal
62+
run: az login --service-principal --username "$AZURE_APP_ID" --password "$AZURE_PASSWORD" --tenant "$AZURE_TENANT"
63+
64+
- name: Run Azure-related integration test
65+
run: go test -v ./pkg/driver/deployment/azure_functions_test.go

‎.github/workflows/e2e_azure.yaml‎

Lines changed: 68 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,68 @@
1+
name: End-to-End Azure Functions Tests
2+
3+
on:
4+
push:
5+
branches:
6+
- main # Trigger the workflow when code is pushed to the main branch
7+
pull_request:
8+
branches:
9+
- main # Trigger the workflow when the PR targets the main branch
10+
workflow_dispatch: # Allows manual triggering of the workflow
11+
12+
env:
13+
GOOS: linux
14+
GO111MODULE: on
15+
16+
jobs:
17+
test-azure:
18+
name: Test E2E Azure Functions Cloud Deployment
19+
runs-on: ubuntu-24.04
20+
env:
21+
AZURE_APP_ID: ${{ secrets.AZURE_APP_ID }}
22+
AZURE_PASSWORD: ${{ secrets.AZURE_PASSWORD }}
23+
AZURE_TENANT: ${{ secrets.AZURE_TENANT }}
24+
25+
steps:
26+
- name: Check if environment variables are set # Validate secrets are passed
27+
run: |
28+
if [[ -z "$AZURE_APP_ID" ]]; then
29+
echo "AZURE_APP_ID is not set. Please check if secrets.AZURE_APP_ID is in the repository."
30+
exit 1
31+
fi
32+
if [[ -z "$AZURE_PASSWORD" ]]; then
33+
echo "AZURE_PASSWORD is not set. Please check if secrets.AZURE_PASSWORD is in the repository."
34+
exit 1
35+
fi
36+
if [[ -z "$AZURE_TENANT" ]]; then
37+
echo "AZURE_TENANT is not set. Please check if secrets.AZURE_TENANT is in the repository."
38+
exit 1
39+
fi
40+
41+
- name: Checkout GitHub Repository
42+
uses: actions/checkout@v4
43+
with:
44+
lfs: true
45+
46+
- name: Install Azure CLI
47+
run: |
48+
curl -sL https://aka.ms/InstallAzureCLIDeb | sudo bash
49+
az --version
50+
51+
- name: Install Golang
52+
uses: actions/setup-go@v5
53+
with:
54+
go-version: 1.22
55+
56+
- name: Set up Python 3.10
57+
uses: actions/setup-python@v4
58+
with:
59+
python-version: '3.10'
60+
61+
- name: Azure CLI Login Using Service Principal
62+
run: az login --service-principal --username "$AZURE_APP_ID" --password "$AZURE_PASSWORD" --tenant "$AZURE_TENANT"
63+
64+
- name: Build and Run Loader
65+
run: go run cmd/loader.go --config cmd/config_azure_trace.json
66+
67+
- name: Check the output
68+
run: test -f "data/out/experiment_duration_5.csv" && test $(grep true data/out/experiment_duration_5.csv | wc -l) -eq 0 # test the output file for errors (true means failure to invoke)

‎.gitignore‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -211,3 +211,9 @@ tools/plotter/test-out
211211

212212
data/traces/azure_*
213213
data/traces/day*
214+
215+
# Azure deployment
216+
azure_functions_for_zip/
217+
azurefunctions_setup/shared_azure_workload/exec_func.py
218+
azure_functions_for_zip_test/
219+
function*.zip

‎azurefunctions_setup/azurefunctionsconfig.yaml‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
# azurefunctionsconfig.yaml
2+
azurefunctionsconfig:
3+
resource_group: invitro-rg # Name of the resource group
4+
storage_account_name: invitrostorage # Name of the storage account
5+
function_app_name: invitro-functionapp # Name of the function app
6+
location: EastUS # Region where resource created

‎azurefunctions_setup/host.json‎

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,16 @@
1+
{
2+
"version": "2.0",
3+
"logging": {
4+
"applicationInsights": {
5+
"samplingSettings": {
6+
"isEnabled": true,
7+
"excludedTypes": "Request"
8+
}
9+
}
10+
},
11+
"extensionBundle": {
12+
"id": "Microsoft.Azure.Functions.ExtensionBundle",
13+
"version": "[4.*, 5.0.0)"
14+
}
15+
16+
}

‎azurefunctions_setup/local.settings.json‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
{
2+
"IsEncrypted": false,
3+
"Values": {
4+
"FUNCTIONS_WORKER_RUNTIME": "python",
5+
"AzureWebJobsFeatureFlags": "EnableWorkerIndexing",
6+
"AzureWebJobsStorage": ""
7+
}
8+
}

‎azurefunctions_setup/requirements.txt‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
azure-functions
2+
numpy>=1.21,<1.26
3+
psutil>=5.9,<6.0

‎azurefunctions_setup/shared_azure_workload/azurefunctionsworkload.py‎

Lines changed: 56 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,56 @@
1+
import time
2+
import socket
3+
import json
4+
import azure.functions as func
5+
import logging
6+
7+
# Note: exec_func.py is not stored here permanently. It is copied from server/trace-func-py during setup.
8+
from .exec_func import execute_function
9+
10+
# Global variable for hostname
11+
hostname = socket.gethostname()
12+
13+
def main(req: func.HttpRequest) -> func.HttpResponse:
14+
logging.info("Processing request.")
15+
16+
start_time = time.time()
17+
18+
# Parse JSON request body
19+
try:
20+
req_body = req.get_json()
21+
logging.info(f"Request body: {req_body}")
22+
except ValueError:
23+
logging.error("Invalid JSON received.")
24+
return func.HttpResponse(
25+
json.dumps({"error": "Invalid JSON"}),
26+
status_code=400,
27+
mimetype="application/json"
28+
)
29+
30+
runtime_milliseconds = req_body.get('RuntimeInMilliSec', 1000)
31+
memory_mebibytes = req_body.get('MemoryInMebiBytes', 128)
32+
33+
logging.info(f"Runtime requested: {runtime_milliseconds} ms, Memory: {memory_mebibytes} MiB")
34+
35+
# Call the execute_function, which needs to be copied from server/trace-func-py/exec_func.py
36+
duration = execute_function("",runtime_milliseconds,memory_mebibytes)
37+
result_msg = f"Workload completed in {duration} microseconds"
38+
39+
# Prepare the response
40+
response = {
41+
"Status": "Success",
42+
"Function": req.url.split("/")[-1],
43+
"MachineName": hostname,
44+
"ExecutionTime": int((time.time() - start_time) * 1_000_000), # Total time (includes HTTP, workload, and response prep)
45+
"DurationInMicroSec": duration, # Time spent on the workload itself
46+
"MemoryUsageInKb": memory_mebibytes * 1024,
47+
"Message": result_msg
48+
}
49+
50+
logging.info(f"Response: {response}")
51+
52+
return func.HttpResponse(
53+
json.dumps(response),
54+
status_code=200,
55+
mimetype="application/json"
56+
)

‎azurefunctions_setup/shared_azure_workload/function.json‎

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,18 @@
1+
{
2+
"bindings": [
3+
{
4+
"authLevel": "anonymous",
5+
"type": "httpTrigger",
6+
"direction": "in",
7+
"name": "req",
8+
"methods": ["post"]
9+
},
10+
{
11+
"type": "http",
12+
"direction": "out",
13+
"name": "$return"
14+
}
15+
],
16+
"scriptFile": "azurefunctionsworkload.py"
17+
}
18+

‎cmd/config_azure_trace.json‎

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,28 @@
1+
{
2+
"Seed": 42,
3+
4+
"Platform": "AzureFunctions",
5+
"InvokeProtocol" : "http1",
6+
"EndpointPort": 80,
7+
8+
"BusyLoopOnSandboxStartup": false,
9+
10+
"TracePath": "data/traces/example",
11+
"Granularity": "minute",
12+
"OutputPathPrefix": "data/out/experiment",
13+
"IATDistribution": "exponential",
14+
"CPULimit": "1vCPU",
15+
"ExperimentDuration": 5,
16+
"WarmupDuration": 0,
17+
18+
"IsPartiallyPanic": false,
19+
"EnableZipkinTracing": false,
20+
"EnableMetricsScrapping": false,
21+
"MetricScrapingPeriodSeconds": 15,
22+
"AutoscalingMetric": "concurrency",
23+
24+
"GRPCConnectionTimeoutSeconds": 15,
25+
"GRPCFunctionTimeoutSeconds": 900,
26+
27+
"DAGMode": false
28+
}

0 commit comments

Comments
 (0)