You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
└── overlays <- Overlays for specific Kubernetes Clusters
70
70
├── aks
71
71
│ ├── configs
72
-
│ │ ├── patch.json.tmpl <- To be converted into patch.json, patch for Data Controller control.json
72
+
│ │ └── patch.json.tmpl <- To be converted into patch.json, patch for Data Controller control.json
73
73
│ └── kustomization.yaml
74
74
├── kubeadm
75
75
│ ├── configs
@@ -112,7 +112,7 @@ There are two files that need to be generated to localize the launcher to run in
112
112
A filled-out sample of the `.test.env` file, generated based on `.test.env.tmpl` is shared below with inline commentary.
113
113
114
114
> [!IMPORTANT]
115
-
> The `export VAR="value"` syntax below is not meant to be run locally to source environment variables from your machine - but is there for the launcher. The launcher mounts this `.test.env` file **as-is** as a Kubernetes `secret` using Kustomize's [`secretGenerator`](https://github.com/kubernetes-sigs/kustomize/blob/master/examples/secretGeneratorPlugin.md#secret-values-from-local-files) (Kustomize takes a file, and turns it into a Kubernetes secret). During initialization, the launcher runs bash's [`source`](https://ss64.com/bash/source.html) command, which imports the environment variables from the as-is mounted `.test.env` file into the launcher's environment.
115
+
> The `export VAR="value"` syntax below is not meant to be run locally to source environment variables from your machine - but is there for the launcher. The launcher mounts this `.test.env` file **as-is** as a Kubernetes `secret` using Kustomize's [`secretGenerator`](https://github.com/kubernetes-sigs/kustomize/blob/master/examples/secretGeneratorPlugin.md#secret-values-from-local-files) (Kustomize takes a file, base64 encodes the entire file's content, and turns it into a Kubernetes secret). During initialization, the launcher runs bash's [`source`](https://ss64.com/bash/source.html) command, which imports the environment variables from the as-is mounted `.test.env` file into the launcher's environment.
116
116
117
117
In other words, after copy-pasting `.test.env.tmpl` and editing to create `.test.env`, the generated file should look similar to the sample below. The process to fill out the `.test.env` file is identical across operating systems and terminals.
#Arcdata extension version override - see detailed explanation below [2]
148
+
#"arcdata" Azure CLI extension version override - see detailed explanation below [2]
149
149
export ARC_DATASERVICES_WHL_OVERRIDE=""
150
150
151
151
# ================
@@ -240,7 +240,7 @@ The extension version to release-train (`ARC_DATASERVICES_EXTENSION_RELEASE_TRAI
240
240
241
241
> Optional: leave this empty in `.test.env` to use the pre-packaged default.
242
242
243
-
The launcher image is pre-packaged with the latest arcdata CLI version at the time of each container image release. However, to work with older releases, it may be necessary to provide the launcher with Azure CLI Blob URL download link, to override the pre-packaged version; e.g to instruct the launcher to install version **1.4.3**, fill in:
243
+
The launcher image is pre-packaged with the latest arcdata CLI version at the time of each container image release. However, to work with older releases and upgrade testing, it may be necessary to provide the launcher with Azure CLI Blob URL download link, to override the pre-packaged version; e.g to instruct the launcher to install version **1.4.3**, fill in:
@@ -251,7 +251,7 @@ The CLI version to Blob URL mapping can be found [here](https://azcliextensionsy
251
251
252
252
> Mandatory: this is required for Connected Cluster Custom Location creation.
253
253
254
-
The following steps are sourced from [Enable custom locations on your cluster](../kubernetes/custom-locations.md#enable-custom-locations-on-your-cluster) to retrieve the Custom Location OID for your Azure AD tenant.
254
+
The following steps are sourced from [Enable custom locations on your cluster](../kubernetes/custom-locations.md#enable-custom-locations-on-your-cluster) to retrieve the unique Custom Location Object ID for your Azure AD tenant.
255
255
256
256
There are two approaches to obtaining the `CUSTOM_LOCATION_OID` for your Azure AD tenant.
257
257
@@ -302,9 +302,9 @@ To use the Azure CLI instead, see [`az storage account generate-sas`](/cli/azure
302
302
303
303
> Optional: leave this empty in `.test.env` to run all stages (equivalent to `0` or blank)
304
304
305
-
The launcher exposes `SKIP_*` variables, to run and skip specific stages.
305
+
The launcher exposes `SKIP_*` variables, to run and skip specific stages - for example, to perform a "cleanup only" run.
306
306
307
-
For example, a "cleanup only" run. Although the launcher is designed to clean up both in the beginning and the end of each run, it's possible forlaunch and/or test-failures to leave residue resources behind. To run the launcherin"cleanup only" mode, set the following variables in`.test.env`:
307
+
Although the launcher is designed to clean up both in the beginning and the end of each run, it's possible forlaunch and/or test-failures to leave residue resources behind. To run the launcherin"cleanup only" mode, set the following variables in`.test.env`:
308
308
309
309
```bash
310
310
export SKIP_PRECLEAN="0"# Run cleanup
@@ -345,14 +345,7 @@ Finished sample of `patch.json`:
345
345
"op": "add",
346
346
"path": "spec.storage.logs.className",
347
347
"value": "default"
348
-
},
349
-
{
350
-
"op": "add",
351
-
"path": "spec.monitoring",
352
-
"value": {
353
-
"enableOpenTelemetry": true
354
-
}
355
-
}
348
+
}
356
349
]
357
350
}
358
351
```
@@ -371,14 +364,13 @@ images:
371
364
newTag: v1.11.0_2022-09-13
372
365
```
373
366
374
-
> [!TIP]
375
-
> At this point - there are **3** places we specified `imageTag`s, for clarity, here's an explanation of the different uses of each. Typically - when testing a given release, all 3 values would be the same:
376
-
>|# | Filename | Variable name | Why? | Used by? |
>| 1 |**`.test.env`**|`DOCKER_TAG`| Sourcing the [Bootstrapper image](https://mcr.microsoft.com/v2/arcdata/arc-bootstrapper/tags/list) as part of [extension install](https://mcr.microsoft.com/v2/arcdata/arcdataservices-extension/tags/list) | [`az k8s-extension create`](/cli/azure/k8s-extension?view=azure-cli-latest&preserve-view=true#az-k8s-extension-create) in the launcher |
379
-
>| 2 |**`patch.json`**|`value.imageTag`| Sourcing the [Data Controller image](https://mcr.microsoft.com/v2/arcdata/arc-controller/tags/list) | [`az arcdata dc create`](/cli/azure/arcdata/dc?view=azure-cli-latest&preserve-view=true#az-arcdata-dc-create) in the launcher |
380
-
>| 3 |**`kustomization.yaml`**|`images.newTag`| Sourcing the [launcher's image](https://mcr.microsoft.com/v2/arcdata/arc-ci-launcher/tags/list) | `kubectl apply`ing the launcher |
367
+
To recap, at this point - there are **3** places we specified `imageTag`s, for clarity, here's an explanation of the different uses of each. Typically - when testing a given release, all 3 values would be the same (aligning to a given release):
| 1 |**`.test.env`**|`DOCKER_TAG`| Sourcing the [Bootstrapper image](https://mcr.microsoft.com/v2/arcdata/arc-bootstrapper/tags/list) as part of [extension install](https://mcr.microsoft.com/v2/arcdata/arcdataservices-extension/tags/list) | [`az k8s-extension create`](/cli/azure/k8s-extension?view=azure-cli-latest&preserve-view=true#az-k8s-extension-create) in the launcher |
372
+
| 2 |**`patch.json`**|`value.imageTag`| Sourcing the [Data Controller image](https://mcr.microsoft.com/v2/arcdata/arc-controller/tags/list) | [`az arcdata dc create`](/cli/azure/arcdata/dc?view=azure-cli-latest&preserve-view=true#az-arcdata-dc-create) in the launcher |
373
+
| 3 |**`kustomization.yaml`**|`images.newTag`| Sourcing the [launcher's image](https://mcr.microsoft.com/v2/arcdata/arc-ci-launcher/tags/list) | `kubectl apply`ing the launcher |
382
374
383
375
### `kubectl apply`
384
376
@@ -409,7 +401,7 @@ Although it's best to deploy the launcher in a cluster with no pre-existing Arc
409
401
410
402
411
403
412
-
This same metadata-discovery and cleanup process is also run upon launcher exit, to leave the cluster in its pre-existing state before the launch.
404
+
This same metadata-discovery and cleanup process is also run upon launcher exit, to leave the cluster as close as possible to it's pre-existing state before the launch.
413
405
414
406
## Steps performed by launcher
415
407
@@ -422,12 +414,12 @@ At a high-level, the launcher performs the following sequence of steps:
422
414
5. Generate a unique set of environment variables based on timestamp for Arc Cluster name, Data Controller and Custom Location/Namespace. Prints out the environment variables, obfuscating sensitive values (e.g. Service Principal Password etc.)
423
415
6. a. For Direct Mode - Onboard the Cluster to Azure Arc, then deploys the Controller via the [unified experience](/create-data-controller-direct-cli?tabs=linux#deploy---unified-experience)
424
416
b. For Indirect Mode: deploy the Data Controller
425
-
7. Once Data Controller is `Ready`, generate a set of Azure CLI ([`az arcdata dc debug`](/cli/azure/arcdata/dc/debug?view=azure-cli-latest&preserve-view=true)) logs and stores locally, labeled as `setup-complete` - as a baseline.
426
-
8. Use the `TESTS_DIRECT/INDIRECT` environment variable from `.test.env` to launch a set of parallelized Sonobuoy test runs based on a space-separated array. These runs execute in a new `sonobuoy` namespace, using `arc-sb-plugin` pod that contains the integration tests.
427
-
9. [Sonobuoy aggregator](https://sonobuoy.io/docs/v0.56.0/plugins/) accumulate the [`junit`test results](https://sonobuoy.io/docs/v0.56.0/results/) and logs per `arc-sb-plugin`test run, which are exported into the launcher
417
+
7. Once Data Controller is `Ready`, generate a set of Azure CLI ([`az arcdata dc debug`](/cli/azure/arcdata/dc/debug?view=azure-cli-latest&preserve-view=true)) logs and store locally, labeled as `setup-complete` - as a baseline.
418
+
8. Use the `TESTS_DIRECT/INDIRECT` environment variable from `.test.env` to launch a set of parallelized Sonobuoy test runs based on a space-separated array (`TESTS_(IN)DIRECT`). These runs execute in a new `sonobuoy` namespace, using `arc-sb-plugin` pod that contains the Pytest validation tests.
419
+
9. [Sonobuoy aggregator](https://sonobuoy.io/docs/v0.56.0/plugins/) accumulate the [`junit` test results](https://sonobuoy.io/docs/v0.56.0/results/) and logs per `arc-sb-plugin` test run, which are exported into the launcher pod.
428
420
10. Return the exit code of the tests, and generates another set of debug logs - Azure CLI and `sonobuoy` - stored locally, labeled as `test-complete`.
429
-
11. Perform a CRD metadata scan, similar to Step 3, to discover existing Arc and Arc Data Services Custom Resources. It then proceeds to destroy all Arc and Arc Data resources in reverse order from deployment, as well as CRDs, Role/ClusterRoles, PV/PVCs etc.
430
-
12. Attempt to use the SAS token `LOGS_STORAGE_ACCOUNT_SAS` provided to create a new Storage Account container named based on `LOGS_STORAGE_CONTAINER`, in the **pre-existing** Storage Account `LOGS_STORAGE_ACCOUNT`. It uploads all localtest results and logs to this storage account as a tarball (see below).
421
+
11. Perform a CRD metadata scan, similar to Step 3, to discover existing Arc and Arc Data Services Custom Resources. Then, proceed to destroy all Arc and Arc Data resources in reverse order from deployment, as well as CRDs, Role/ClusterRoles, PV/PVCs etc.
422
+
12. Attempt to use the SAS token `LOGS_STORAGE_ACCOUNT_SAS` provided to create a new Storage Account container named based on `LOGS_STORAGE_CONTAINER`, in the **pre-existing** Storage Account `LOGS_STORAGE_ACCOUNT`. If Storage Account container already exists, use it. Upload all local test results and logs to this storage container as a tarball (see below).
![prmerger-automator[bot]](https://avatars.githubusercontent.com/u/22479449?v=4&size=40){kind=avatar}
0 commit comments