|
| 1 | +--- |
| 2 | +title: Run tests after install or upgrade operations with Azure Operator Service Manager |
| 3 | +description: Learn about using helm test as part of a network function install or upgrade. |
| 4 | +author: msftadam |
| 5 | +ms.author: adamdor |
| 6 | +ms.date: 02/19/2024 |
| 7 | +ms.topic: upgrade-and-migration-article |
| 8 | +ms.service: azure-operator-service-manager |
| 9 | +--- |
| 10 | + |
| 11 | +# Run tests after install or upgrade |
| 12 | +This article describes how Azure Operator Service Manager (AOSM) can run tests on network functions (NFs) as part of install or upgrade operations. When enabled, each network function application (nfApp) is tested after component install or upgrade completion. A successful result across all nfApp tests is required for the NF operation status to complete successfully. |
| 13 | + |
| 14 | +## Overview |
| 15 | +As part of the safe upgrade program, AOSM supports the use of helm tests as a gating step during network function install or upgrade operations. The following workflow describes the logic used by AOSM when including this extra layer of validation: |
| 16 | +* Users author their own tests and include them within the helm package during NF onboarding. |
| 17 | +* Only where enabled, AOSM executes these helm tests on each nfApp. |
| 18 | +* Upon test success, AOSM proceeds to the next nfApp. |
| 19 | +* Upon test failure, AOSM honors `rollbackOnTestFail` to determine if the nfApp is rolled back. |
| 20 | +* The parent NF operation terminates with failure if any nfApp fails a configured test. |
| 21 | +* Upon parent failure, AOSM honors the configured method of NF failure control, either `pause-on-failure` or `rollback-on-failure`. |
| 22 | + |
| 23 | +> [!NOTE] |
| 24 | +> Helm test is only supported as part of the install or upgrade operation and can't be run standalone. |
| 25 | +
|
| 26 | +## Authoring helm tests |
| 27 | +The publisher is responsible for authoring the helm tests during construction of the helm charts. The helm tests are defined in the helm chart under the folder: `<ChartName>/Templates/`. Each test includes a job definition that specifies a container environment and command to run. The container environment should exit successfully for a test to be considered a success. The job definition must include the helm test hook annotation `(helm.sh/hook: test)` to be recognized as a test by helm. |
| 28 | + |
| 29 | +## Enable helm tests during operations |
| 30 | +AOSM provides a set of configurable install and upgrade options for each nfApp. These existing options are extended with a new `testOptions` parameter. With this parameter, the user can specify `testOptions` settings per nfApp and per type of operation. The `testOptions` parameter supports the following parameters: |
| 31 | + |
| 32 | +* `enable` |
| 33 | + * Enables or disables the helm test on a nfApp after install or upgrade completes. |
| 34 | + * Default value is false. |
| 35 | +* `timeout` |
| 36 | + * Takes a value that represents the test time-out in minutes. |
| 37 | + * Default value is 20 minutes. |
| 38 | +* `rollbackOnTestFailure` |
| 39 | + * Enables or disables rollback on nfApp helm test failure. |
| 40 | + * Default value is true. |
| 41 | +* `filter` |
| 42 | + * Allows for a method to run only a subset of tests. Accepts a list of strings, where each string in the list represents a test to execute. |
| 43 | + * Default value is no filter and all tests are run. |
| 44 | + |
| 45 | +## Exposing helm test control via parameters |
| 46 | +AOSM already supports the NF payload parameters `installOptions` and `upgradeOptions` for each nfApp under `roleOverrideValues`. These parameters are extended to include new `testOptions` settings. Exposing these new parameter settings allows the Operator to control the upgrade behavior at run-time. See the three following examples demonstrating usage of `testOptions`. |
| 47 | + |
| 48 | +## roleOverrideValues escaped example |
| 49 | +Following is an escaped example `roleOverrideValues` with `testOptions` under `installOptions` and `upgradeOptions` for a component named `application1`. This example uses a `filter`, to execute only tests which match the string provided, uses a custom time-out, and enables `rollbackOnTestFailures`. |
| 50 | + |
| 51 | +``` |
| 52 | +"roleOverrideValues": [ |
| 53 | +"{\"name\":\"application1\",\"deployParametersMappingRuleProfile\":{\"helmMappingRuleProfile\":{\"helmPackageVersion\":\"2.1.3\",\"values\":\"{\\\"roleOneParam\\\":\\\"roleOneOverrideValue\\\"}\",\"options\":{\"installOptions\":{\"atomic\":\”true\”,\"wait\":\"true\",\"timeout\":\"30m\",\”testOptions\”:{\”enable\”:\”true\”,\”timeout\”:\”15\”,\”rollbackOnTestFailure\”:\”true\”,\”filter\”:[\”test1\”,\”test2\”,\”test3\”]}},\"upgradeOptions\":{\"atomic\": \”true\”,\"wait\":\"true\",\"timeout\":\"30\", \”testOptions\”:{\”enable ”:\”true\”,\”timeout\”:\”15\”,\”rollbackOnTestFailure\”:\”true\”,\” filter \”:[\”test1\”,\”test2\”,\”test3\”]}}}}}}"] |
| 54 | +``` |
| 55 | + |
| 56 | +## roleOverrideValues unescaped example |
| 57 | +Following is an unescaped example `roleOverrideValues` NF Payload with `testOptions` under `installOptions` and `upgradeOptions` for a component named `hellotest`. This example uses a `filter` to execute only tests which match the string provided, uses a custom time-out, and enables `rollbackOnTestFailures`. |
| 58 | + |
| 59 | +``` |
| 60 | +"roleOverrideValues": ["{ |
| 61 | + "name":"hellotest", |
| 62 | + "deployParametersMappingRuleProfile": { |
| 63 | + "helmMappingRuleProfile": { |
| 64 | + "options": { |
| 65 | + "installOptions": { |
| 66 | + "atomic":"true", |
| 67 | + "wait":"true", |
| 68 | + "timeout":"1" |
| 69 | + “testOptions”: { |
| 70 | + “enable”: “true”, |
| 71 | + “timeout”: “10”, |
| 72 | + “rollbackOnTestFailure”: “true”, |
| 73 | + "filter”: [“test1”, “test2”] } }, |
| 74 | + "upgradeOptions": { |
| 75 | + "atomic":"true", |
| 76 | + "wait":"true", |
| 77 | + "timeout":"2", |
| 78 | + “testOptions”: { |
| 79 | + “enable”: “true”, |
| 80 | + “timeout”: “10”, |
| 81 | + “rollbackOnTestFailure”: “true”, |
| 82 | + "filter”: [“test1”, “test2”] } } } } } }" |
| 83 | +] |
| 84 | +``` |
| 85 | + |
| 86 | +## NF payload example |
| 87 | +Following is an example NF Payload with `testOptions` under `installOptions` and `upgradeOptions` for a component named `application1`. This example uses a `filter` to execute only tests which match the string provided, uses a custom time-out and enables `rollbackOnTestFailures`. |
| 88 | + |
| 89 | +``` |
| 90 | +{ |
| 91 | + "location": "eastus", |
| 92 | + "properties": { |
| 93 | + "publisherName": "testVendor", |
| 94 | + "publisherScope": "Public", |
| 95 | + "networkFunctionDefinitionGroupName": "testnetworkFunctionDefinitionGroupName", |
| 96 | + "networkFunctionDefinitionVersion": "1.0.1", |
| 97 | + "networkFunctionDefinitionOfferingLocation": "eastus", |
| 98 | + "nfviType": "AzureArcKubernetes", |
| 99 | + "nfviId": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/testResourceGroup/providers/Microsoft.ExtendedLocation/customLocations/testCustomLocation", |
| 100 | + "allowSoftwareUpdate": true, |
| 101 | + "deploymentValues": "{\"releaseName\":\"testReleaseName\",\"namespace\":\"testNamespace\",\"wait\":\"false\"}", |
| 102 | + "roleOverrideValues": [ |
| 103 | + "{\"name\":\"application1\",\"deployParametersMappingRuleProfile\":{\"helmMappingRuleProfile\":{\"helmPackageVersion\":\"2.1.3\",\"values\":\"{\\\"roleOneParam\\\":\\\"roleOneOverrideValue\\\"}\",\"options\":{\"installOptions\":{\"atomic\":\”true\”,\"wait\":\"true\",\"timeout\":\"30m\",\” testOptions \”:{\” enable \”:\”true\”,\” timeout\”:\”15\”,\”rollbackOnTestFailure\”:\”true\”,\” filter \”:[\”test1\”,\”test2\”,\”test3\”]}},\"upgradeOptions\":{\"atomic\": \”true\”,\"wait\":\"true\",\"timeout\":\"30\", \” testOptions \”:{\” enable \”:\”true\”,\” timeout\”:\”15\”,\”rollbackOnTestFailure\”:\”true\”,\” filter \”:[\”test1\”,\”test2\”,\”test3\”]}}}}}}" |
| 104 | + ] |
| 105 | + } |
| 106 | +} |
| 107 | +``` |
0 commit comments