You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
This article introduces Azure Operator Service Manager (AOSM) safe upgrade practices (SUP). This feature set enables an end user to safely execute complex upgrades of container network function (CNF) workloads hosted on Azure Operator Nexus, in compliance with partner In Service Software Upgrade (ISSU) requirements, where applicable. Look for future articles to expand on advanced SUP features and capabilities.
12
+
This article introduces Azure Operator Service Manager (AOSM) safe upgrade practices (SUP). This feature set enables the safe execution of complex container network function (CNF) hosted on Azure Operator Nexus. These upgrades are structured in general compliance with partner In Service Software Upgrade (ISSU) requirements. Look for future articles to expand on advanced SUP features and capabilities.
13
13
14
14
## Introduction to safe upgrades
15
-
A given network service supported by AOSM is composed of one to many CNFs which, over time, require software upgrades. For each upgrade, it is necessary to run one to many helm operations, updating dependent network function applications (nfApps), in a particular order, in a manner which least impacts the network service. AOSM SUP represents a set of features, which enables safe automation of these operations on Azure Operator Nexus.
15
+
A given network service supported by AOSM is composed of one to many CNFs which, over time, require software upgrades. For each upgrade, it's necessary to run one to many helm operations, updating dependent network function applications (nfApps), in a particular order, in a manner which least impacts the network service. AOSM SUP represents a set of features, which enables safe automation of these operations on Azure Operator Nexus.
16
16
17
17
* SNS Reput Support - Execute helm upgrade operation across all nfApps in network function design version (NFDV).
18
18
* Nexus Platform - Support SNS reput operations on Nexus platform targets.
19
19
* Operation Time-outs - Ability to set operational time-outs for each nfApp operation.
20
20
* Synchronous Operations - Ability to run one serial nfApp operation at a time.
21
21
* Control Upgrade Order - Define different nfApp sequence for install and upgrade.
22
22
* Pause On Failure - Default behavior pauses after an nfApp operation failure.
23
-
* Rollback On Failure - Optional behavior, rollsback all completed nfApps prior to operation failure.
23
+
* Rollback On Failure - Optional behavior, rollback completed nfApps before failed nfApp.
24
24
* Single Chart Test Validation - Running a helm test operation after a create or update.
25
25
* Skip nfApp on No Change - Skip processing of nfApps where no changes result.
26
26
* Image Preloading - Ability to preload images to edge repository.
@@ -32,7 +32,7 @@ For each nfApp, the reput update request supports increasing a helm chart versio
32
32
33
33
* nfApps are processed following either updateDependsOn ordering, or in the sequential order they appear.
34
34
* nfApps with parameter `applicationEnabled` set to disable are skipped.
35
-
* nfApps with parameter `skipUpgrade` set to enabled are skipped if no changes detected.
35
+
* nfApps with parameter `skipUpgrade` set to `enabled` are skipped if no changes detected.
36
36
* nfApps which are common between old and new NFDV are upgraded.
37
37
* nfApps which are only in the new NFDV are installed.
38
38
* nfApps deployed, but not referenced by the new NFDV, are deleted.
@@ -41,24 +41,24 @@ To ensure outcomes, nfApp testing is supported using helm, either helm upgrade p
41
41
42
42
## Considerations for in-service upgrades
43
43
Azure Operator Service Manager generally supports in service upgrades, an upgrade method which advances a deployment version without interrupting the running service. Some considerations are necessary to ensure the proper behavior of AOSM during ISSU operations.
44
-
* Where AOSM performs an upgrade against an ordered set of multiple nfApps, AOSM first upgrades or creates all new nfApps, then deletes all old nfApps. This approach ensures service is not impacted until all new nfApps are ready but requires extra platform capacity for transient hosting of both old and new nfApps.
45
-
* Where AOSM upgrades an nfApp with multiple replica, AOSM honors the deployment profile settings for rolling or recreate option. Where rolling is used, expose the values `maxUnavailable` and `maxSurge` as CGS parameters, which can then be set via operator CGV at run-time.
44
+
* Where AOSM performs an upgrade against an ordered set of multiple nfApps, AOSM first upgrades or creates all new nfApps, then deletes all old nfApps. This approach ensures service isn't impacted until all new nfApps are ready but requires extra platform capacity for transient hosting of both old and new nfApps.
45
+
* Where AOSM upgrades an nfApp with multiple replica, AOSM honors the deployment profile settings for either the rolling or recreate options. Where rolling is used, expose the values `maxUnavailable` and `maxSurge` as CGS parameters, which can then be set via operator CGV at run-time.
46
46
47
-
Ultimately, the ability for a given service to be upgraded without interruption is a feature of the service itself. Consult further with the service publisher to understand the in-service upgrade capabilities and ensure they are aligned with the proper AOSM behavioral options.
47
+
Ultimately, the ability for a given service to be upgraded without interruption is a feature of the service itself. Consult further with the service publisher to understand the in-service upgrade capabilities and ensure they're aligned with the proper AOSM behavioral options.
48
48
49
49
## Safe upgrade prerequisites
50
50
When planning for an upgrade using Azure Operator Service Manager, address the following requirements in advance of upgrade execution to optimize the time spent attempting the upgrade.
51
51
52
52
- Onboard updated artifacts using publisher and/or designer workflows.
53
53
- In most cases, use the existing publisher to host new version artifacts.
54
54
- Using an existing publisher supports `helm upgrade` to update an SNS to a different version.
55
-
- Using a new publisher requies a `helm delete` on the current SNS and then a `helm install` for the new SNS version.
56
-
- Artifact store, network service design group (NSDG), and network function design group (NFDG) are immutabe and cannot change.
55
+
- Using a new publisher requires a `helm delete` on the current SNS and then a `helm install` for the new SNS version.
56
+
- Artifact store, network service design group (NSDG), and network function design group (NFDG) are immutable and cannot change.
57
57
- Changing one of these resources requires deployment of a new SNS.
58
58
- A new artifact manifest is needed to store the new charts and images.
59
59
- See [onboarding documentation](how-to-manage-artifacts-nexus.md) for details on uploading new charts and images.
60
60
- A new NFDV, and optionally network service design version (NSDV), is needed.
61
-
- NFDV changes can be complex, we cover only basic changes in this article.
61
+
- NFDV changes can be complex. We cover only basic changes in this article.
62
62
- New NSDV is only required if a new configuration group schema (CGS) version is being introduced.
63
63
- If necessary, new CGS.
64
64
- Required if an upgrade introduces new exposed configuration parameters.
@@ -77,13 +77,13 @@ When planning for an upgrade using Azure Operator Service Manager, address the f
77
77
Follow the following process to trigger an upgrade with Azure Operator Service Manager.
78
78
79
79
### Create new NFDV resource
80
-
For new NFDV versions, it must be in a valid SemVer format. The new version can be an upgrade, a greater value versus the deployed version, or an downgrade, a lower value versus the deployed version. The new version can differ by major, minor or patch values.
80
+
For new NFDV versions, it must be in a valid SemVer format. The new version can be an upgrade, a greater value versus the deployed version, or a downgrade, a lower value versus the deployed version. The new version can differ by major, minor, or patch values.
81
81
82
82
### Update new NFDV parameters
83
-
Helm chart versions can be updated, or Helm values can be updated or parameterized as necessary. New nfApps can also be added where they did not exist in deployed version.
83
+
Helm chart versions can be updated, or Helm values can be updated or parameterized as necessary. New nfApps can also be added where they didn't exist in deployed version.
84
84
85
85
### Update NFDV for desired nfApp order
86
-
UpdateDependsOn is an NFDV parameter used to specify ordering of nfApps during update operations. If `updateDependsOn`is not provided, serial ordering of CNF applications, as appearing in the NFDV is used.
86
+
UpdateDependsOn is an NFDV parameter used to specify ordering of nfApps during update operations. If `updateDependsOn`isn't provided, serial ordering of CNF applications, as appearing in the NFDV is used.
87
87
88
88
### Update ARM template for desired upgrade behavior
89
89
Make sure to set any desired CNF application `timeout`, the `atomic` parameter, and `rollbackOnTestFailure` parameter. It may be useful to change these parameters over time as more confidence is gained in the upgrade.
@@ -101,16 +101,16 @@ In cases where a reput update fails, the following process can be followed to re
101
101
Resolve the root cause for nfApp failure by analyzing logs and other debugging information.
102
102
103
103
### Manually skip completed charts
104
-
After fixing the failed nfApp, but before attempting an upgrade retry, consider changing the `applicationEnablement` parameter to accelerate retry behavior. This parameter can be set false, where an nfApp should be skipped. This parameter can be useful where an nfApp does not require an upgraded.
104
+
After fixing the failed nfApp, but before attempting an upgrade retry, consider changing the `applicationEnablement` parameter to accelerate retry behavior. This parameter can be set false, where an nfApp should be skipped. This parameter can be useful where an nfApp doesn't require an upgraded.
105
105
106
106
### Issue SNS reput retry (repeat until success)
107
-
By default, the reput retries nfApps in the declared update order, unless they are skipped using `applicationEnablement` flag.
107
+
By default, the reput retries nfApps in the declared update order, unless they're skipped using `applicationEnablement` flag.
108
108
109
109
## Skip nfApps using applicationEnablement
110
-
In the NFDV resource, under `deployParametersMappingRuleProfile` there is the property `applicationEnablement` of type enum, which takes values: Unknown, Enabled, or disabled. It can be used to exclude nfApp operations during network function (NF) deployment.
110
+
In the NFDV resource, under `deployParametersMappingRuleProfile` there's the property `applicationEnablement` of type enum, which takes values: Unknown, Enabled, or disabled. It can be used to exclude nfApp operations during network function (NF) deployment.
111
111
112
112
### Publisher changes
113
-
For the `applicationEnablement` property, the publisher has two options: either provide a default value or parameterize it. In the following example, we have provided a default value for `hellotest` which can later be used in an override.
113
+
For the `applicationEnablement` property, the publisher has two options: either provide a default value or parameterize it. The following example sets `enabled` as the default value for `hellotest` which can later be changed via override.
114
114
115
115
```json
116
116
{
@@ -150,11 +150,11 @@ The `skipUpgrade` feature is designed to optimize the time taken for CNF upgrade
150
150
### Precheck Criteria
151
151
An upgrade can be skipped if all the following conditions are met:
152
152
1. The `nfApplication` provisioning state is Succeeded.
153
-
2.There is no change in the Helm chart name or version.
154
-
3.There is no change in the Helm values.
153
+
2.there's no change in the Helm chart name or version.
154
+
3.there's no change in the Helm values.
155
155
156
156
### Enabling or disabling the skipUpgrade feature
157
-
The `skipUpgrade` feature is **disabled by default**. If this optional parameter is not specified in `roleOverrideValues` under `upgradeOptions`, CNF upgrades proceed in the traditional manner, where the `nfApplications` are upgraded at the cluster level.
157
+
The `skipUpgrade` feature is **disabled by default**. If this optional parameter isn't specified in `roleOverrideValues` under `upgradeOptions`, CNF upgrades proceed in the traditional manner, where the `nfApplications` are upgraded at the cluster level.
158
158
159
159
#### Enabling SkipUpgrade within Network Function Resource
160
160
To enable the SkipUpgrade feature via `roleOverrideValues`, refer to the following example.
@@ -182,4 +182,4 @@ To enable the SkipUpgrade feature via `roleOverrideValues`, refer to the followi
182
182
-**nfApplication: `hellotest`**
183
183
- The `skipUpgrade` flag is enabled. If the upgrade request for `hellotest` meets the precheck criteria, the upgrade is skipped.
184
184
-**nfApplication: `runnerTest`**
185
-
- The `skipUpgrade` flag is not specified. Therefore, `runnerTest` executes a traditional Helm upgrade at the cluster level, even if the precheck criteria are met.
185
+
- The `skipUpgrade` flag isn't specified. Therefore, `runnerTest` executes a traditional Helm upgrade at the cluster level, even if the precheck criteria are met.
0 commit comments