edit pass: how-to-manage-kubernetes-instance-types

ShawnJackson · ShawnJackson · commit e8112ff7a716 · 2023-08-08T15:39:03.000-05:00
diff --git a/articles/machine-learning/how-to-manage-kubernetes-instance-types.md b/articles/machine-learning/how-to-manage-kubernetes-instance-types.md
@@ -14,22 +14,22 @@ ms.custom: build-spring-2022, cliv2, sdkv2, event-tier1-build-2022
 
 # Create and manage instance types for efficient utilization of compute resources
 
-Instance types are an Azure Machine Learning concept that allows targeting certain types of compute nodes for training and inference workloads. For an Azure VM, an example for an instance type is `STANDARD_D2_V3`.
+Instance types are an Azure Machine Learning concept that allows targeting certain types of compute nodes for training and inference workloads. For an Azure virtual machine (VM), an example of an instance type is `STANDARD_D2_V3`.
 
-In Kubernetes clusters, instance types are represented in a custom resource definition (CRD) that is installed with the Azure Machine Learning extension. Two elements in Azure Machine Learning extension represent the instance types:
+In Kubernetes clusters, instance types are represented in a custom resource definition (CRD) that's installed with the Azure Machine Learning extension. Two elements in the Azure Machine Learning extension represent the instance types:
 
 - Use [nodeSelector](https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/#nodeselector) to specify which node a pod should run on. The node must have a corresponding label.
-- In the [resources](https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/) section, you can set the compute resources (CPU, memory and NVIDIA GPU) for the pod.
+- In the [resources](https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/) section, you can set the compute resources (CPU, memory, and NVIDIA GPU) for the pod.
 
-If you [specify a nodeSelector when deploying the Azure Machine Learning extension](./how-to-deploy-kubernetes-extension.md#review-azure-machine-learning-extension-configuration-settings), the nodeSelector will be applied to all instance types. This means that:
+If you [specify a nodeSelector field when deploying the Azure Machine Learning extension](./how-to-deploy-kubernetes-extension.md#review-azure-machine-learning-extension-configuration-settings), the `nodeSelector` field will be applied to all instance types. This means that:
 
-- For each instance type creating, the specified nodeSelector should be a subset of the extension-specified nodeSelector.
-- If you use an instance type **with nodeSelector**, the workload will run on any node matching both the extension-specified nodeSelector and the instance type-specified nodeSelector.
-- If you use an instance type **without a nodeSelector**, the workload will run on any node matching the extension-specified nodeSelector.
+- For each instance type that you create, the specified `nodeSelector` field should be a subset of the extension-specified `nodeSelector` field.
+- If you use an instance type with `nodeSelector`, the workload will run on any node that matches both the extension-specified `nodeSelector` field and the instance type-specified `nodeSelector` field.
+- If you use an instance type without a `nodeSelector` field, the workload will run on any node that matches the extension-specified `nodeSelector` field.
 
 ## Create a default instance type
 
-By default, a `defaultinstancetype` with the following definition is created when you attach a Kubernetes cluster to an Azure Machine Learning workspace.
+By default, an instance type called `defaultinstancetype` is created when you attach a Kubernetes cluster to an Azure Machine Learning workspace. Here's the definition:
 
 ```yaml
 resources:
@@ -42,12 +42,14 @@ resources:
     nvidia.com/gpu: null
 ```
 
-If you don't apply a `nodeSelector`, it means the pod can get scheduled on any node. The workload's pods are assigned default resources with 0.1 cpu cores, 2-GB memory and 0 GPU for request. The resources used by the workload's pods are limited to 2 cpu cores and 8-GB memory:
+If you don't apply a `nodeSelector` field, the pod can be scheduled on any node. The workload's pods are assigned default resources with 0.1 CPU cores, 2 GB of memory, and 0 GPUs for the request. The resources that the workload's pods use are limited to 2 CPU cores and 8 GB of memory.
 
-The default instance type purposefully uses little resources. To ensure all ML workloads run with appropriate resources, for example GPU resource, it is highly recommended to create custom instance types.
+The default instance type purposefully uses few resources. To ensure that all machine learning workloads run with appropriate resources (for example, GPU resource), we highly recommend that you create custom instance types.
 
-- `defaultinstancetype` will not appear as an InstanceType custom resource in the cluster when running the command ```kubectl get instancetype```, but it will appear in all clients (UI, CLI, SDK).
-- `defaultinstancetype` can be overridden with a custom instance type definition having the same name as `defaultinstancetype` (see [Create custom instance types](#create-a-custom-instance-type) section)
+Keep in mind the following points about the default instance type:
+
+- `defaultinstancetype` doesn't appear as an `InstanceType` custom resource in the cluster when you're running the command ```kubectl get instancetype```, but it does appear in all clients (UI, Azure CLI, SDK).
+- `defaultinstancetype` can be overridden with a [custom instance type](#create-a-custom-instance-type) definition that has the same name.
 
 ## Create a custom instance type
 
@@ -57,7 +59,7 @@ To create a new instance type, create a new custom resource for the instance typ
 kubectl apply -f my_instance_type.yaml
 ```
 
-With `my_instance_type.yaml`:
+Here are the contents of *my_instance_type.yaml*:
 
 ```yaml
 apiVersion: amlarc.azureml.com/v1alpha1
@@ -77,31 +79,31 @@ spec:
       memory: "1500Mi"
 ```
 
-The following steps create an instance type with the labeled behavior:
+The preceding code creates an instance type with the labeled behavior:
 
-- Pods are scheduled only on nodes with label `mylabel: mylabelvalue`.
-- Pods are assigned resource requests of `700m` CPU and `1500Mi` memory.
-- Pods are assigned resource limits of `1` CPU, `2Gi` memory and `1` NVIDIA GPU.
+- Pods are scheduled only on nodes that have the label `mylabel: mylabelvalue`.
+- Pods are assigned resource requests of `700m` for CPU and `1500Mi` for memory.
+- Pods are assigned resource limits of `1` for CPU, `2Gi` for memory, and `1` for NVIDIA GPU.
 
-Creation of custom instance types must meet the following parameters and definition rules, otherwise the instance type creation fails:
+Creation of custom instance types must meet the following parameters and definition rules, or it will fail:
 
-| Parameter | Required | Description |
+| Parameter | Required or optional | Description |
 | --- | --- | --- |
-| name | required | String values, which must be unique in cluster.|
-| CPU request | required | String values, which cannot be 0 or empty. <br>You can specify the CPU in millicores; for example, `100m`. You can also specify it as full numbers; for example, `"1"` is equivalent to `1000m`.|
-| Memory request | required | String values, which cannot be 0 or empty. <br>You can specify the memory as a full number + suffix; for example, `1024Mi` for 1024 MiB.|
-| CPU limit | required | String values, which cannot be 0 or empty. <br>You can specify the CPU in millicores; for example, `100m`. You can also specify it as full numbers; for example, `"1"` is equivalent to `1000m`.|
-| Memory limit | required | String values, which cannot be 0 or empty. <br>You can specify the memory as a full number + suffix; for example, `1024Mi` for 1024 MiB.|
-| GPU | optional | Integer values, which can only be specified in the `limits` section. <br>For more information, see the Kubernetes [documentation](https://kubernetes.io/docs/tasks/manage-gpus/scheduling-gpus/#using-device-plugins). |
-| nodeSelector | optional | Map of string keys and values. |
+| `name` | Required | String values, which must be unique in a cluster.|
+| `CPU request` | Required | String values, which can't be zero or empty. <br>You can specify the CPU in millicores; for example, `100m`. You can also specify it as full numbers. For example, `"1"` is equivalent to `1000m`.|
+| `Memory request` | Required | String values, which can't be zero or empty. <br>You can specify the memory as a full number + suffix; for example, `1024Mi` for 1,024 mebibytes (MiB).|
+| `CPU limit` | Required | String values, which can't be zero or empty. <br>You can specify the CPU in millicores; for example, `100m`. You can also specify it as full numbers. For example, `"1"` is equivalent to `1000m`.|
+| `Memory limit` | Required | String values, which can't be zero or empty. <br>You can specify the memory as a full number + suffix; for example, `1024Mi` for 1024 MiB.|
+| `GPU` | Optional | Integer values, which can only be specified in the `limits` section. <br>For more information, see the [Kubernetes documentation](https://kubernetes.io/docs/tasks/manage-gpus/scheduling-gpus/#using-device-plugins). |
+| `nodeSelector` | Optional | Map of string keys and values. |
 
 It's also possible to create multiple instance types at once:
 
 ```bash
 kubectl apply -f my_instance_type_list.yaml
 ```
 
-With `my_instance_type_list.yaml`:
+Here are the contents of *my_instance_type_list.yaml*:
 
 ```yaml
 apiVersion: amlarc.azureml.com/v1alpha1
@@ -132,16 +134,16 @@ items:
           memory: "1Gi"
 ```
 
-The above example creates two instance types: `cpusmall` and `defaultinstancetype`. This `defaultinstancetype` definition overrides the `defaultinstancetype` definition created when Kubernetes cluster was attached to Azure Machine Learning workspace.
+The preceding example creates two instance types: `cpusmall` and `defaultinstancetype`. This `defaultinstancetype` definition overrides the `defaultinstancetype` definition that was created when you attached the Kubernetes cluster to the Azure Machine Learning workspace.
 
-If you submit a training or inference workload without an instance type, it uses the `defaultinstancetype`. To specify a default instance type for a Kubernetes cluster, create an instance type with name `defaultinstancetype`. It's automatically recognized as the default.
+If you submit a training or inference workload without an instance type, it uses `defaultinstancetype`. To specify a default instance type for a Kubernetes cluster, create an instance type with the name `defaultinstancetype`. It's automatically recognized as the default.
 
 ## Select an instance type to submit a training job
 
 ### [Azure CLI](#tab/select-instancetype-to-trainingjob-with-cli)
 
-To select an instance type for a training job using CLI (V2), specify its name as part of the
-`resources` properties section in job YAML. For example:
+To select an instance type for a training job by using the Azure CLI (V2), specify its name as part of the
+`resources` properties section in the job YAML. For example:
 
 ```yaml
 command: python -c "print('Hello world!')"
@@ -154,7 +156,7 @@ resources:
 
 ### [Python SDK](#tab/select-instancetype-to-trainingjob-with-sdk)
 
-To select an instance type for a training job using SDK (V2), specify its name for `instance_type` property in `command` class. For example:
+To select an instance type for a training job by using the SDK (V2), specify its name for the `instance_type` property in the `command` class. For example:
 
 ```python
 from azure.ai.ml import command
@@ -170,13 +172,13 @@ command_job = command(
 
 ---
 
-In the above example, replace `<Kubernetes-compute_target_name>` with the name of your Kubernetes compute target and replace `<instance_type_name>` with the name of the instance type you wish to select. If there's no `instance_type` property specified, the system uses `defaultinstancetype` to submit the job.
+In the preceding example, replace `<Kubernetes-compute_target_name>` with the name of your Kubernetes compute target. Replace `<instance_type_name>` with the name of the instance type that you want to select. If you don't specify an `instance_type` property, the system uses `defaultinstancetype` to submit the job.
 
 ## Select an instance type to deploy a model
 
 ### [Azure CLI](#tab/select-instancetype-to-modeldeployment-with-cli)
 
-To select an instance type for a model deployment using CLI (V2), specify its name for the `instance_type` property in the deployment YAML. For example:
+To select an instance type for a model deployment by using the Azure CLI (V2), specify its name for the `instance_type` property in the deployment YAML. For example:
 
 ```yaml
 name: blue
@@ -195,7 +197,7 @@ environment:
 
 ### [Python SDK](#tab/select-instancetype-to-modeldeployment-with-sdk)
 
-To select an instance type for a model deployment using SDK (V2), specify its name for the `instance_type` property in the `KubernetesOnlineDeployment` class. For example:
+To select an instance type for a model deployment by using the SDK (V2), specify its name for the `instance_type` property in the `KubernetesOnlineDeployment` class. For example:
 
 ```python
 from azure.ai.ml import KubernetesOnlineDeployment,Model,Environment,CodeConfiguration
@@ -222,14 +224,14 @@ blue_deployment = KubernetesOnlineDeployment(
 
 ---
 
-In the above example, replace `<instance_type_name>` with the name of the instance type you wish to select. If there's no `instance_type` property specified, the system uses `defaultinstancetype` to deploy the model.
+In the preceding example, replace `<instance_type_name>` with the name of the instance type that you want to select. If you don't specify an `instance_type` property, the system uses `defaultinstancetype` to deploy the model.
 
 > [!IMPORTANT]
-> For MLFlow model deployment, the resource request require at least 2 CPU and 4 GB memory. Otherwise, the deployment will fail.
+> For MLflow model deployment, the resource request requires at least 2 CPU cores and 4 GB of memory. Otherwise, the deployment will fail.
 
 ### Resource section validation
 
-If you're using the `resource section` to define the resource request and limit of your model deployments, for example:
+You can use the `resources` section to define the resource request and limit of your model deployments. For example:
 
 #### [Azure CLI](#tab/define-resource-to-modeldeployment-with-cli)
 
@@ -295,26 +297,23 @@ blue_deployment = KubernetesOnlineDeployment(
 
 ---
 
-If you use the `resource section`, the valid resource definition need to meet the following rules, otherwise the model deployment fails due to the invalid resource definition:
+If you use the `resources` section, a valid resource definition needs to meet the following rules. An invalid resource definition will cause the model deployment to fail.
 
-| Parameter | If necessary | Description |
+| Parameter | Required or optional | Description |
 | --- | --- | --- |
-| `requests:`<br>`cpu:`| Required | String values, which can't be 0 or empty. <br>You can specify the CPU in millicores, for example `100m`, or in full numbers, for example `"1"` is equivalent to `1000m`.|
-| `requests:`<br>`memory:` | Required | String values, which can't be 0 or empty. <br>You can specify the memory as a full number + suffix, for example `1024Mi` for 1024 MiB. <br>Memory can't be less than **1 MBytes**.|
-| `limits:`<br>`cpu:` | Optional <br>(only required when need GPU) | String values, which can't be 0 or empty. <br>You can specify the CPU in millicores, for example `100m`, or in full numbers, for example `"1"` is equivalent to `1000m`. |
-| `limits:`<br>`memory:` | Optional <br>(only required when need GPU) | String values, which can't be 0 or empty. <br>You can specify the memory as a full number + suffix, for example `1024Mi` for 1024 MiB.|
-| `limits:`<br>`nvidia.com/gpu:` | Optional <br>(only required when need GPU) | Integer values, which can't be empty and can only be specified in the `limits` section. <br>For more information, see the Kubernetes [documentation](https://kubernetes.io/docs/tasks/manage-gpus/scheduling-gpus/#using-device-plugins). <br>If require CPU only, you can omit the entire `limits` section.|
-
-> [!NOTE]
-> If the resource section definition is invalid, the deployment will fail.
+| `requests:`<br>`cpu:`| Required | String values, which can't be zero or empty. <br>You can specify the CPU in millicores; for example, `100m`. You can also specify it in full numbers. For example, `"1"` is equivalent to `1000m`.|
+| `requests:`<br>`memory:` | Required | String values, which can't be zero or empty. <br>You can specify the memory as a full number + suffix; for example, `1024Mi` for 1024 MiB. <br>Memory can't be less than 1 MB.|
+| `limits:`<br>`cpu:` | Optional <br>(required only when you need GPU) | String values, which can't be zero or empty. <br>You can specify the CPU in millicores; for example `100m`. You can also specify it in full numbers. For example, `"1"` is equivalent to `1000m`. |
+| `limits:`<br>`memory:` | Optional <br>(required only when you need GPU) | String values, which can't be zero or empty. <br>You can specify the memory as a full number + suffix; for example, `1024Mi` for 1,024 MiB.|
+| `limits:`<br>`nvidia.com/gpu:` | Optional <br>(required only when you need GPU) | Integer values, which can't be empty and can be specified only in the `limits` section. <br>For more information, see the [Kubernetes documentation](https://kubernetes.io/docs/tasks/manage-gpus/scheduling-gpus/#using-device-plugins). <br>If you require CPU only, you can omit the entire `limits` section.|
 
-The `instance type` is *required* for model deployment. If you have defined the resource section, and it will be validated against the instance type, the rules are as follows:
+The instance type is *required* for model deployment. If you defined the `resources` section, and it will be validated against the instance type, the rules are as follows:
 
-- With a valid resource section definition, the resource limits must be less than instance type limits, otherwise deployment will fail. 
-- If the user does not define instance type, the `defaultinstancetype` will be used to be validated with resource section.
-- If the user does not define resource section, the instance type will be used to create deployment.
+- With a valid `resource` section definition, the resource limits must be less than the instance type limits. Otherwise, deployment will fail.
+- If you don't define an instance type, the system uses `defaultinstancetype` for validation with the `resources` section.
+- If you don't define the `resources` section, the system uses the instance type to create the deployment.
 
 ## Next steps
 
 - [Azure Machine Learning inference router and connectivity requirements](./how-to-kubernetes-inference-routing-azureml-fe.md)
-- [Secure AKS inferencing environment](./how-to-secure-kubernetes-inferencing-environment.md)
+- [Secure Azure Kubernetes Service inferencing environment](./how-to-secure-kubernetes-inferencing-environment.md)
