Skip to content

Commit 424edd2

Browse files
authored
Merge pull request #88971 from csteegz/coverste/update-fe
Add updating AML components
2 parents 9be79c3 + 6175d0d commit 424edd2

File tree

1 file changed

+40
-0
lines changed

1 file changed

+40
-0
lines changed

articles/machine-learning/service/resource-known-issues.md

Lines changed: 40 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -172,3 +172,43 @@ For example, you will receive an error if you try to create or attach a compute
172172
If you receive an error `Unable to upload project files to working directory in AzureFile because the storage is overloaded`, apply following workarounds.
173173

174174
If you are using file share for other workloads, such as data transfer, the recommendation is to use blobs so that file share is free to be used for submitting runs. You may also split the workload between two different workspaces.
175+
176+
## Webservices in Azure Kubernetes Service failures
177+
178+
Many webservice failures in Azure Kubernetes Service can be debugged by connecting to the cluster using `kubectl`. You can get the `kubeconfig.json` for an Azure Kubernetes Service Cluster by running
179+
180+
```bash
181+
az aks get-credentials -g <rg> -n <aks cluster name>
182+
```
183+
184+
## Updating Azure Machine Learning components in AKS cluster
185+
186+
Updates to Azure Machine Learning components installed in an Azure Kubernetes Service cluster must be manually applied. You can apply these clusters by detaching the cluster from the Azure Machine Learning workspace, and then re-attaching the cluster to the workspace. If SSL is enabled in the cluster, you will need to supply the SSL certificate and private key when re-attaching the cluster.
187+
188+
```python
189+
compute_target = ComputeTarget(workspace=ws, name=clusterWorkspaceName)
190+
compute_target.detach()
191+
compute_target.wait_for_completion(show_output=True)
192+
193+
attach_config = AksCompute.attach_configuration(resource_group=resourceGroup, cluster_name=kubernetesClusterName)
194+
195+
## If SSL is enabled.
196+
attach_config.enable_ssl(
197+
ssl_cert_pem_file="cert.pem",
198+
ssl_key_pem_file="key.pem",
199+
ssl_cname=sslCname)
200+
201+
attach_config.validate_configuration()
202+
203+
compute_target = ComputeTarget.attach(workspace=ws, name=args.clusterWorkspaceName, attach_configuration=attach_config)
204+
compute_target.wait_for_completion(show_output=True)
205+
```
206+
207+
If you no longer have the SSL certificate and private key, or you are using a certificate generated by Azure Machine Learning, you can retrieve the files prior to detaching the cluster by connecting to the cluster using `kubectl` and retrieving the secret `azuremlfessl`.
208+
209+
```bash
210+
kubectl get secret/azuremlfessl -o yaml
211+
```
212+
213+
>[!Note]
214+
>Kubernetes stores the secrets in base-64 encoded format. You will need to base-64 decode the `cert.pem` and `key.pem` components of the secrets prior to providing them to `attach_config.enable_ssl`.

0 commit comments

Comments
 (0)