Skip to content

Commit b8727cb

Browse files
authored
Merge pull request #278347 from MicrosoftDocs/main
6/14/2024 PM Publish
2 parents 46f075e + 0e2fddf commit b8727cb

File tree

88 files changed

+984
-1122
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

88 files changed

+984
-1122
lines changed
Lines changed: 68 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,68 @@
1+
name: "📄 Feedback control template"
2+
title: "Feedback"
3+
description: >-
4+
⛔ This template is intended for use by the feedback control on the bottom of every page on the
5+
live site. If you aren't using the feedback control, choose one of the other templates.⛔
6+
labels:
7+
- "needs-triage"
8+
body:
9+
- type: markdown
10+
attributes:
11+
value: "## 🎁Enter your feedback🎁"
12+
- type: markdown
13+
attributes:
14+
value: Select the issue type, and describe the issue in the text box below. Add as much detail as needed to help us resolve the issue.
15+
- type: dropdown
16+
id: issue-type
17+
attributes:
18+
label: Type of issue
19+
options:
20+
- Typo
21+
- Code doesn't work
22+
- Missing information
23+
- Outdated article
24+
- Other (describe below)
25+
validations:
26+
required: true
27+
- type: textarea
28+
id: userfeedback
29+
validations:
30+
required: true
31+
attributes:
32+
label: Feedback
33+
description: >-
34+
If possible, please provide extended details that will add context and help the team update
35+
the documentation. Additional details may not be useful for typos, grammar, formatting, etc.
36+
For technical or factual errors, please include code snippets and output to show how the
37+
documentation is incorrect.
38+
- type: markdown
39+
attributes:
40+
value: "## 🚧 Article information 🚧"
41+
- type: markdown
42+
attributes:
43+
value: "*Don't modify the following fields*. They are automatically filled in for you. Doing so will disconnect your issue from the affected article. *Don't edit them*."
44+
- type: input
45+
id: pageUrl
46+
validations:
47+
required: true
48+
attributes:
49+
label: Page URL
50+
- type: input
51+
id: contentSourceUrl
52+
validations:
53+
required: true
54+
attributes:
55+
label: Content source URL
56+
- type: input
57+
id: author
58+
validations:
59+
required: true
60+
attributes:
61+
label: Author
62+
description: GitHub Id of the author
63+
- type: input
64+
id: documentVersionIndependentId
65+
validations:
66+
required: true
67+
attributes:
68+
label: Document Id

articles/ai-services/openai/how-to/use-your-data-securely.md

Lines changed: 47 additions & 42 deletions
Large diffs are not rendered by default.
7.46 KB
Loading
17.3 KB
Loading
8.9 KB
Loading
100 KB
Loading
32.8 KB
Loading

articles/aks/TOC.yml

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -883,6 +883,8 @@
883883
href: open-ai-secure-access-quickstart.md
884884
- name: Deploy an AI model with the AI toolchain operator
885885
href: ai-toolchain-operator.md
886+
- name: Deploy data and ML pipelines with Flyte
887+
href: use-flyte.md
886888
- name: DevOps
887889
items:
888890
- name: Azure DevOps Project

articles/aks/use-flyte.md

Lines changed: 158 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,158 @@
1+
---
2+
title: Build and deploy data and machine learning pipelines with Flyte on Azure Kubernetes Service (AKS)
3+
titleSuffix: Azure Kubernetes Service
4+
description: Learn about Flyte, an open-source platform for building and deploying data and machine learning pipelines on Azure Kubernetes Service (AKS).
5+
ms.topic: how-to
6+
ms.date: 06/06/2024
7+
author: schaffererin
8+
ms.author: schaffererin
9+
ms.service: azure-kubernetes-service
10+
---
11+
12+
# Build and deploy data and machine learning pipelines with Flyte on Azure Kubernetes Service (AKS)
13+
14+
This article shows you how to use Flyte on Azure Kubernetes Service (AKS). Flyte is an open-source workflow orchestrator that unifies machine learning, data engineering, and data analytics stacks to help you build robust and reliable applications. When using Flyte as a Kubernetes-native workflow automation tool, you can focus on experimentation and providing business value without increasing your scope to infrastructure and resource management. Keep in mind that Flyte isn't officially supported by Microsoft, so use it at your own discretion.
15+
16+
For more information, see [Introduction to Flyte][flyte].
17+
18+
## Flyte use cases
19+
20+
Flyte can be used for a variety of use cases, including:
21+
22+
* Deliver models for streamlined profit and loss financial calculations.
23+
* Process petabytes of data to efficiently conduct 3D mapping of new areas.
24+
* Quickly rollback to previous versions and minimize impact of bugs in your pipelines.
25+
26+
For more information, see [Core Flyte use cases](https://docs.flyte.org/en/latest/core_use_cases/index.html).
27+
28+
## Prerequisites
29+
30+
* An Azure subscription. If you don't have an Azure subscription, you can create a [free account][azure-free].
31+
* If you have multiple subscriptions, make sure you select the correct one using the `az account set --subscription <subscription-id>` command.
32+
* The Azure CLI installed and configured. Check your version using the `az --version` command. If you need to install or upgrade, see [Install the Azure CLI][install-azure-cli].
33+
* The Helm CLI installed and updated. Check your version using the `helm version` command. If you need to install or upgrade, see [Install Helm][install-helm].
34+
* The `kubectl` CLI installed and updated. Install it locally using the `az aks install-cli` command or using [Install kubectl][install-kubectl].
35+
* A local Docker development environment. For more information, see [Get Docker][get-docker].
36+
* `flytekit` and `flytectl` installed. For more information, see [Flyte installation][flyte-install].
37+
38+
> [!NOTE]
39+
> If you're using the Azure Cloud Shell, the Azure CLI, Helm, and kubectl are already installed.
40+
41+
### Set environment variables
42+
43+
* Set environment variables for use throughout the article. Replace the placeholder values with your own values.
44+
45+
```bash
46+
export RESOURCE_GROUP="<resource-group-name>"
47+
export LOCATION="<location>"
48+
export CLUSTER_NAME="<cluster-name>"
49+
export DNS_NAME_PREFIX="<dns-name-prefix>"
50+
```
51+
52+
## Create an AKS cluster
53+
54+
1. Create an Azure resource group for the AKS cluster using the [`az group create`][az-group-create] command.
55+
56+
```azurecli-interactive
57+
az group create --name $RESOURCE_GROUP --location $LOCATION
58+
```
59+
60+
2. Create an AKS cluster using the [`az aks create`][az-aks-create] command with the `--enable-azure-rbac`, `--enable-managed-identity`, `--enable-aad`, and `--dns-name-prefix` parameters.
61+
62+
```azurecli-interactive
63+
az aks create --resource-group $RESOURCE_GROUP --name $CLUSTER_NAME --enable-azure-rbac --enable-managed-identity --enable-aad --dns-name-prefix $DNS_NAME_PREFIX --generate-ssh-keys
64+
```
65+
66+
## Connect to your AKS cluster
67+
68+
* Configure `kubectl` to connect to your AKS cluster using the [`az aks get-credentials`][az-aks-get-credentials] command.
69+
70+
```azurecli-interactive
71+
az aks get-credentials --resource-group $RESOURCE_GROUP --name $CLUSTER_NAME
72+
```
73+
74+
## Add the Flyte Helm repository
75+
76+
* Add the Flyte Helm repository using the `helm repo add` command.
77+
78+
```bash
79+
helm repo add flyteorg https://flyteorg.github.io/flyte
80+
```
81+
82+
## Find Flyte Helm charts
83+
84+
1. Search for Flyte Helm charts using the `helm search repo` command.
85+
86+
```bash
87+
helm search repo flyteorg
88+
```
89+
90+
The following example output shows some of the available Flyte Helm charts:
91+
92+
```output
93+
NAME CHART VERSION APP VERSION DESCRIPTION
94+
flyteorg/flyte v1.12.0 A Helm chart for Flyte Sandbox
95+
flyteorg/flyte-binary v1.12.0 1.16.0 Chart for basic single Flyte executable deployment
96+
flyteorg/flyte-core v1.12.0 A Helm chart for Flyte core
97+
flyteorg/flyte-deps v1.12.0 A Helm chart for Flyte dependencies
98+
flyteorg/flyte-sandbox 0.1.0 1.16.1 A Helm chart for the Flyte local sandbox
99+
flyteorg/flyteagent v0.1.10 A Helm chart for Flyte Agent
100+
```
101+
102+
2. Update the repository using the `helm repo update` command.
103+
104+
```bash
105+
helm repo update
106+
```
107+
108+
## Deploy a Flyte chart on AKS
109+
110+
In this section, you deploy the flyte-binary Helm chart so you can begin building and deploying data and machine learning pipelines with Flyte on AKS. The flyte-binary chart is a basic single Flyte executable deployment.
111+
112+
1. Create a namespace for your Flyte deployment using the `kubectl create namespace` command.
113+
114+
```bash
115+
kubectl create namespace <namespace-name>
116+
```
117+
118+
2. Install a Flyte Helm chart using the `helm install` command. In this example, we use the `flyte-binary` chart.
119+
120+
```bash
121+
helm install flyte-binary flyteorg/flyte-core --namespace <namespace-name>
122+
```
123+
124+
3. Verify that the Flyte deployment is running using the `kubectl get services` command.
125+
126+
```bash
127+
kubectl get services --namespace <namespace-name> --output wide
128+
```
129+
130+
The following condensed example output shows the Flyte deployment:
131+
132+
```output
133+
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
134+
flyteorg-flyte-binary-grpc ClusterIP xx.x.xx.xxx <none> 81/TCP 1m
135+
flyteorg-flyte-binary-http ClusterIP xx.x.xx.xxx <none> 80/TCP 1m
136+
flyteorg-flyte-binary-webhook ClusterIP xx.x.xx.xxx <none> 80/TCP 1m
137+
```
138+
139+
## Next steps
140+
141+
In this article, you learned how to deploy a Flyte chart on AKS. To start building and deploying data and machine learning pipelines, see the following articles:
142+
143+
* [Perform exploratory data analysis (EDA) with Flyte and Jupyter notebooks][flyte-eda]
144+
* [Orchestrate an ML pipeline with Flyte to predict housing prices across regions][flyte-pipelines]
145+
146+
<!-- LINKS -->
147+
[az-group-create]: /cli/azure/group#az-group-create
148+
[az-aks-create]: /cli/azure/aks#az-aks-create
149+
[az-aks-get-credentials]: /cli/azure/aks#az-aks-get-credentials
150+
[flyte]: https://docs.flyte.org/en/latest/introduction.html
151+
[azure-free]: https://azure.microsoft.com/free
152+
[install-azure-cli]: /cli/azure/install-azure-cli
153+
[install-helm]: https://helm.sh/docs/intro/install/
154+
[install-kubectl]: https://kubernetes.io/docs/tasks/tools/install-kubectl/
155+
[get-docker]: https://docs.docker.com/get-docker/
156+
[flyte-install]: https://flyte-next.readthedocs.io/en/latest/introduction.html#installation
157+
[flyte-eda]: https://docs.flyte.org/en/latest/flytesnacks/examples/exploratory_data_analysis/index.html
158+
[flyte-pipelines]: https://docs.flyte.org/en/latest/flytesnacks/examples/house_price_prediction/index.html

articles/aks/use-managed-identity.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -135,14 +135,14 @@ CLIENT_ID=$(az aks show \
135135

136136
To grant a system-assigned managed identity permissions to a resource in Azure, call the [`az role assignment create`][az-role-assignment-create] command to assign an Azure RBAC role to the managed identity.
137137

138-
For a VNet, attached Azure disk, static IP address, or route table outside the default worker node resource group, you need to assign the `Contributor` role on the custom resource group.
138+
For a VNet, attached Azure disk, static IP address, or route table outside the default worker node resource group, you need to assign the ` Network Contributor` role on the custom resource group.
139139

140-
For example, assign the `Contributor` role on the custom resource group using the [`az role assignment create`][az-role-assignment-create] command. For the `--scope` parameter, provide the resource ID for the resource group for the cluster.
140+
For example, assign the `Network Contributor` role on the custom resource group using the [`az role assignment create`][az-role-assignment-create] command. For the `--scope` parameter, provide the resource ID for the resource group for the cluster.
141141

142142
```azurecli-interactive
143143
az role assignment create \
144144
--assignee $CLIENT_ID \
145-
--role "Contributor" \
145+
--role "Network Contributor" \
146146
--scope "<resource-group-id>"
147147
```
148148

0 commit comments

Comments
 (0)