Skip to content

Commit 515a5da

Browse files
fbsolo-ms1fbsolo-ms1
committed
File updates for Amar Badal.
1 parent ffbbd4b commit 515a5da

File tree

2 files changed

+429
-0
lines changed

2 files changed

+429
-0
lines changed

‎articles/machine-learning/how-to-connection.md

Lines changed: 239 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,239 @@
1+
---
2+
title: Use connections
3+
titleSuffix: Azure Machine Learning
4+
description: Learn how to use connections to connect to External data sources for training with Azure Machine Learning.
5+
services: machine-learning
6+
ms.service: machine-learning
7+
ms.subservice: mldata
8+
ms.topic: how-to
9+
ms.author: AmarBadal
10+
author: AmarBadal
11+
ms.reviewer: franksolomon
12+
ms.date: 04/04/2023
13+
ms.custom: data4ml
14+
15+
# Customer intent: As an experienced data scientist with Python skills, I have data located in external sources outside of Azure. I need to make that data available to the Azure Machine Learning platform, to train my machine learning models.
16+
---
17+
18+
# Create connections
19+
20+
[!INCLUDE [dev v2](../../includes/machine-learning-dev-v2.md)]
21+
22+
In this article, learn how to connect to data sources located outside of Azure, to make that data available to Azure Machine Learning services. For this data availability, Azure supports connections to these external sources:
23+
- Snowflake DB
24+
- Amazon S3
25+
- Azure SQL DB
26+
27+
## Prerequisites
28+
29+
- An Azure subscription. If you don't have an Azure subscription, create a free account before you begin. Try the [free or paid version of Azure Machine Learning](https://azure.microsoft.com/free/).
30+
31+
- The [Azure Machine Learning SDK for Python](https://aka.ms/sdk-v2-install).
32+
33+
- An Azure Machine Learning workspace.
34+
35+
> [!NOTE]
36+
> An Azure Machine Learning connection stores the credentials passed during connection creation in the Workspace Azure Key Vault. A connection references the credentials from that location for further use. The YAML cna pass the credentials. A CLI command or SDK can override them. We recommend that you **avoid** credential storage in YAML files.
37+
38+
## Create a Snowflake DB connection
39+
40+
# [CLI: Username/password](#tab/cli-username-password)
41+
This YAML script creates a Snowflake DB connection. Be sure to update the appropriate values:
42+
43+
```yaml
44+
# my_snowflakedb_connection.yaml
45+
$schema: http://azureml/sdk-2-0/Connection.json
46+
type: snowflake
47+
name: my_snowflakedb_connection # add your datastore name here
48+
49+
target: jdbc:snowflake://<myaccount>.snowflakecomputing.com/?db=<mydb>&warehouse=<mywarehouse>&role=<myrole>
50+
# add the Snowflake account, database, warehouse name and role name here. If no role name provided it will default to PUBLIC
51+
credentials:
52+
type: username_password
53+
username: <username> # add the Snowflake database user name here or leave this blank and type in CLI command line
54+
password: <password> # add the Snowflake database password here or leave this blank and type in CLI command line
55+
```
56+
57+
Create the Azure Machine Learning datastore in the CLI:
58+
59+
### Option 1: Use the username and password in a YAML script
60+
61+
```azurecli
62+
az ml connection create --file my_snowflakedb_connection.yaml
63+
```
64+
65+
### Option 2: Override the username and password at the command line
66+
67+
```azurecli
68+
az ml connection create --file my_snowflakedb_connection.yaml --set credentials.username="XXXXX" credentials.password="XXXXX"
69+
```
70+
71+
# [Python SDK: username/ password](#tab/sdk-username-password)
72+
73+
### Option 1: Load the connection in a YAML script
74+
75+
```python
76+
from azure.ai.ml import MLClient, load_workspace_connection
77+
78+
ml_client = MLClient.from_config()
79+
80+
wps_connection = load_workspace_connection(source="./my_snowflakedb_connection.yaml")
81+
wps_connection.credentials.username="XXXXX"
82+
wps_connection.credentials.password="XXXXXXXX"
83+
ml_client.connections.create_or_update(workspace_connection=wps_connection)
84+
85+
```
86+
---
87+
88+
### Option 2: Use WorkspaceConnection() in a Python script
89+
90+
```python
91+
from azure.ai.ml import MLClient
92+
from azure.ai.ml.entities import WorkspaceConnection
93+
from azure.ai.ml.entities import UsernamePasswordConfiguration
94+
95+
target= "jdbc:snowflake://<myaccount>.snowflakecomputing.com/?db=<mydb>&warehouse=<mywarehouse>&role=<myrole>"
96+
# add the Snowflake account, database, warehouse name and role name here. If no role name provided it will default to PUBLIC
97+
98+
wps_connection = WorkspaceConnection(type="snowflake",
99+
target= target,
100+
credentials= UsernamePasswordConfiguration(username="XXXXX", password="XXXXXX")
101+
)
102+
103+
ml_client.connections.create_or_update(workspace_connection=wps_connection)
104+
105+
```
106+
107+
## Create an Azure SQL DB connection
108+
109+
# [CLI: Username/password](#tab/cli-sql-username-password)
110+
111+
This YAML script creates an Azure SQL DB connection. Be sure to update the appropriate values:
112+
113+
```yaml
114+
# my_sqldb_connection.yaml
115+
$schema: http://azureml/sdk-2-0/Connection.json
116+
117+
type: azuresqldb
118+
name: my_sqldb_connection
119+
120+
target: Server=tcp:<myservername>,<port>;Database=<mydatabase>;Trusted_Connection=False;Encrypt=True;Connection Timeout=30
121+
# add the sql servername, port addresss and database
122+
credentials:
123+
type: sql_auth
124+
username: <username> # add the sql database user name here or leave this blank and type in CLI command line
125+
password: <password> # add the sql database password here or leave this blank and type in CLI command line
126+
```
127+
128+
Create the Azure Machine Learning datastore in the CLI:
129+
130+
### Option 1: Use the username/ password in a YAML script
131+
132+
```azurecli
133+
az ml connection create --file my_sqldb_connection.yaml
134+
```
135+
136+
### Option 2: Override the username and password in the YAML file
137+
138+
```azurecli
139+
az ml connection create --file my_sqldb_connection.yaml --set credentials.username="XXXXX" credentials.password="XXXXX"
140+
```
141+
142+
# [Python SDK: username/ password](#tab/sdk-sql-username-password)
143+
144+
### Option 1: Load the connection in a YAML script
145+
146+
```python
147+
from azure.ai.ml import MLClient, load_workspace_connection
148+
149+
ml_client = MLClient.from_config()
150+
151+
wps_connection = load_workspace_connection(source="./my_sqldb_connection.yaml")
152+
wps_connection.credentials.username="XXXXXX"
153+
wps_connection.credentials.password="XXXXXxXXX"
154+
ml_client.connections.create_or_update(workspace_connection=wps_connection)
155+
156+
```
157+
---
158+
159+
### Option 2: Using WorkspaceConnection()
160+
161+
```python
162+
from azure.ai.ml import MLClient
163+
from azure.ai.ml.entities import WorkspaceConnection
164+
from azure.ai.ml.entities import UsernamePasswordConfiguration
165+
166+
target= "Server=tcp:<myservername>,<port>;Database=<mydatabase>;Trusted_Connection=False;Encrypt=True;Connection Timeout=30"
167+
# add the sql servername, port addresss and database
168+
169+
wps_connection = WorkspaceConnection(type="azure_sql_db",
170+
target= target,
171+
credentials= UsernamePasswordConfiguration(username="XXXXX", password="XXXXXX")
172+
)
173+
174+
ml_client.connections.create_or_update(workspace_connection=wps_connection)
175+
176+
```
177+
178+
## Create Amazon S3 connection
179+
180+
# [CLI: Access key](#tab/cli-s3-access-key)
181+
182+
Create an Amazon S3 connection with the following YAML file. Be sure to update the appropriate values:
183+
184+
```yaml
185+
# my_s3_connection.yaml
186+
$schema: http://azureml/sdk-2-0/Connection.json
187+
188+
type: s3
189+
name: my_s3_connection
190+
191+
target: https://<mybucket>.amazonaws.com # add the s3 bucket details
192+
credentials:
193+
type: access_key
194+
access_key_id: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX # add access key id
195+
secret_access_key: XxXxXxXXXXXXXxXxXxxXxxXXXXXXXXxXxxXXxXXXXXXXxxxXxXXxXXXXXxXXxXXXxXxXxxxXXxXXxXXXXXxXxxXX # add access key secret
196+
```
197+
198+
Create the Azure Machine Learning datastore in the CLI:
199+
200+
```azurecli
201+
az ml connection create --file my_s3_connection.yaml
202+
```
203+
204+
# [Python SDK: Access key](#tab/sdk-s3-access-key)
205+
206+
### Option 1: Load the connection in a YAML script
207+
208+
```python
209+
from azure.ai.ml import MLClient, load_workspace_connection
210+
211+
ml_client = MLClient.from_config()
212+
213+
214+
wps_connection = load_workspace_connection(source="./my_s3_connection.yaml")
215+
ml_client.connections.create_or_update(workspace_connection=wps_connection)
216+
217+
```
218+
219+
### Option 2: Use WorkspaceConnection() in a Python script
220+
221+
```python
222+
from azure.ai.ml import MLClient
223+
from azure.ai.ml.entities import WorkspaceConnection
224+
from azure.ai.ml.entities import AccessKeyConfiguration
225+
226+
target = "https://<mybucket>.amazonaws.com" # add the s3 bucket details
227+
wps_connection = WorkspaceConnection(type="s3",
228+
target= target,
229+
credentials= AccessKeyConfiguration(access_key_id="XXXXXX",acsecret_access_key="XXXXXXXX")
230+
)
231+
232+
ml_client.connections.create_or_update(workspace_connection=wps_connection)
233+
234+
```
235+
---
236+
237+
## Next steps
238+
239+
- [Import data assets](how-to-import-data-assets.md#import-data-assets)

0 commit comments

Comments
 (0)