Skip to content

Commit a220b4a

Browse files
authored
Merge pull request #233544 from fbsolo-ms1/release-one-file-for-SK
Amar Badal requested some new files.
2 parents 420f63d + 1d60312 commit a220b4a

File tree

3 files changed

+444
-0
lines changed

3 files changed

+444
-0
lines changed
Lines changed: 241 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,241 @@
1+
---
2+
title: Use connections
3+
titleSuffix: Azure Machine Learning
4+
description: Learn how to use connections to connect to External data sources for training with Azure Machine Learning.
5+
services: machine-learning
6+
ms.service: machine-learning
7+
ms.subservice: mldata
8+
ms.topic: how-to
9+
ms.author: ambadal
10+
author: AmarBadal
11+
ms.reviewer: franksolomon
12+
ms.date: 04/11/2023
13+
ms.custom: data4ml
14+
15+
# Customer intent: As an experienced data scientist with Python skills, I have data located in external sources outside of Azure. I need to make that data available to the Azure Machine Learning platform, to train my machine learning models.
16+
---
17+
18+
# Create connections
19+
20+
[!INCLUDE [dev v2](../../includes/machine-learning-dev-v2.md)]
21+
22+
In this article, learn how to connect to data sources located outside of Azure, to make that data available to Azure Machine Learning services. For this data availability, Azure supports connections to these external sources:
23+
- Snowflake DB
24+
- Amazon S3
25+
- Azure SQL DB
26+
27+
## Prerequisites
28+
29+
- An Azure subscription. If you don't have an Azure subscription, create a free account before you begin. Try the [free or paid version of Azure Machine Learning](https://azure.microsoft.com/free/).
30+
31+
- The [Azure Machine Learning SDK for Python](https://aka.ms/sdk-v2-install).
32+
33+
- An Azure Machine Learning workspace.
34+
35+
> [!NOTE]
36+
> An Azure Machine Learning connection securely stores the credentials passed during connection creation in the Workspace Azure Key Vault. A connection references the credentials from the key vault storage location for further use. You won't need to directly deal with the credentials after they are stored in the key vault. You have the option to store the credentials in the YAML file. A CLI command or SDK can override them. We recommend that you **avoid** credential storage in a YAML file, because a security breach could lead to a credential leak.
37+
38+
## Create a Snowflake DB connection
39+
40+
# [CLI: Username/password](#tab/cli-username-password)
41+
This YAML file creates a Snowflake DB connection. Be sure to update the appropriate values:
42+
43+
```yaml
44+
# my_snowflakedb_connection.yaml
45+
$schema: http://azureml/sdk-2-0/Connection.json
46+
type: snowflake
47+
name: my_snowflakedb_connection # add your datastore name here
48+
49+
target: jdbc:snowflake://<myaccount>.snowflakecomputing.com/?db=<mydb>&warehouse=<mywarehouse>&role=<myrole>
50+
# add the Snowflake account, database, warehouse name and role name here. If no role name provided it will default to PUBLIC
51+
credentials:
52+
type: username_password
53+
username: <username> # add the Snowflake database user name here or leave this blank and type in CLI command line
54+
password: <password> # add the Snowflake database password here or leave this blank and type in CLI command line
55+
```
56+
57+
Create the Azure Machine Learning connection in the CLI:
58+
59+
### Option 1: Use the username and password in YAML file
60+
61+
```azurecli
62+
az ml connection create --file my_snowflakedb_connection.yaml
63+
```
64+
65+
### Option 2: Override the username and password at the command line
66+
67+
```azurecli
68+
az ml connection create --file my_snowflakedb_connection.yaml --set credentials.username="XXXXX" credentials.password="XXXXX"
69+
```
70+
71+
# [Python SDK: username/ password](#tab/sdk-username-password)
72+
73+
### Option 1: Load connection from YAML file
74+
75+
```python
76+
from azure.ai.ml import MLClient, load_workspace_connection
77+
78+
ml_client = MLClient.from_config()
79+
80+
wps_connection = load_workspace_connection(source="./my_snowflakedb_connection.yaml")
81+
wps_connection.credentials.username="XXXXX"
82+
wps_connection.credentials.password="XXXXXXXX"
83+
ml_client.connections.create_or_update(workspace_connection=wps_connection)
84+
85+
```
86+
87+
### Option 2: Use WorkspaceConnection() in a Python script
88+
89+
```python
90+
from azure.ai.ml import MLClient
91+
from azure.ai.ml.entities import WorkspaceConnection
92+
from azure.ai.ml.entities import UsernamePasswordConfiguration
93+
94+
target= "jdbc:snowflake://<myaccount>.snowflakecomputing.com/?db=<mydb>&warehouse=<mywarehouse>&role=<myrole>"
95+
# add the Snowflake account, database, warehouse name and role name here. If no role name provided it will default to PUBLIC
96+
97+
wps_connection = WorkspaceConnection(type="snowflake",
98+
target= target,
99+
credentials= UsernamePasswordConfiguration(username="XXXXX", password="XXXXXX")
100+
)
101+
102+
ml_client.connections.create_or_update(workspace_connection=wps_connection)
103+
104+
```
105+
106+
---
107+
108+
## Create an Azure SQL DB connection
109+
110+
# [CLI: Username/password](#tab/cli-sql-username-password)
111+
112+
This YAML script creates an Azure SQL DB connection. Be sure to update the appropriate values:
113+
114+
```yaml
115+
# my_sqldb_connection.yaml
116+
$schema: http://azureml/sdk-2-0/Connection.json
117+
118+
type: azuresqldb
119+
name: my_sqldb_connection
120+
121+
target: Server=tcp:<myservername>,<port>;Database=<mydatabase>;Trusted_Connection=False;Encrypt=True;Connection Timeout=30
122+
# add the sql servername, port addresss and database
123+
credentials:
124+
type: sql_auth
125+
username: <username> # add the sql database user name here or leave this blank and type in CLI command line
126+
password: <password> # add the sql database password here or leave this blank and type in CLI command line
127+
```
128+
129+
Create the Azure Machine Learning connection in the CLI:
130+
131+
### Option 1: Use the username / password from YAML file
132+
133+
```azurecli
134+
az ml connection create --file my_sqldb_connection.yaml
135+
```
136+
137+
### Option 2: Override the username and password in YAML file
138+
139+
```azurecli
140+
az ml connection create --file my_sqldb_connection.yaml --set credentials.username="XXXXX" credentials.password="XXXXX"
141+
```
142+
143+
# [Python SDK: username/ password](#tab/sdk-sql-username-password)
144+
145+
### Option 1: Load connection from YAML file
146+
147+
```python
148+
from azure.ai.ml import MLClient, load_workspace_connection
149+
150+
ml_client = MLClient.from_config()
151+
152+
wps_connection = load_workspace_connection(source="./my_sqldb_connection.yaml")
153+
wps_connection.credentials.username="XXXXXX"
154+
wps_connection.credentials.password="XXXXXxXXX"
155+
ml_client.connections.create_or_update(workspace_connection=wps_connection)
156+
157+
```
158+
159+
### Option 2: Using WorkspaceConnection()
160+
161+
```python
162+
from azure.ai.ml import MLClient
163+
from azure.ai.ml.entities import WorkspaceConnection
164+
from azure.ai.ml.entities import UsernamePasswordConfiguration
165+
166+
target= "Server=tcp:<myservername>,<port>;Database=<mydatabase>;Trusted_Connection=False;Encrypt=True;Connection Timeout=30"
167+
# add the sql servername, port addresss and database
168+
169+
wps_connection = WorkspaceConnection(type="azure_sql_db",
170+
target= target,
171+
credentials= UsernamePasswordConfiguration(username="XXXXX", password="XXXXXX")
172+
)
173+
174+
ml_client.connections.create_or_update(workspace_connection=wps_connection)
175+
176+
```
177+
178+
---
179+
180+
## Create Amazon S3 connection
181+
182+
# [CLI: Access key](#tab/cli-s3-access-key)
183+
184+
Create an Amazon S3 connection with the following YAML file. Be sure to update the appropriate values:
185+
186+
```yaml
187+
# my_s3_connection.yaml
188+
$schema: http://azureml/sdk-2-0/Connection.json
189+
190+
type: s3
191+
name: my_s3_connection
192+
193+
target: https://<mybucket>.amazonaws.com # add the s3 bucket details
194+
credentials:
195+
type: access_key
196+
access_key_id: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX # add access key id
197+
secret_access_key: XxXxXxXXXXXXXxXxXxxXxxXXXXXXXXxXxxXXxXXXXXXXxxxXxXXxXXXXXxXXxXXXxXxXxxxXXxXXxXXXXXxXxxXX # add access key secret
198+
```
199+
200+
Create the Azure Machine Learning connection in the CLI:
201+
202+
```azurecli
203+
az ml connection create --file my_s3_connection.yaml
204+
```
205+
206+
# [Python SDK: Access key](#tab/sdk-s3-access-key)
207+
208+
### Option 1: Load connection from YAML file
209+
210+
```python
211+
from azure.ai.ml import MLClient, load_workspace_connection
212+
213+
ml_client = MLClient.from_config()
214+
215+
216+
wps_connection = load_workspace_connection(source="./my_s3_connection.yaml")
217+
ml_client.connections.create_or_update(workspace_connection=wps_connection)
218+
219+
```
220+
221+
### Option 2: Use WorkspaceConnection() in a Python script
222+
223+
```python
224+
from azure.ai.ml import MLClient
225+
from azure.ai.ml.entities import WorkspaceConnection
226+
from azure.ai.ml.entities import AccessKeyConfiguration
227+
228+
target = "https://<mybucket>.amazonaws.com" # add the s3 bucket details
229+
wps_connection = WorkspaceConnection(type="s3",
230+
target= target,
231+
credentials= AccessKeyConfiguration(access_key_id="XXXXXX",acsecret_access_key="XXXXXXXX")
232+
)
233+
234+
ml_client.connections.create_or_update(workspace_connection=wps_connection)
235+
236+
```
237+
---
238+
239+
## Next steps
240+
241+
- [Import data assets](how-to-import-data-assets.md#import-data-assets)

0 commit comments

Comments
 (0)