Skip to content

Commit 6bc3b5f

Browse files
author
helenz-msft
committed
submit for review
1 parent 3a9e993 commit 6bc3b5f

23 files changed

+118
-0
lines changed
Lines changed: 116 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,116 @@
1+
---
2+
title: Linking Tables in OneLake to Azure ML Through UI
3+
titleSuffix: Azure Machine Learning, Microsoft Fabric
4+
description: Learn how to link table in Lakehouse to Azure ML studio through UI.
5+
services: machine-learning
6+
ms.service: azure-machine-learning
7+
ms.subservice: mldata
8+
ms.topic: how-to
9+
ms.author: helenzeng
10+
author: helenzusa1
11+
ms.reviewer: franksolomon
12+
ms.date: 02/10/2025
13+
ms.custom: data4ml
14+
# Customer intent: I have some Tables in OneLake, I want to use Azure ML studio for ML activities, so I need to link the Tables in AML, and I prefer to do it through UI.
15+
---
16+
17+
# Introduction of the solution
18+
There were some existing solutions which can build a link in Azure ML to OneLake, get the data and create datastore in AML.
19+
However, in those solutions, the data in OneLake is of type ‘Files’. Please refer to reference section.
20+
Some customers have data in OneLake as type ‘Tables’. The existing solutions creating datastore in AML don’t work.
21+
22+
:::image type="content" source="media/create-datastore-with-UI/1-table-in-fabric.png" alt-text="Screenshot that shows how the table looks like in Microsoft Fabric." lightbox="./media/create-datastore-with-UI/1-table-in-fabric.png":::
23+
24+
Also some customers prefer doing in UI. Therefore, a new solution is needed to link AML to Tables in OneLake using UI.
25+
26+
In this article, you will learn how to link tables in OneLake to Azure ML studio through UI.
27+
28+
## Overall idea
29+
30+
:::image type="content" source="media/create-datastore-with-UI/2-overall-idea.png" alt-text="Screenshot that shows overall idea." lightbox="./media/create-datastore-with-UI/2-overall-idea.png":::
31+
32+
33+
## Step 1. Create data lake storage account in Azure portal
34+
35+
:::image type="content" source="media/create-datastore-with-UI/3-create-storage-account.png" alt-text="Screenshot that shows an example of data lake storage account in Azure portal." lightbox="./media/create-datastore-with-UI/3-create-storage-account.png":::
36+
37+
## Step 2. Enable storage account key access
38+
39+
:::image type="content" source="media/create-datastore-with-UI/4-enable-key-access.png" alt-text="Screenshot that shows how to enable key access of data lake storage account in Azure portal." lightbox="./media/create-datastore-with-UI/4-enable-key-access.png":::
40+
41+
## Step 3. Assign roles to the storage account
42+
43+
:::image type="content" source="media/create-datastore-with-UI/5-assign-roles.png" alt-text="Screenshot that shows how to assign roles of data lake storage account in Azure portal." lightbox="./media/create-datastore-with-UI/5-assign-roles.png":::
44+
45+
## Step 4. Create a container
46+
47+
:::image type="content" source="media/create-datastore-with-UI/6-create-container.png" alt-text="Screenshot that shows how to create a container of data lake storage account in Azure portal." lightbox="./media/create-datastore-with-UI/6-create-container.png":::
48+
49+
## Step 5. In Fabric, create a pipeline
50+
51+
:::image type="content" source="media/create-datastore-with-UI/7-create-pipeline.png" alt-text="Screenshot that shows how to create a data pipeline in Fabric." lightbox="./media/create-datastore-with-UI/7-create-pipeline.png":::
52+
53+
## Step 6. Use pipeline, start 'Copy data assistant'
54+
55+
:::image type="content" source="media/create-datastore-with-UI/8-copy-data-assistant.png" alt-text="Screenshot that shows how to start Copy data assistant in Fabric." lightbox="./media/create-datastore-with-UI/8-copy-data-assistant.png":::
56+
57+
## Step 7. In 'Copy data assistant', select Azure Blobs
58+
59+
:::image type="content" source="media/create-datastore-with-UI/9-select-azure-blob.png" alt-text="Screenshot that shows how to select Azure blobs in Copy data assistant in Fabric." lightbox="./media/create-datastore-with-UI/9-select-azure-blob.png":::
60+
61+
## Step 8. Set storage account, create connection
62+
63+
:::image type="content" source="media/create-datastore-with-UI/10-create-connection.png" alt-text="Screenshot that shows how to create connection in Fabric data pepiline." lightbox="./media/create-datastore-with-UI/10-create-connection.png":::
64+
65+
## Step 9. Select destination folder
66+
67+
:::image type="content" source="media/create-datastore-with-UI/11-select-destination-folder.png" alt-text="Screenshot that shows how to select destination folder." lightbox="./media/create-datastore-with-UI/11-select-destination-folder.png":::
68+
69+
## Step 10. Connect to data destination
70+
71+
:::image type="content" source="media/create-datastore-with-UI/12-connect-data-destination.png" alt-text="Screenshot that shows how to connect to data destination." lightbox="./media/create-datastore-with-UI/12-connect-data-destination.png":::
72+
73+
## Step 11. Copy activity is scheduled
74+
75+
:::image type="content" source="media/create-datastore-with-UI/13-copy-activity-scheduled.png" alt-text="Screenshot that shows the copy activity is scheduled." lightbox="./media/create-datastore-with-UI/13-copy-activity-scheduled.png":::
76+
77+
## Step 12. Copy success
78+
79+
:::image type="content" source="media/create-datastore-with-UI/14-copy-activity-success.png" alt-text="Screenshot that shows the copy is successful." lightbox="./media/create-datastore-with-UI/14-copy-activity-success.png":::
80+
81+
## Step 13. Check storage account, container in Azure portal
82+
83+
:::image type="content" source="media/create-datastore-with-UI/15-check-container.png" alt-text="Screenshot that shows how to verify the data in Azure storage account container." lightbox="./media/create-datastore-with-UI/15-check-container.png":::
84+
85+
## Step 14. In ML studio create data asset, type 'File'
86+
87+
:::image type="content" source="media/create-datastore-with-UI/16-create-data-asset.png" alt-text="Screenshot that shows how to create a data asset in Azure ML studio." lightbox="./media/create-datastore-with-UI/16-create-data-asset.png":::
88+
89+
## Step 15. Select 'From Azure storage'
90+
91+
:::image type="content" source="media/create-datastore-with-UI/17-select-azure-storage.png" alt-text="Screenshot that shows how to select Azure storage." lightbox="./media/create-datastore-with-UI/17-select-azure-storage.png":::
92+
93+
## Step 16. It will bring you to create 'New datastore'
94+
95+
:::image type="content" source="media/create-datastore-with-UI/18-new-datastore.png" alt-text="Screenshot that shows how to create new datastore in Azure ML." lightbox="./media/create-datastore-with-UI/18-new-datastore.png":::
96+
97+
## Step 17. Or you can directly 'Create datastore' from the beginning
98+
99+
:::image type="content" source="media/create-datastore-with-UI/19-create-datastore.png" alt-text="Screenshot that shows how to create a datastore in Azure ML." lightbox="./media/create-datastore-with-UI/19-create-datastore.png":::
100+
101+
## Step 18. Datastore is created
102+
103+
:::image type="content" source="media/create-datastore-with-UI/20-datastore-created.png" alt-text="Screenshot that shows the datastore looks like after creation." lightbox="./media/create-datastore-with-UI/20-datastore-created.png":::
104+
105+
## Step 19. Access the datastore
106+
107+
:::image type="content" source="media/create-datastore-with-UI/21-access-datastore.png" alt-text="Screenshot that shows how to access a datastore in Azure ML." lightbox="./media/create-datastore-with-UI/21-access-datastore.png":::
108+
109+
## References
110+
111+
- https://community.fabric.microsoft.com/t5/Data-Engineering/Read-from-a-specified-table-from-lakehouse-in-One-workspace/m-p/4234885
112+
- https://fabric.guru/delta-lake-tables-for-optimal-direct-lake-performance-in-fabric-python-notebook
113+
- https://learn.microsoft.com/en-us/azure/machine-learning/how-to-datastore?view=azureml-api-2&tabs=sdk-identity-based-access%2Csdk-adls-identity-access%2Csdk-azfiles-accountkey%2Csdk-adlsgen1-identity-access%2Csdk-onelake-identity-access#create-a-onelake-microsoft-fabric-datastore-preview
114+
- https://learn.microsoft.com/en-us/fabric/data-engineering/spark-data-warehouse-connector
115+
- https://learn.microsoft.com/en-us/azure/storage/blobs/data-lake-storage-abfs-driver
116+
- https://github.com/azeltov/aml_one_lake

articles/machine-learning/how-to-datastore.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -544,6 +544,8 @@ az ml datastore create --file my_onelakesp_datastore.yml
544544
```
545545
---
546546

547+
## [Create a datastore for OneLake Table with UI](create-datastore-with-UI.md#create-datastore-with-UI)
548+
547549
## Next steps
548550

549551
- [Access data in a job](how-to-read-write-data-v2.md#access-data-in-a-job)
409 KB
Loading
296 KB
Loading
301 KB
Loading
285 KB
Loading
308 KB
Loading
316 KB
Loading
474 KB
Loading
224 KB
Loading

0 commit comments

Comments
 (0)