Skip to content

Commit 8b633cf

Browse files
authored
Merge pull request #285097 from WilliamDAssafMSFT/20240820-sfi-images
20240820 sfi images
2 parents f1fa481 + 09f8dac commit 8b633cf

17 files changed

+42
-91
lines changed

articles/synapse-analytics/sql-data-warehouse/load-data-from-azure-blob-storage-using-copy.md

Lines changed: 34 additions & 65 deletions
Original file line numberDiff line numberDiff line change
@@ -4,13 +4,13 @@ description: Tutorial uses Azure portal and SQL Server Management Studio to load
44
author: joannapea
55
ms.author: joanpo
66
ms.reviewer: wiassaf
7-
ms.date: 11/23/2020
7+
ms.date: 08/20/2024
88
ms.service: azure-synapse-analytics
99
ms.subservice: sql-dw
1010
ms.topic: conceptual
11-
ms.custom: azure-synapse
11+
ms.custom:
12+
- azure-synapse
1213
---
13-
1414
# Tutorial: Load the New York Taxicab dataset
1515

1616
This tutorial uses the [COPY statement](/sql/t-sql/statements/copy-into-transact-sql?view=azure-sqldw-latest&preserve-view=true) to load New York Taxicab dataset from an Azure Blob Storage account. The tutorial uses the [Azure portal](https://portal.azure.com) and [SQL Server Management Studio (SSMS)](/sql/ssms/download-sql-server-management-studio-ssms?toc=/azure/synapse-analytics/sql-data-warehouse/toc.json&bc=/azure/synapse-analytics/sql-data-warehouse/breadcrumb/toc.json&view=azure-sqldw-latest&preserve-view=true) to:
@@ -28,72 +28,62 @@ If you don't have an Azure subscription, [create a free Azure account](https://a
2828

2929
Before you begin this tutorial, download and install the newest version of [SQL Server Management Studio (SSMS)](/sql/ssms/download-sql-server-management-studio-ssms?toc=/azure/synapse-analytics/sql-data-warehouse/toc.json&bc=/azure/synapse-analytics/sql-data-warehouse/breadcrumb/toc.json&view=azure-sqldw-latest&preserve-view=true).
3030

31-
This tutorial assumes you have already created a SQL dedicated pool from the following [tutorial](./create-data-warehouse-portal.md#connect-to-the-server-as-server-admin).
31+
This tutorial assumes you have already [created a SQL dedicated pool](./create-data-warehouse-portal.md#connect-to-the-server-as-server-admin).
3232

3333
## Create a user for loading data
3434

3535
The server admin account is meant to perform management operations, and is not suited for running queries on user data. Loading data is a memory-intensive operation. Memory maximums are defined according to the [data warehouse units](what-is-a-data-warehouse-unit-dwu-cdwu.md) and [resource class](resource-classes-for-workload-management.md) configured.
3636

3737
It's best to create a login and user that is dedicated for loading data. Then add the loading user to a [resource class](resource-classes-for-workload-management.md) that enables an appropriate maximum memory allocation.
3838

39-
Connect as the server admin so you can create logins and users. Use these steps to create a login and user called **LoaderRC20**. Then assign the user to the **staticrc20** resource class.
40-
41-
1. In SSMS, right-select **master** to show a drop-down menu, and choose **New Query**. A new query window opens.
39+
Connect as the server admin so you can create logins and users. Use these steps to create a login and user called `LoaderRC20`. Then assign the user to the `staticrc20` resource class.
4240

43-
![New query in master](./media/load-data-from-azure-blob-storage-using-polybase/create-loader-login.png)
41+
1. In SSMS, right-select `master` to show a dropdown menu, and choose **New Query**. A new query window opens.
4442

45-
2. In the query window, enter these T-SQL commands to create a login and user named LoaderRC20, substituting your own password for 'a123STRONGpassword!'.
43+
1. In the query window, enter these T-SQL commands to create a login and user named `LoaderRC20`, substituting your own strong password.
4644

4745
```sql
48-
CREATE LOGIN LoaderRC20 WITH PASSWORD = 'a123STRONGpassword!';
46+
CREATE LOGIN LoaderRC20 WITH PASSWORD = '<strong password here>';
4947
CREATE USER LoaderRC20 FOR LOGIN LoaderRC20;
5048
```
5149

52-
3. Select **Execute**.
50+
1. Select **Execute**.
5351

54-
4. Right-click **mySampleDataWarehouse**, and choose **New Query**. A new query Window opens.
52+
1. Right-click **mySampleDataWarehouse**, and choose **New Query**. A new query Window opens.
5553

56-
![New query on sample data warehouse](./media/load-data-from-azure-blob-storage-using-polybase/create-loading-user.png)
57-
58-
5. Enter the following T-SQL commands to create a database user named LoaderRC20 for the LoaderRC20 login. The second line grants the new user CONTROL permissions on the new data warehouse. These permissions are similar to making the user the owner of the database. The third line adds the new user as a member of the staticrc20 [resource class](resource-classes-for-workload-management.md).
54+
1. Enter the following T-SQL commands to create a database user named `LoaderRC20` for the `LoaderRC20` login. The second line grants the new user CONTROL permissions on the new data warehouse. These permissions are similar to making the user the owner of the database. The third line adds the new user as a member of the `staticrc20` [resource class](resource-classes-for-workload-management.md).
5955

6056
```sql
6157
CREATE USER LoaderRC20 FOR LOGIN LoaderRC20;
6258
GRANT CONTROL ON DATABASE::[mySampleDataWarehouse] to LoaderRC20;
6359
EXEC sp_addrolemember 'staticrc20', 'LoaderRC20';
6460
```
6561

66-
6. Select **Execute**.
62+
1. Select **Execute**.
6763

6864
## Connect to the server as the loading user
6965

70-
The first step toward loading data is to login as LoaderRC20.
71-
72-
1. In Object Explorer, select the **Connect** drop down menu and select **Database Engine**. The **Connect to Server** dialog box appears.
66+
The first step toward loading data is to login as `LoaderRC20`.
7367

74-
![Connect with new login](./media/load-data-from-azure-blob-storage-using-polybase/connect-as-loading-user.png)
68+
1. In Object Explorer, select the **Connect** dropdown menu and select **Database Engine**. The **Connect to Server** dialog box appears.
7569

76-
2. Enter the fully qualified server name, and enter **LoaderRC20** as the Login. Enter your password for LoaderRC20.
70+
1. Enter the fully qualified server name, and enter `LoaderRC20` as the Login. Enter your password for LoaderRC20.
7771

78-
3. Select **Connect**.
72+
1. Select **Connect**.
7973

80-
4. When your connection is ready, you will see two server connections in Object Explorer. One connection as ServerAdmin and one connection as LoaderRC20.
81-
82-
![Connection is successful](./media/load-data-from-azure-blob-storage-using-polybase/connected-as-new-login.png)
74+
1. When your connection is ready, you'll see two server connections in Object Explorer. One connection as ServerAdmin and one connection as LoaderRC20.
8375
8476
## Create tables for the sample data
8577
86-
You are ready to begin the process of loading data into your new data warehouse. This part of the tutorial shows you how to use the COPY statement to load the New York City taxi cab dataset from an Azure Storage blob. For future reference, to learn how to get your data to Azure Blob Storage or to load it directly from your source, see the [loading overview](design-elt-data-loading.md).
78+
You're ready to begin the process of loading data into your new data warehouse. This part of the tutorial shows you how to use the COPY statement to load the New York City taxi cab dataset from an Azure Storage blob. For future reference, to learn how to get your data to Azure Blob Storage or to load it directly from your source, see the [loading overview](design-elt-data-loading.md).
8779

8880
Run the following SQL scripts and specify information about the data you wish to load. This information includes where the data is located, the format of the contents of the data, and the table definition for the data.
8981

90-
1. In the previous section, you logged into your data warehouse as LoaderRC20. In SSMS, right-click your LoaderRC20 connection and select **New Query**. A new query window appears.
91-
92-
![New loading query window](./media/load-data-from-azure-blob-storage-using-polybase/new-loading-query.png)
82+
1. In the previous section, you logged into your data warehouse as `LoaderRC20`. In SSMS, right-click your LoaderRC20 connection and select **New Query**. A new query window appears.
9383

94-
2. Compare your query window to the previous image. Verify your new query window is running as LoaderRC20 and performing queries on your MySampleDataWarehouse database. Use this query window to perform all of the loading steps.
84+
1. Compare your query window to the previous image. Verify your new query window is running as `LoaderRC20` and performing queries on your `MySampleDataWarehouse` database. Use this query window to perform all of the loading steps.
9585

96-
7. Run the following T-SQL statements to create the tables:
86+
1. Run the following T-SQL statements to create the tables:
9787

9888
```sql
9989
CREATE TABLE [dbo].[Date]
@@ -327,7 +317,7 @@ This section uses the [COPY statement to load](/sql/t-sql/statements/copy-into-t
327317
OPTION (LABEL = 'COPY : Load [dbo].[Trip] - Taxi dataset');
328318
```
329319

330-
2. View your data as it loads. You're loading several GBs of data and compressing it into highly performant clustered columnstore indexes. Run the following query that uses a dynamic management views (DMVs) to show the status of the load.
320+
1. View your data as it loads. You're loading several GBs of data and compressing it into highly performant clustered columnstore indexes. Run the following query that uses a dynamic management views (DMVs) to show the status of the load.
331321
332322
```sql
333323
SELECT r.[request_id]
@@ -353,59 +343,38 @@ This section uses the [COPY statement to load](/sql/t-sql/statements/copy-into-t
353343
, r.command;
354344
```
355345
356-
3. View all system queries.
346+
1. View all system queries.
357347
358348
```sql
359349
SELECT * FROM sys.dm_pdw_exec_requests;
360350
```
361351
362-
4. Enjoy seeing your data nicely loaded into your data warehouse.
352+
1. Enjoy your data nicely loaded into your data warehouse.
363353
364-
![View loaded tables](./media/load-data-from-azure-blob-storage-using-polybase/view-loaded-tables.png)
365354
366355
## Clean up resources
367356
368357
You are being charged for compute resources and data that you loaded into your data warehouse. These are billed separately.
369358
370-
* If you want to keep the data in storage, you can pause compute when you aren't using the data warehouse. By pausing compute you will only be charge for data storage and you can resume the compute whenever you are ready to work with the data.
371-
* If you want to remove future charges, you can delete the data warehouse.
359+
- If you want to keep the data in storage, you can pause compute when you aren't using the data warehouse. By pausing compute, you will only be charge for data storage and you can resume the compute whenever you're ready to work with the data.
360+
- If you want to remove future charges, you can delete the data warehouse.
372361
373362
Follow these steps to clean up resources as you desire.
374363
375-
1. Log in to the [Azure portal](https://portal.azure.com), select your data warehouse.
376-
377-
![Clean up resources](./media/load-data-from-azure-blob-storage-using-polybase/clean-up-resources.png)
364+
1. Sign in to the [Azure portal](https://portal.azure.com), and select your data warehouse.
378365
379-
2. To pause compute, select the **Pause** button. When the data warehouse is paused, you will see a **Start** button. To resume compute, select **Start**.
366+
1. To pause compute, select the **Pause** button. When the data warehouse is paused, you see a **Start** button. To resume compute, select **Start**.
380367
381-
3. To remove the data warehouse so you won't be charged for compute or storage, select **Delete**.
382-
383-
4. To remove the server you created, select **mynewserver-20180430.database.windows.net** in the previous image, and then select **Delete**. Be careful with this as deleting the server will delete all databases assigned to the server.
384-
385-
5. To remove the resource group, select **myResourceGroup**, and then select **Delete resource group**.
386-
387-
## Next steps
388-
389-
In this tutorial, you learned how to create a data warehouse and create a user for loading data. You used the simple [COPY statement](/sql/t-sql/statements/copy-into-transact-sql?view=azure-sqldw-latest&preserve-view=true#examples) to load data into your data warehouse.
390-
391-
You did these things:
392-
> [!div class="checklist"]
393-
>
394-
> * Created a data warehouse in the Azure portal
395-
> * Set up a server-level firewall rule in the Azure portal
396-
> * Connected to the data warehouse with SSMS
397-
> * Created a user designated for loading data
398-
> * Created the tables for the sample data
399-
> * Used the COPY T-SQL statement to load data into your data warehouse
400-
> * Viewed the progress of data as it is loading
368+
1. To remove the data warehouse so you won't be charged for compute or storage, select **Delete**.
401369

402-
Advance to the development overview to learn how to migrate an existing database to Azure Synapse Analytics:
370+
1. To remove the server you created, select **mynewserver-20180430.database.windows.net** in the previous image, and then select **Delete**. Be careful with this as deleting the server deletes all databases assigned to the server.
403371

404-
> [!div class="nextstepaction"]
405-
> [Design decisions to migrate an existing database to Azure Synapse Analytics](sql-data-warehouse-overview-develop.md)
372+
1. To remove the resource group, select **myResourceGroup**, and then select **Delete resource group**.
406373

407-
For more loading examples and references, view the following documentation:
374+
## Related content
408375

376+
- [COPY statement](/sql/t-sql/statements/copy-into-transact-sql?view=azure-sqldw-latest&preserve-view=true#examples)
377+
- [Design decisions to migrate an existing database to Azure Synapse Analytics](sql-data-warehouse-overview-develop.md)
409378
- [COPY statement reference documentation](/sql/t-sql/statements/copy-into-transact-sql?view=azure-sqldw-latest&preserve-view=true#syntax)
410379
- [COPY examples for each authentication method](./quickstart-bulk-load-copy-tsql-examples.md)
411380
- [COPY quickstart for a single table](./quickstart-bulk-load-copy-tsql.md)
Loading
Binary file not shown.

0 commit comments

Comments
 (0)