Skip to content

Commit cd62442

Browse files
committed
adding etl docs
1 parent a171b75 commit cd62442

File tree

2 files changed

+56
-7
lines changed

2 files changed

+56
-7
lines changed
Lines changed: 54 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,58 @@
11
# WideWorldImporters ETL Workflow
22

3-
This folder contains documentation for the sample.
3+
The ETL package WWI_Integration is used to migrate data from the WideWorldImporters database to the WideWorldImportersDW database as the data changes. The package is run periodically (most commonly daily).
44

5-
Start with [root.md](root.md)
5+
## Overview
66

7-
Note that these contents will most likely be migrated to MSDN.
7+
The design of the package uses SQL Server Integration Services (SSIS) to orchestrate bulk T-SQL operations (rather than as separate transformations within SSIS) to ensure high performance.
8+
9+
Dimensions are loaded first, followed by Fact tables. The package can be re-run at any time after a failure.
10+
11+
The workflow is as follows:
12+
13+
![Alt text](/media/wide-world-importers-etl-workflow.png "WideWorldImporters ETL Workflow")
14+
15+
It starts with an expression task that works out the appropriate cutoff time. This time is the current time less a few seconds. (This is more robust than requesting data right to the current time). It then truncates any milliseconds from the time.
16+
17+
The main processing starts by populating the Date dimension table. It ensures that all dates for the current year have been populated in the table.
18+
19+
After this, a series of data flow tasks loads each dimension, then each fact.
20+
21+
## Prerequisites
22+
23+
- SQL Server 2016 (or higher) with the databases WideWorldImporters and WideWorldImportersDW. These can be on the same or different instances of SQL Server.
24+
- SQL Server Management Studio (SSMS)
25+
- SQL Server 2016 Integration Services (SSIS).
26+
- Make sure you have created an SSIS Catalog. If not, right click **Integration Services** in SSMS Object Explorer, and choose **Add Catalog**. Follow the defaults. It will ask you to enable sqlclr and provide a password.
27+
28+
29+
## Download
30+
31+
The latest release of the sample:
32+
33+
[wide-world-importers-v0.1](https://github.com/Microsoft/sql-server-samples/releases/tag/wide-world-importers-v0.1)
34+
35+
Download the SSIS package file **Daily ETL.ispac**.
36+
37+
Source code to recreate the sample database is available from the following location.
38+
39+
[wide-world-importers](https://github.com/Microsoft/sql-server-samples/tree/master/samples/databases/wide-world-importers/wwi-integration-etl)
40+
41+
## Install
42+
43+
1. Deploy the SSIS package.
44+
- Open the "Daily ETL.ispac" package from Windows Explorer. This will launch the Integration Services Deployment Wizard.
45+
- Under "Select Source" follow the default Project Deployment, with the path pointing to the "Daily ETL.ispac" package.
46+
- Under "Select Destination" enter the name of the server that hosts the SSIS catalog.
47+
- Select a path under the SSIS catalog, for example under a new folder "WideWorldImporters".
48+
- Finalize the wizard by clicking Deploy.
49+
50+
2. Create a SQL Server Agent job for the ETL process.
51+
- In SSMS, right-click "SQL Server Agent" and select New->Job.
52+
- Pick a name, for example "WideWorldImporters ETL".
53+
- Add a Job Step of type "SQL Server Integration Services Package".
54+
- Select the server with the SSIS catalog, and select the "Daily ETL" package.
55+
- Under Configuration->Connection Managers ensure the connections to the source and target are configured correctly. The default is to connect to the local instance.
56+
- Click OK to create the Job.
57+
58+
3. Execute or schedule the Job.
Lines changed: 2 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,5 @@
11
# WideWorldImportersDW OLAP Sample Queries
22

3-
This folder contains documentation for the sample.
3+
Refer to the sample-scripts.zip file that is included with the release of the sample, or refer to the source code:
44

5-
Start with [root.md](root.md)
6-
7-
Note that these contents will most likely be migrated to MSDN.
5+
[wide-world-importers/sample-scripts](https://github.com/Microsoft/sql-server-samples/tree/master/samples/databases/wide-world-importers/sample-scripts)

0 commit comments

Comments
 (0)