|
1 | 1 | # WideWorldImporters ETL Workflow
|
2 | 2 |
|
3 |
| -This folder contains documentation for the sample. |
| 3 | +The ETL package WWI_Integration is used to migrate data from the WideWorldImporters database to the WideWorldImportersDW database as the data changes. The package is run periodically (most commonly daily). |
4 | 4 |
|
5 |
| -Start with [root.md](root.md) |
| 5 | +## Overview |
6 | 6 |
|
7 |
| -Note that these contents will most likely be migrated to MSDN. |
| 7 | +The design of the package uses SQL Server Integration Services (SSIS) to orchestrate bulk T-SQL operations (rather than as separate transformations within SSIS) to ensure high performance. |
| 8 | + |
| 9 | +Dimensions are loaded first, followed by Fact tables. The package can be re-run at any time after a failure. |
| 10 | + |
| 11 | +The workflow is as follows: |
| 12 | + |
| 13 | + |
| 14 | + |
| 15 | +It starts with an expression task that works out the appropriate cutoff time. This time is the current time less a few seconds. (This is more robust than requesting data right to the current time). It then truncates any milliseconds from the time. |
| 16 | + |
| 17 | +The main processing starts by populating the Date dimension table. It ensures that all dates for the current year have been populated in the table. |
| 18 | + |
| 19 | +After this, a series of data flow tasks loads each dimension, then each fact. |
| 20 | + |
| 21 | +## Prerequisites |
| 22 | + |
| 23 | +- SQL Server 2016 (or higher) with the databases WideWorldImporters and WideWorldImportersDW. These can be on the same or different instances of SQL Server. |
| 24 | +- SQL Server Management Studio (SSMS) |
| 25 | +- SQL Server 2016 Integration Services (SSIS). |
| 26 | + - Make sure you have created an SSIS Catalog. If not, right click **Integration Services** in SSMS Object Explorer, and choose **Add Catalog**. Follow the defaults. It will ask you to enable sqlclr and provide a password. |
| 27 | + |
| 28 | + |
| 29 | +## Download |
| 30 | + |
| 31 | +The latest release of the sample: |
| 32 | + |
| 33 | +[wide-world-importers-v0.1](https://github.com/Microsoft/sql-server-samples/releases/tag/wide-world-importers-v0.1) |
| 34 | + |
| 35 | +Download the SSIS package file **Daily ETL.ispac**. |
| 36 | + |
| 37 | +Source code to recreate the sample database is available from the following location. |
| 38 | + |
| 39 | +[wide-world-importers](https://github.com/Microsoft/sql-server-samples/tree/master/samples/databases/wide-world-importers/wwi-integration-etl) |
| 40 | + |
| 41 | +## Install |
| 42 | + |
| 43 | +1. Deploy the SSIS package. |
| 44 | + - Open the "Daily ETL.ispac" package from Windows Explorer. This will launch the Integration Services Deployment Wizard. |
| 45 | + - Under "Select Source" follow the default Project Deployment, with the path pointing to the "Daily ETL.ispac" package. |
| 46 | + - Under "Select Destination" enter the name of the server that hosts the SSIS catalog. |
| 47 | + - Select a path under the SSIS catalog, for example under a new folder "WideWorldImporters". |
| 48 | + - Finalize the wizard by clicking Deploy. |
| 49 | + |
| 50 | +2. Create a SQL Server Agent job for the ETL process. |
| 51 | + - In SSMS, right-click "SQL Server Agent" and select New->Job. |
| 52 | + - Pick a name, for example "WideWorldImporters ETL". |
| 53 | + - Add a Job Step of type "SQL Server Integration Services Package". |
| 54 | + - Select the server with the SSIS catalog, and select the "Daily ETL" package. |
| 55 | + - Under Configuration->Connection Managers ensure the connections to the source and target are configured correctly. The default is to connect to the local instance. |
| 56 | + - Click OK to create the Job. |
| 57 | + |
| 58 | +3. Execute or schedule the Job. |
0 commit comments