Skip to content

Commit 95b3e50

Browse files
jodebruijodebrui
committed
Merge pull request #42 from Microsoft/jodebrui
adding documentation draft
2 parents c497359 + 19fa461 commit 95b3e50

File tree

23 files changed

+1149
-34
lines changed

23 files changed

+1149
-34
lines changed

‎samples/databases/wide-world-importers/README.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
# WideWorldImporters Sample Database for SQL Server and Azure SQL Database
22

3-
WideWorldImporters is a sample for SQL Server and Azure SQL Database. It showcases best practices in database design, as well as how to best leverage SQL Server features in a database.
3+
WideWorldImporters is a sample for SQL Server and Azure SQL Database. It showcases database design, as well as how to best leverage SQL Server features in a database.
44

55
WideWorldImporters is a wholesale company. Transactions and real-time analytics are performed in the database WideWorldImporters. The database WideWorldImportersDW is an OLAP database, focused on analytics.
66

‎samples/databases/wide-world-importers/documentation/README.md‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
# Documentation for the WideWorldImporters Sample Database
2+
3+
This folder contains documentation for the sample.
4+
5+
Start with [root.md](root.md)
6+
7+
Note that these contents will most likely be migrated to MSDN.

‎samples/databases/wide-world-importers/documentation/root.md‎

Lines changed: 65 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,65 @@
1+
# Wide World Importers Sample for SQL Server and Azure SQL Database
2+
3+
Wide World Importers is a comprehensive database sample that both illustrates database design, and illustrates how SQL Server features can be leveraged in an application.
4+
5+
Note that the sample is meant to be representative of a typical database. It does not include every feature of SQL Server. The design of the database follows one common set of standards, but there are many ways one might build a database.
6+
7+
The source code for the sample can be found on the SQL Server Samples GitHub repository:
8+
[wide-world-importers](https://github.com/Microsoft/sql-server-samples/tree/master/samples/databases/wide-world-importers).
9+
10+
The latest released version of the sample:
11+
[wide-world-importers-v0.1](https://github.com/Microsoft/sql-server-samples/releases/tag/wide-world-importers-v0.1)
12+
13+
The documentation for the sample is organized as follows:
14+
15+
## Overview
16+
17+
__[Wide World Importers Overview](wwi-overview.md)__
18+
19+
Overview of the sample company Wide World Importers, and the workflows addressed by the sample.
20+
21+
## Main OLTP Database WideWorldImporters
22+
23+
__[WideWorldImporters Installation and Configuration](wwi-oltp-htap-installation.md)__
24+
25+
Instructions for the installation and configuration of the core database WideWorldImporters that is used for transaction processing (OLTP - OnLine Transaction Processing) and operational analytics (HTAP - Hybrid Transactional/Analytical Processing).
26+
27+
__[WideWorldImporters Database Catalog](wwi-oltp-htap-catalog.md)__
28+
29+
Description of the schemas and tables used in the WideWorldImporters database.
30+
31+
__[WideWorldImporters Use of SQL Server Features and Capabilities](wwi-oltp-htap-sql-features.md)__
32+
33+
Describes how WideWorldImporters leverages core SQL Server features.
34+
35+
__[WideWorldImporters Sample Queries](wwi-oltp-htap-sample-queries.md)__
36+
37+
Sample queries for the WideWorldImporters database.
38+
39+
## Data Warehousing and Analytics Database WideWorldImportersDW
40+
41+
__[WideWorldImportersDW Installation and Configuration](wwi-olap-installation.md)__
42+
43+
Instructions for the installation and configuration of the OLAP database WideWorldImportersDW.
44+
45+
__[WideWorldImportersDW OLAP Database Catalog](wwi-olap-catalog.md)__
46+
47+
Description of the schemas and tables used in the WideWorldImportersDW database, which is the sample database for data warehousing and analytics processing (OLAP).
48+
49+
__[WideWorldImporters ETL Workflow](wwi-etl.md)__
50+
51+
Workflow for the ETL (Extract, Transform, Load) process that migrates data from the transactional database WideWorldImporters to the data warehouse WideWorldImportersDW.
52+
53+
__[WideWorldImportersDW Use of SQL Server Features and Capabilities](wwi-olap-sql-features.md)__
54+
55+
Describes how the WideWorldImportersDW leverages SQL Server features for analytics processing.
56+
57+
__[WideWorldImportersDW OLAP Sample Queries](wwi-olap-sample-queries.md)__
58+
59+
Sample analytics queries leveraging the WideWorldImportersDW database.
60+
61+
## Data generation
62+
63+
__[WideWorldImporters Data Generation](wwi-data-generation.md)__
64+
65+
Describes how additional data can be generated in the sample database, for example inserting sales and purchase data up to the current date.

‎samples/databases/wide-world-importers/documentation/wwi-data-generation.md‎

Lines changed: 35 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,35 @@
1+
# WideWorldImporters Data Generation
2+
3+
The released versions of the WideWorldImporters and WideWorldImportersDW databases contains data starting January 1st 2013, up to the day these databases were generated.
4+
5+
If the sample databases are used at a later date, for demonstration or illustration purposes, it may be beneficial to include more recent sample data in the database.
6+
7+
## Data Generation in WideWorldImporters
8+
9+
To generate sample data up to the current date, follow these steps:
10+
11+
1. If you have not yet done so, install a clean version of the WideWorldImporters database. For installation instructions, [WideWorldImporters Installation and Configuration](wwi-oltp-htap-installation.md).
12+
2. Execute the following statement in the database:
13+
14+
EXEC WideWorldImporters.DataLoadSimulation.PopulateDataToCurrentDate
15+
@AverageNumberOfCustomerOrdersPerDay = 60,
16+
@SaturdayPercentageOfNormalWorkDay = 50,
17+
@SundayPercentageOfNormalWorkDay = 0,
18+
@IsSilentMode = 1,
19+
@AreDatesPrinted = 1;
20+
21+
This statement adds sample sales and purchase data in the database, up to the current date. It outputs the progress of the data generation day-by-day. It will take rougly 10 minutes for every year that needs data. Note that there are some differences in the data generated between runs, since there is a random factor in the data generation.
22+
23+
To increase or decrease the amount of data generated, in terms of orders per day, change the value for the parameter `@AverageNumberOfCustomerOrdersPerDay`. The parameters `@SaturdayPercentageOfNormalWorkDay` and `@SundayPercentageOfNormalWorkDay` are used to determine the order volume for weekend days.
24+
25+
## Importing Data in WideWorldImportersDW
26+
27+
To import sample data up to the current date in the OLAP database WideWorldImportersDW, follow these steps:
28+
29+
1. Execute the data generation logic in the WideWorldImporters OLTP database, using the steps above.
30+
2. If you have not yet done so, install a clean version of the WideWorldImportersDW database. For installation instructions, [WideWorldImporters Installation and Configuration](wwi-olap-installation.md).
31+
3. Reseed the OLAP database by executing the following statement in the database:
32+
33+
EXECUTE [Application].Configuration_ReseedETL
34+
35+
4. Run the SSIS package **Daily ETL.ispac** to import the data into the OLAP database. For instructions on how to run the ETL job, see [WideWorldImporters ETL Workflow](wwi-etl.md).

‎samples/databases/wide-world-importers/documentation/wwi-etl.md‎

Lines changed: 58 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,58 @@
1+
# WideWorldImporters ETL Workflow
2+
3+
The ETL package WWI_Integration is used to migrate data from the WideWorldImporters database to the WideWorldImportersDW database as the data changes. The package is run periodically (most commonly daily).
4+
5+
## Overview
6+
7+
The design of the package uses SQL Server Integration Services (SSIS) to orchestrate bulk T-SQL operations (rather than as separate transformations within SSIS) to ensure high performance.
8+
9+
Dimensions are loaded first, followed by Fact tables. The package can be re-run at any time after a failure.
10+
11+
The workflow is as follows:
12+
13+
![Alt text](/media/wide-world-importers-etl-workflow.png "WideWorldImporters ETL Workflow")
14+
15+
It starts with an expression task that works out the appropriate cutoff time. This time is the current time less a few seconds. (This is more robust than requesting data right to the current time). It then truncates any milliseconds from the time.
16+
17+
The main processing starts by populating the Date dimension table. It ensures that all dates for the current year have been populated in the table.
18+
19+
After this, a series of data flow tasks loads each dimension, then each fact.
20+
21+
## Prerequisites
22+
23+
- SQL Server 2016 (or higher) with the databases WideWorldImporters and WideWorldImportersDW. These can be on the same or different instances of SQL Server.
24+
- SQL Server Management Studio (SSMS)
25+
- SQL Server 2016 Integration Services (SSIS).
26+
- Make sure you have created an SSIS Catalog. If not, right click **Integration Services** in SSMS Object Explorer, and choose **Add Catalog**. Follow the defaults. It will ask you to enable sqlclr and provide a password.
27+
28+
29+
## Download
30+
31+
The latest release of the sample:
32+
33+
[wide-world-importers-v0.1](https://github.com/Microsoft/sql-server-samples/releases/tag/wide-world-importers-v0.1)
34+
35+
Download the SSIS package file **Daily ETL.ispac**.
36+
37+
Source code to recreate the sample database is available from the following location.
38+
39+
[wide-world-importers](https://github.com/Microsoft/sql-server-samples/tree/master/samples/databases/wide-world-importers/wwi-integration-etl)
40+
41+
## Install
42+
43+
1. Deploy the SSIS package.
44+
- Open the "Daily ETL.ispac" package from Windows Explorer. This will launch the Integration Services Deployment Wizard.
45+
- Under "Select Source" follow the default Project Deployment, with the path pointing to the "Daily ETL.ispac" package.
46+
- Under "Select Destination" enter the name of the server that hosts the SSIS catalog.
47+
- Select a path under the SSIS catalog, for example under a new folder "WideWorldImporters".
48+
- Finalize the wizard by clicking Deploy.
49+
50+
2. Create a SQL Server Agent job for the ETL process.
51+
- In SSMS, right-click "SQL Server Agent" and select New->Job.
52+
- Pick a name, for example "WideWorldImporters ETL".
53+
- Add a Job Step of type "SQL Server Integration Services Package".
54+
- Select the server with the SSIS catalog, and select the "Daily ETL" package.
55+
- Under Configuration->Connection Managers ensure the connections to the source and target are configured correctly. The default is to connect to the local instance.
56+
- Click OK to create the Job.
57+
58+
3. Execute or schedule the Job.

‎samples/databases/wide-world-importers/documentation/wwi-olap-catalog.md‎

Lines changed: 47 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,47 @@
1+
# WideWorldImportersDW OLAP Database Catalog
2+
3+
The WideWorldImportersDW database is used for data warehousing and analytical processing. The transactional data about sales and purchases is generated in the WideWorldImporters database, and loaded into the WideWorldImportersDW database using a [daily ETL process](wwi-etl.md).
4+
5+
The data in WideWorldImportersDW thus mirrors the data in WideWorldImporters, but the tables are organized differently. While WideWorldImporters has a traditional normalized schema, WideWorldImportersDW uses the [star schema](https://wikipedia.org/wiki/Star_schema) approach for its table design. Besides the fact and dimension tables, the database includes a number of staging tables that are used in the ETL process.
6+
7+
## Schemas
8+
9+
The different types of tables are organized in three schemas.
10+
11+
|Schema|Description|
12+
|-----------------------------|---------------------|
13+
|Dimension|Dimension tables.|
14+
|Fact|Fact tables.|
15+
|Integration|Staging tables and other objects needed for ETL.|
16+
17+
## Tables
18+
19+
The dimension and fact tables are listed below. The tables in the Integration schema are used only for the ETL process, and are not listed.
20+
21+
### Dimension tables
22+
23+
WideWorldImportersDW has the following dimension tables. The description includes the relationship with the source tables in the WideWorldImporters database.
24+
25+
|Table|Source tables|
26+
|-----------------------------|---------------------|
27+
|City|`Application.Cities`, `Application.StateProvinces`, `Application.Countries`.|
28+
|Customer|`Sales.Customers`, `Sales.BuyingGroups`, `Sales.CustomerCategories`.|
29+
|Date|New table with information about dates, including financial year (based on November 1st start for financial year).|
30+
|Employee|`Application.People`.|
31+
|StockItem|`Warehouse.StockItems`, `Warehouse.Colors`, `Warehouse.PackageType`.|
32+
|Supplier|`Purchasing.Suppliers`, `Purchasing.SupplierCategories`.|
33+
|PaymentMethod|`Application.PaymentMethods`.|
34+
|TransactionType|`Application.TransactionTypes`.|
35+
36+
### Fact tables
37+
38+
WideWorldImportersDW has the following dimension tables. The description includes the relationship with the source tables in the WideWorldImporters database, as well as the classes of analytics/reporting queries each fact table is typically used with.
39+
40+
|Table|Source tables|Sample Analytics|
41+
|-----------------------------|---------------------|
42+
|Order|`Sales.Orders` and `Sales.OrderLines`|Sales people, picker/packer productivity, and on time to pick orders. In addition, low stock situations leading to back orders.|
43+
|Sale|`Sales.Invoices` and `Sales.InvoiceLines`|Sales dates, delivery dates, profitability over time, profitability by sales person.|
44+
|Purchase|`Purchasing.PurchaseOrderLines`|Expected vs actual lead times|
45+
|Transaction|`Sales.CustomerTransactions` and `Purchasing.SupplierTransactions`|Measuring issue dates vs finalization dates, and amounts.|
46+
|Movement|`Warehouse.StockTransactions`|Movements over time.|
47+
|Stock Holding|`Warehouse.StockItemHoldings`|On-hand stock levels and value|

‎samples/databases/wide-world-importers/documentation/wwi-olap-installation.md‎

Lines changed: 50 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,50 @@
1+
# WideWorldImportersDW Installation and Configuration
2+
3+
4+
- [SQL Server 2016](https://www.microsoft.com/en-us/evalcenter/evaluate-sql-server-2016) (or higher) or [Azure SQL Database](https://azure.microsoft.com/services/sql-database/). To use the Full version of the sample, use SQL Server Evaluation/Developer/Enterprise Edition.
5+
- [SQL Server Management Studio](https://msdn.microsoft.com/library/mt238290.aspx). For the best results use the April 2016 preview or later.
6+
7+
## Download
8+
9+
The latest release of the sample:
10+
11+
[wide-world-importers-v0.1](https://github.com/Microsoft/sql-server-samples/releases/tag/wide-world-importers-v0.1)
12+
13+
Download the sample WideWorldImportersDW database backup/bacpac that corresponds to your edition of SQL Server or Azure SQL Database.
14+
15+
Source code to recreate the sample database is available from the following location. Note that data population is based on ETL from the OLTP database (WideWorldImporters):
16+
17+
[wide-world-importers](https://github.com/Microsoft/sql-server-samples/tree/master/samples/databases/wide-world-importers/wwi-dw-database-scripts)
18+
19+
## Install
20+
21+
22+
### SQL Server
23+
24+
To restore a backup to a SQL Server instance, you can use Management Studio.
25+
1. Open SQL Server Management Studio and connect to the target SQL Server instance.
26+
2. Right-click on the **Databases** node, and select **Restore Database**.
27+
3. Select **Device** and click on the button **...**
28+
4. In the dialog **Select backup devices**, click **Add**, navigate to the database backup in the filesystem of the server, and select the backup. Click **OK**.
29+
5. If needed, change the target location for the data and log files, in the **Files** pane. Note that it is best practice to place data and log files on different drives.
30+
6. Click **OK**. This will initiate the database restore. After it completes, you will have the database WideWorldImporters installed on your SQL Server instance.
31+
32+
### Azure SQL Database
33+
34+
To import a bacpac into a new SQL Database, you can use Management Studio.
35+
1. (optional) If you do not yet have a SQL Server in Azure, navigate to the [Azure portal](https://portal.azure.com/) and create a new SQL Database. In the process of create a database, you will create a server. Make note of the server.
36+
- See [this tutorial](https://azure.microsoft.com/documentation/articles/sql-database-get-started/) to create a database in minutes
37+
2. Open SQL Server Management Studio and connect to your server in Azure.
38+
3. Right-click on the **Databases** node, and select **Import Data-Tier Application**.
39+
4. In the **Import Settings** select **Import from local disk** and select the bacpac of the sample database from your file system.
40+
5. Under **Database Settings** change the database name to *WideWorldImportersDW* and select the target edition and service objective to use.
41+
6. Click **Next** and **Finish** to kick off deployment. It will take a few minutes to complete. When specifying a service objective lower than S2 it may take longer.
42+
43+
## Configuration
44+
45+
The sample database can make use of PolyBase to query files in Hadoop or Azure blob storage. However, that feature is not installed by default with SQL Server - you need to select it during SQL Server setup. Therefore, a post-installation step is required.
46+
47+
1. In SQL Server Management Studio, connect to the WideWorldImportersDW database and open a new query window.
48+
2. Run the following T-SQL command to enable the use of PolyBase in the database:
49+
50+
EXECUTE [Application].[Configuration_ApplyPolyBase]

‎samples/databases/wide-world-importers/documentation/wwi-olap-sample-queries.md‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
# WideWorldImportersDW OLAP Sample Queries
2+
3+
Refer to the sample-scripts.zip file that is included with the release of the sample, or refer to the source code:
4+
5+
[wide-world-importers/sample-scripts](https://github.com/Microsoft/sql-server-samples/tree/master/samples/databases/wide-world-importers/sample-scripts)

0 commit comments

Comments
 (0)