|
| 1 | +# SQL Ultimate Course |
1 | 2 |
|
2 | | -# Data Warehouse and Analytics Project |
| 3 | +Welcome to the **SQL Ultimate Course** repository! 🚀 |
| 4 | +This course will cover everything you need to master SQL! |
3 | 5 |
|
4 | | -Welcome to the **Data Warehouse and Analytics Project** repository! 🚀 |
5 | | -This project demonstrates a comprehensive data warehousing and analytics solution, from building a data warehouse to generating actionable insights. Designed as a portfolio project, it highlights industry best practices in data engineering and analytics. |
6 | | - |
7 | | ---- |
8 | | -## 🏗️ Data Architecture |
9 | | - |
10 | | -The data architecture for this project follows Medallion Architecture **Bronze**, **Silver**, and **Gold** layers: |
11 | | - |
12 | | - |
13 | | -1. **Bronze Layer**: Stores raw data as-is from the source systems. Data is ingested from CSV Files into SQL Server Database. |
14 | | -2. **Silver Layer**: This layer includes data cleansing, standardization, and normalization processes to prepare data for analysis. |
15 | | -3. **Gold Layer**: Houses business-ready data modeled into a star schema required for reporting and analytics. |
16 | | - |
17 | | ---- |
18 | | -## 📖 Project Overview |
19 | | - |
20 | | -This project involves: |
21 | | - |
22 | | -1. **Data Architecture**: Designing a Modern Data Warehouse Using Medallion Architecture **Bronze**, **Silver**, and **Gold** layers. |
23 | | -2. **ETL Pipelines**: Extracting, transforming, and loading data from source systems into the warehouse. |
24 | | -3. **Data Modeling**: Developing fact and dimension tables optimized for analytical queries. |
25 | | -4. **Analytics & Reporting**: Creating SQL-based reports and dashboards for actionable insights. |
26 | | - |
27 | | -🎯 This repository is an excellent resource for professionals and students looking to showcase expertise in: |
28 | | -- SQL Development |
29 | | -- Data Architect |
30 | | -- Data Engineering |
31 | | -- ETL Pipeline Developer |
32 | | -- Data Modeling |
33 | | -- Data Analytics |
34 | 6 |
|
35 | 7 | --- |
36 | 8 |
|
37 | 9 | ## 🛠️ Important Links & Tools: |
38 | 10 |
|
39 | 11 | Everything is for Free! |
40 | | -- **[Datasets](datasets/):** Access to the project dataset (csv files). |
| 12 | +- **[Datasets](datasets/):** Access to the course dataset (csv files). |
41 | 13 | - **[SQL Server Express](https://www.microsoft.com/en-us/sql-server/sql-server-downloads):** Lightweight server for hosting your SQL database. |
42 | 14 | - **[SQL Server Management Studio (SSMS)](https://learn.microsoft.com/en-us/sql/ssms/download-sql-server-management-studio-ssms?view=sql-server-ver16):** GUI for managing and interacting with databases. |
43 | | -- **[Git Repository](https://github.com/):** Set up a GitHub account and repository to manage, version, and collaborate on your code efficiently. |
44 | | -- **[DrawIO](https://www.drawio.com/):** Design data architecture, models, flows, and diagrams. |
45 | | -- **[Notion](https://www.notion.com/):** All-in-one tool for project management and organization. |
46 | | -- **[Notion Project Steps](https://thankful-pangolin-2ca.notion.site/SQL-Data-Warehouse-Project-16ed041640ef80489667cfe2f380b269?pvs=4):** Access to All Project Phases and Tasks. |
47 | 15 |
|
48 | 16 | --- |
49 | 17 |
|
50 | | -## 🚀 Project Requirements |
51 | | - |
52 | | -### Building the Data Warehouse (Data Engineering) |
53 | | - |
54 | | -#### Objective |
55 | | -Develop a modern data warehouse using SQL Server to consolidate sales data, enabling analytical reporting and informed decision-making. |
56 | | - |
57 | | -#### Specifications |
58 | | -- **Data Sources**: Import data from two source systems (ERP and CRM) provided as CSV files. |
59 | | -- **Data Quality**: Cleanse and resolve data quality issues prior to analysis. |
60 | | -- **Integration**: Combine both sources into a single, user-friendly data model designed for analytical queries. |
61 | | -- **Scope**: Focus on the latest dataset only; historization of data is not required. |
62 | | -- **Documentation**: Provide clear documentation of the data model to support both business stakeholders and analytics teams. |
63 | | - |
64 | | ---- |
65 | | - |
66 | | -### BI: Analytics & Reporting (Data Analysis) |
67 | | - |
68 | | -#### Objective |
69 | | -Develop SQL-based analytics to deliver detailed insights into: |
70 | | -- **Customer Behavior** |
71 | | -- **Product Performance** |
72 | | -- **Sales Trends** |
73 | | - |
74 | | -These insights empower stakeholders with key business metrics, enabling strategic decision-making. |
75 | | - |
76 | | -For more details, refer to [docs/requirements.md](docs/requirements.md). |
77 | | - |
78 | 18 | ## 📂 Repository Structure |
79 | 19 | ``` |
80 | | -data-warehouse-project/ |
81 | | -│ |
82 | | -├── datasets/ # Raw datasets used for the project (ERP and CRM data) |
83 | | -│ |
84 | | -├── docs/ # Project documentation and architecture details |
85 | | -│ ├── etl.drawio # Draw.io file shows all different techniquies and methods of ETL |
86 | | -│ ├── data_architecture.drawio # Draw.io file shows the project's architecture |
87 | | -│ ├── data_catalog.md # Catalog of datasets, including field descriptions and metadata |
88 | | -│ ├── data_flow.drawio # Draw.io file for the data flow diagram |
89 | | -│ ├── data_models.drawio # Draw.io file for data models (star schema) |
90 | | -│ ├── naming-conventions.md # Consistent naming guidelines for tables, columns, and files |
| 20 | +sql-ultimate-course/ |
91 | 21 | │ |
92 | | -├── scripts/ # SQL scripts for ETL and transformations |
93 | | -│ ├── bronze/ # Scripts for extracting and loading raw data |
94 | | -│ ├── silver/ # Scripts for cleaning and transforming data |
95 | | -│ ├── gold/ # Scripts for creating analytical models |
| 22 | +├── datasets/ # Raw datasets used for the course (ERP and CRM data) |
96 | 23 | │ |
97 | | -├── tests/ # Test scripts and quality files |
| 24 | +├── docs/ # course documentation and architecture details |
98 | 25 | │ |
99 | | -├── README.md # Project overview and instructions |
100 | | -├── LICENSE # License information for the repository |
101 | | -├── .gitignore # Files and directories to be ignored by Git |
102 | | -└── requirements.txt # Dependencies and requirements for the project |
| 26 | +└── scripts/ # All codes presented in the course |
103 | 27 | ``` |
104 | 28 | --- |
105 | 29 |
|
106 | 30 |
|
107 | 31 | ## 🛡️ License |
108 | 32 |
|
109 | | -This project is licensed under the [MIT License](LICENSE). You are free to use, modify, and share this project with proper attribution. |
| 33 | +This course is licensed under the [MIT License](LICENSE). You are free to use, modify, and share this project with proper attribution. |
110 | 34 |
|
111 | 35 | ## 🌟 About Me |
112 | 36 |
|
|
0 commit comments