Skip to content

About

Designed and implemented a modern Enterprise Data Warehouse using Microsoft SQL Server, encompassing structured ETL processes, dimensional data modelling, and analytics enablement.

Topics

Resources

Stars

0 stars

Watchers

0 watching

Forks

Latest commit

 

History

71 Commits

Folders and files

NameName
Last commit message
Last commit date
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

🗄️ Data Warehouse and Analytics Project

Welcome to the Data Warehouse and Analytics Project repository!🚀

This project presents a comprehensive, end-to-end data warehousing and analytics solution — covering the complete lifecycle from data ingestion and transformation to analytical modeling and insight generation. It is designed as a portfolio project to demonstrate practical implementation of industry best practices in data engineering, data modeling, and business analytics.

📖 Project Overview

This project involves:

  1. Data Architecture: Designing a Modern Data Warehouse Using Medallion Architecture Bronze, Silver, and Gold layers.
  2. ETL Pipelines: Extracting, transforming, and loading data from source systems into the warehouse.
  3. Data Modeling: Developing fact and dimension tables optimized for analytical queries.
  4. Analytics & Reporting: Creating SQL-based reports and dashboards for actionable insights.

🎯 This repository is an excellent resource for professionals and students looking to showcase expertise in:

  • SQL Development
  • Data Architect
  • Data Engineering
  • ETL Pipeline Developer
  • Data Modeling
  • Data Analytics

🚀 Project Requirements

🛠️ Data Warehouse Development (Data Engineering)

Objective

Design and implement a modern data warehouse using SQL Server to consolidate sales data and enable reliable analytical reporting and data-driven decision-making.

Specifications

  • Data Ingestion: Extract and load data from ERP and CRM source systems provided in CSV format.
  • Data Quality Management: Perform data cleansing and validation to address inconsistencies and ensure data accuracy.
  • Data Integration & Modeling: Integrate both data sources into a unified, analysis-ready data model optimized for querying and reporting.
  • Project Scope: Focus exclusively on the latest available dataset; historical data tracking is out of scope.
  • Documentation: Produce comprehensive documentation of the data model to support both business stakeholders and technical teams.

📊 BI: Analytics & Reporting (Data Analysis)

Objective

Design and implement SQL-based analytical solutions to generate meaningful business insights across key performance domains, including:

  • Customer Insights
  • Product Analytics
  • Sales Growth Metrics

These analytics deliver actionable insights and critical business metrics, equipping stakeholders with the information required to support strategic planning and data-driven decision-making.


🏗️ Data Architecture

The data architecture for this project follows Medallion Architecture Bronze, Silver, and Gold layers:

Data Architecture

  1. Bronze Layer: Stores data in its original form as received from source systems. Data is ingested from CSV files into the SQL Server database without any transformation.
  2. Silver Layer: This layer includes data cleansing, standardization, and normalization processes to prepare data for analysis.
  3. Gold Layer: Houses business-ready data modeled into a star schema required for reporting and analytics.

📂 Repository Structure

data-warehouse-project/
│
├── datasets/                           # Raw datasets used for the project (ERP and CRM data)
│
├── docs/                               # Project documentation and architecture details
│   ├── etl.drawio                      # Draw.io file shows all different techniquies and methods of ETL
│   ├── Data_Architecture.drawio        # Draw.io file shows the project's architecture
│   ├── data_catalog.md                 # Catalog of datasets, including field descriptions and metadata
│   ├── DATA_Flow.drawio                # Draw.io file for the data flow diagram
│   ├── DATA (Star Schema).drawio              # Draw.io file for data models (star schema)
│   ├── naming-conventions.md           # Consistent naming guidelines for tables, columns, and files
│
├── scripts/                            # SQL scripts for ETL and transformations
│   ├── Bronze/                         # Scripts for extracting and loading raw data
│   ├── Silver/                         # Scripts for cleaning and transforming data
│   ├── Gold/                           # Scripts for creating analytical models
│
├── tests/                              # Test scripts and quality files
│
├── README.md                           # Project overview and instructions
├── LICENSE                             # License information for the repository
├── .gitignore                          # Files and directories to be ignored by Git
└── requirements.txt                    # Dependencies and requirements for the project

🛡️ License

This project is licensed under the MIT License. You are free to use, modify, and share this project with proper attribution.

🔎 About Me

Hi, I’m Kshitij Gudekar — a former Data Analyst expanding my expertise in Data Engineering.

My professional experience includes delivering analytical insights and building data-driven solutions. I am currently focused on designing scalable data warehouses, optimizing ETL workflows, and implementing reliable data architectures for effective decision-making.

About

Designed and implemented a modern Enterprise Data Warehouse using Microsoft SQL Server, encompassing structured ETL processes, dimensional data modelling, and analytics enablement.

Topics

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages