Skip to content

README.md

Latest commit

 

History

History
77 lines (59 loc) · 2.09 KB

README.md

File metadata and controls

77 lines (59 loc) · 2.09 KB

📊 Udemy Course Analysis with ELT Stack

This project analyzes 98,000+ Udemy courses using the ELT stack (Elasticsearch, Logstash, Kibana), orchestrated with Docker. The goal is to gain insights into online courses by performing data wrangling, querying, and visualization.

🚀 Tech Stack

  • Elasticsearch → Data indexing & queries
  • Logstash → Data extraction & transformation
  • Kibana → Interactive dashboards & visualizations
  • Docker → Containerized setup for easy deployment

📂 Project Structure

📂 Other files/
   ├── docker-compose.yaml      # ELK Stack container setup
   ├── index.json               # Elasticsearch index mapping
   ├── logstash/                # Logstash configuration files
       ├── logstash.conf         # Logstash configuration file
   ├── kibanaDashboard.ndjson   # Kibana dashboard import file

🛠️ Setup & Usage

1️⃣ Start Elasticsearch and Kibana

cd "Other files"
docker-compose up es01 kibana

2️⃣ Create the Index in Elasticsearch

curl -u elastic:admin -X PUT "localhost:9200/udemy_courses"

3️⃣ Apply the Index Mapping

curl -u elastic:admin -X PUT "localhost:9200/udemy_courses/_mapping" \
    -H "Content-Type: application/json" -d @index.json

4️⃣ Start Logstash

docker-compose up -d logstash

5️⃣ Import CSV Data

cp /path/to/dataset.csv logstash/csvData/

6️⃣ Restart Logstash to Load Data

docker-compose restart logstash

📊 Example Queries

  • Top 5 free Python courses based on ratings and reviews
  • Course distribution by category and level
  • Comparison of ratings for free vs. paid courses
  • Most popular business courses

📜 Credits

  • Author: Marco Minaudo
  • Course: Systems and Methods for Big and Unstructured Data (SMBUD)
  • Academic Year: 2024-2025

🌟 If you find this project useful, feel free to star ⭐ the repository!