Skip to content

Real time IoT data arrives in streams, so I built a streaming pipeline using RabbitMQ, MongoDB, and Flask to enable real‑time analysis and vizn of power grid data through an intuitive frontend

Notifications You must be signed in to change notification settings

AdityaShankar1/Time-Series-Streaming-RabbitMQ

Repository files navigation

Time-Series-Streaming-RabbitMQ

1. 🏭 Project Overview

This project demonstrates a time-series streaming pipeline using RabbitMQ, MongoDB, Pandas, and Flask.
It simulates how electricity grid data (consumption, production, energy mix) can be ingested, persisted, analyzed, and visualized — a workflow similar to downstream IoT data analytics.

The focus is on systems engineering: message brokers, persistence, analytics, and containerization.


2. 🔑 Key Features

  • Message Broker Integration: RabbitMQ streams simulated time-series data.
  • Database Persistence: MongoDB stores incoming messages for durability.
  • Analytics Engine: Pandas computes export/import hours, energy mix, and hourly consumption patterns.
  • Visualization: Matplotlib plots (daily averages, energy distribution pie, hourly consumption).
  • Frontend: Flask app with login (hardcoded credentials) and report page.
  • Testing: Pytest suite for routes and analytics functions.
  • Containerization: Docker + Docker Compose for reproducible deployment.

3. 🛠️ Technical Stack

  • Language: Python 3.14
  • Libraries: Flask, Pandas, Matplotlib, Pytest, Pika (RabbitMQ), PyMongo
  • Message Broker: RabbitMQ
  • Database: MongoDB
  • Containerization: Docker, Docker Compose

4. 🏗️ Architecture

[Producer] → RabbitMQ → [Consumer] → MongoDB → [Analytics (Pandas)] → Flask Dashboard


5. 🚀 How to Run

Local

python app.py

6. 📷 Output Screenshots

First Prototype:

Screenshot 2025-12-31 at 3 14 31 AM Screenshot 2025-12-31 at 3 15 12 AM Screenshot 2025-12-31 at 3 15 25 AM Screenshot 2025-12-31 at 3 15 31 AM

Post WCAG Compliance Enhancements:

Screenshot 2025-12-31 at 6 27 37 PM Screenshot 2025-12-31 at 6 28 32 PM

About

Real time IoT data arrives in streams, so I built a streaming pipeline using RabbitMQ, MongoDB, and Flask to enable real‑time analysis and vizn of power grid data through an intuitive frontend

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published