A comprehensive real-time transportation data visualization and routing system for Singapore, built with a modern polyglot persistence architecture combining SQL and NoSQL databases.
- Overview
- Features
- Architecture
- Tech Stack
- Prerequisites
- Installation
- Configuration
- Running the Application
- API Documentation
- Database Schema
- Project Structure
- Data Sources
- Performance Optimization
- Contributing
- License
The Smart City Framework is an advanced transportation management system designed for Singapore's urban infrastructure. It provides real-time traffic monitoring, intelligent routing with live traffic data, and comprehensive visualization of transportation assets including buses, trains, taxis, and parking facilities.
Built as a comprehensive academic project for INF2003 Database Systems, this system demonstrates advanced database concepts including polyglot persistence, query optimization, graph algorithms, and real-time data processing. The project integrates multiple Singapore government APIs (LTA DataMall, Data.gov.sg, NEA, URA) into a unified platform that combines historical analytics with real-time monitoring.
🎯 Real-World Application: Unlike typical academic projects, this system processes real data from Singapore's transportation infrastructure and can be used for actual urban planning and navigation.
🗄️ Polyglot Persistence Mastery: Strategic use of four different database technologies (MariaDB, MongoDB, Neo4j, Redis), each selected for its optimal use case - demonstrating deep understanding of when and why to use different database paradigms.
⚡ Performance Engineering: Achieved 481x speedup through intelligent indexing and query optimization, reducing database complexity from 28 tables to 9 core tables + 4 views while maintaining full functionality.
🌐 Full-Stack Integration: Complete three-tier architecture with Python FastAPI, Node.js Express, and interactive frontend, all communicating seamlessly through RESTful APIs.
📊 Real-Time Intelligence: Live data updates every 2-5 minutes through automated cron jobs, with intelligent caching strategies and traffic-aware routing that adapts to current road conditions.
- Real-time Traffic Intelligence: Live updates every 2-5 minutes from LTA DataMall
- Graph-based Routing: Neo4j-powered shortest path with traffic-aware duration calculations
- Polyglot Persistence: Strategic database selection for optimal performance
- Comprehensive Coverage: 5,000+ bus stops, 26,000+ routes, 382 carparks, 165 train stations
- Interactive Dashboard: Real-time map visualization with traffic cameras and incidents
- Production-Ready: Deployed architecture with proper error handling, logging, and monitoring
The Smart City Dashboard features a clean, modern interface with a purple gradient theme:
┌─────────────────────────────────────────────────────────────────┐
│ 🏙️ Smart City Dashboard │
│ Singapore Transportation Infrastructure & Route Planning │
├────────────┬────────────────────────────────────────────────────┤
│ │ │
│ 📊 Stats │ 🗺️ INTERACTIVE MAP │
│ -------- │ │
│ 5,042 │ ⚠️ Live Congestion & Alerts │
│ Bus Stops │ [Scrolling: Heavy traffic on PIE...] │
│ │ │
│ 🗺️ Layers│ 🔵 🔴 🟢 🟠 🟣 ← Color-coded markers │
│ ☑ Buses │ │
│ ☑ Trains │ [Interactive Leaflet map with OpenStreetMap] │
│ ☑ Parks │ │
│ ☐ Taxis │ Click to explore • Pan to navigate │
│ ☑ Cameras │ │
│ │ │
│ 🔄 Refresh│ │
│ 🗑️ Clear │ │
│ │ │
│ Legend │ │
│ 🔵 Bus │ │
│ 🔴 Train │ │
│ 🟢 Park │ │
└────────────┴────────────────────────────────────────────────────┘
Switch to Routes tab for traffic-aware navigation:
┌────────────────────────────────────────────────────────────────┐
│ 🧭 Route Planner │
├────────────┬───────────────────────────────────────────────────┤
│ │ │
│ From: │ 🔴 START (Red Pin) │
│ 1.3521, │ │
│ 103.8198 │ ↓ │
│ │ 🟢───→ Color-coded route │
│ To: │ 🟡───→ by traffic speed │
│ 1.2966, │ 🟠───→ │
│ 103.7764 │ 🔴───→ │
│ │ ↓ │
│ Find │ 🟢 END (Green Pin) │
│ Route │ │
│ │ Route: Fastest Route │
│ Route │ Distance: 15.2 km │
│ Info: │ Duration: 18.5 mins │
│ -------- │ │
│ ➡️ Continue│ Directions: │
│ onto PIE │ ➡️ Continue on PIE (2.3km) │
│ (2.3km) │ ⬅️ Turn left onto CTE (5.1km) │
│ │ ➡️ Turn right onto Orchard (800m) │
│ Nearby │ ✓ Arrived at destination │
│ Bus Stops │ │
│ • Stop A │ Traffic Legend: │
│ • Stop B │ 🟢 >60 km/h 🟡 40-60 🟠 20-40 🔴 <20 │
└────────────┴───────────────────────────────────────────────────┘
Real-time scrolling alerts at the top of the map:
┌──────────────────────────────────────────────────────────────┐
│ ⚠️ Live Congestion & Alerts │
│ [Scrolling] 📢 Heavy traffic on PIE → 🚇 MRT delay NS Line → │
│ 💥 Accident at Bukit Timah Rd → 🚧 Roadwork... │
└──────────────────────────────────────────────────────────────┘
1. Layer Visualization
- Click checkbox to show/hide layers
- Real-time counter updates (e.g., "Bus Stops (5,042)")
- Color-coded markers for quick identification
2. Traffic Camera Feeds
- Purple markers for camera locations
- Click marker to view live image feed
- Camera ID and location details in popup
3. Traffic Incidents
- Dynamic emoji icons (💥 Accident, 🚧 Roadwork, 🌊 Flood)
- Incident type and description in popup
- Real-time updates every 2 minutes
4. Carpark Availability
- Green markers for parking facilities
- Shows available lots and lot type
- Development name and location
5. Route Visualization
Traffic Speed Color Coding:
🟢 Green Line = Fast (>60 km/h) - Smooth flow
🟡 Yellow Line = Moderate (40-60) - Normal traffic
🟠 Orange Line = Slow (20-40) - Heavy traffic
🔴 Red Line = Congestion (<20) - Jammed
- Live Traffic Speed Bands: Real-time road segment speeds across Singapore
- Traffic Incidents: Up-to-date accident and roadwork information
- Traffic Cameras: 90+ live camera feeds with image URLs
- VMS/EMAS Alerts: Variable Message Signs and traffic management updates
- Live Alert Marquee: Scrolling real-time alerts at top of dashboard
- Traffic-Aware Navigation: Dynamic route calculation using current traffic conditions
- Turn-by-Turn Directions: Detailed navigation instructions with distance markers
- Congestion Analysis: Light/Moderate/Heavy traffic classification
- Multi-Route Options: Alternative route suggestions
- Visual Traffic Overlay: Color-coded route segments by current speed
- Interactive Map Selection: Click-to-route interface for easy planning
- Bus Services: 580 services with real-time schedule information
- Train Network: Complete MRT/LRT station coverage
- Taxi Stands: 316+ designated taxi pickup locations
- Service Alerts: Real-time train service disruptions
- Nearby Stop Discovery: Auto-suggest 5 closest bus stops to destination
- Carpark Availability: Live parking lot availability data
- Location Search: Find nearby parking with coordinates
- Multi-Agency Data: HDB, URA, LTA parking facilities
- Interactive Map: Leaflet-based map with layer controls
- Tab-Based Interface: Organized Layers and Routes views
- Real-Time Statistics: Live counters for all transportation assets
- Color-Coded Markers: Visual distinction between transportation types
- Responsive Design: Modern gradient purple theme
- Loading States: User-friendly loading indicators
- Error Handling: Graceful error messages and recovery
┌─────────────────────────────────────────────────────────────┐
│ FRONTEND LAYER │
│ │
│ HTML5 + CSS3 + Leaflet.js + Chart.js │
│ Interactive Map | Real-time Updates | Traffic Cameras │
└─────────────────────────────────────────────────────────────┘
▼
┌─────────────────────────────────────────────────────────────┐
│ APPLICATION LAYER │
│ │
│ ┌────────────────────┐ ┌─────────────────────┐ │
│ │ FastAPI (Python) │ │ Node.js Express │ │
│ │ Port: 8000 │ │ Port: 5000 │ │
│ │ - Routing API │ │ - SQL API │ │
│ │ - Traffic API │ │ - Historical Data │ │
│ │ - NoSQL Queries │ │ - CRUD Operations │ │
│ └────────────────────┘ └─────────────────────┘ │
│ │
│ ┌──────────────────────────────────────────────────────┐ │
│ │ Cron Job Manager (Node.js) │ │
│ │ - Every 2 min: VMS/EMAS, Traffic Incidents │ │
│ │ - Every 5 min: Speed Bands, Neo4j Graph Updates │ │
│ │ - Every 5 min: Train Service Alerts │ │
│ └──────────────────────────────────────────────────────┘ │
└─────────────────────────────────────────────────────────────┘
▼
┌─────────────────────────────────────────────────────────────┐
│ DATABASE LAYER │
│ │
│ ┌─────────────┐ ┌──────────┐ ┌────────┐ ┌─────────┐ │
│ │ MariaDB │ │ MongoDB │ │ Neo4j │ │ Redis │ │
│ │ │ │ │ │ │ │ │ │
│ │ Historical │ │ Real-time│ │ Road │ │ Cache │ │
│ │ Structured │ │ Streams │ │ Network│ │ Layer │ │
│ │ Data │ │ Traffic │ │ Graph │ │ │ │
│ └─────────────┘ └──────────┘ └────────┘ └─────────┘ │
└─────────────────────────────────────────────────────────────┘
| Database | Purpose | Data Types | Update Frequency |
|---|---|---|---|
| MariaDB | Historical structured data | Bus stops, routes, services, stations, carparks | Static/Daily |
| MongoDB | Real-time dynamic data | Traffic incidents, VMS, train alerts, speed bands | 2-5 minutes |
| Neo4j | Road network graph | Traffic-aware routing, pathfinding | 5 minutes |
| Redis | High-speed caching | Frequently accessed queries | As needed |
┌─────────────────────────────────────────────────────────────────┐
│ EXTERNAL DATA SOURCES │
├─────────────────────────────────────────────────────────────────┤
│ │
│ 📡 LTA DataMall API 📊 Data.gov.sg │
│ ├── Bus data ├── Historical stats │
│ ├── Train alerts ├── Infrastructure locations │
│ ├── Traffic cameras └── Urban planning data │
│ ├── Speed bands │
│ ├── Incidents 🌡️ NEA API │
│ └── Parking └── Weather data │
│ │
│ 🏢 URA API │
│ └── Carpark data │
└──────────────────┬───────────────────────────────────────────────┘
│
↓
┌─────────────────────────────────────────────────────────────────┐
│ CRON JOB MANAGER (Node.js) │
├─────────────────────────────────────────────────────────────────┤
│ ⏰ Every 2 minutes: VMS/EMAS, Traffic Incidents │
│ ⏰ Every 5 minutes: Train Alerts, Speed Bands │
│ ⏰ Neo4j Updates: Graph weights from speed data │
│ ⏰ Startup Check: Build road network if empty │
└──────────────────┬───────────────────────────────────────────────┘
│
┌────────┴────────┐
↓ ↓
┌──────────────────┐ ┌──────────────────┐
│ MongoDB │ │ Neo4j Graph │
│ (Real-time) │ │ (Routing) │
├──────────────────┤ ├──────────────────┤
│ • Incidents │ │ • Point nodes │
│ • VMS alerts │ │ • Road segments │
│ • Train alerts │ │ • Traffic speeds │
│ • Speed bands │ │ • Durations │
│ │ │ • Shortest path │
│ TTL: 2-5 min │ │ Update: 5 min │
└──────────────────┘ └──────────────────┘
│ │
│ ┌────────┴────────┐
│ ↓ ↓
│ ┌──────────────┐ ┌──────────────┐
│ │ MariaDB │ │ Redis │
│ │ (Historical)│ │ (Cache) │
│ ├──────────────┤ ├──────────────┤
│ │ • Bus stops │ │ • Query cache│
│ │ • Routes │ │ • Session │
│ │ • Stations │ │ • API cache │
│ │ • Carparks │ │ │
│ │ • Cameras │ │ TTL: Varies │
│ │ │ └──────────────┘
│ │ Static/Daily │ │
│ └──────────────┘ │
│ │ │
└─────────┴─────────────────┘
│
┌─────────┴─────────┐
↓ ↓
┌──────────────────┐ ┌──────────────────┐
│ FastAPI │ │ Node.js │
│ (Python) │ │ (Express) │
│ Port: 8001 │ │ Port: 8000 │
├──────────────────┤ ├──────────────────┤
│ • Routing API │ │ • SQL queries │
│ • Traffic API │ │ • Historical data│
│ • NoSQL queries │ │ • CRUD ops │
│ • Neo4j routing │ │ • Statistics │
│ • Cache mgmt │ │ • Bus/Train API │
└──────────────────┘ └──────────────────┘
│ │
└────────┬───────────┘
↓
┌─────────────────────────────────────────────────────────────────┐
│ FRONTEND (HTML/CSS/JS) │
│ Port: 5500 │
├─────────────────────────────────────────────────────────────────┤
│ │
│ 🗺️ Layers Tab 🧭 Routes Tab │
│ ├── Statistics dashboard ├── Click-to-route │
│ ├── Layer toggles ├── Traffic visualization │
│ ├── Marker controls ├── Turn-by-turn directions │
│ └── Data refresh └── Nearby stops │
│ │
│ 📍 Interactive Leaflet Map with Real-time Updates │
│ ⚠️ Live Alert Marquee (VMS, Incidents, Train Alerts) │
│ │
└─────────────────────────────────────────────────────────────────┘
│
↓
👤 END USER
When to use each database:
| Requirement | Database Choice | Reason |
|---|---|---|
| Static bus stop locations | MariaDB | Relational structure, infrequent updates |
| Real-time traffic incidents | MongoDB | Document-based, frequent updates |
| Traffic-aware routing | Neo4j | Graph relationships, pathfinding algorithms |
| High-speed API response cache | Redis | In-memory, sub-millisecond access |
| Complex queries with JOINs | MariaDB | SQL relationships, ACID compliance |
| Flexible schema for API data | MongoDB | Schemaless, handles varying API responses |
| Shortest path calculations | Neo4j | Native graph traversal, Dijkstra algorithm |
| Session management | Redis | TTL support, key-value store |
This project was developed for INF2003 Database Systems at Singapore Institute of Technology, demonstrating concepts that go "far beyond" the module curriculum:
Core Database Concepts Demonstrated:
- ✅ Complex SQL queries with multiple JOINs and aggregations
- ✅ Database normalization and denormalization strategies
- ✅ Index optimization and query performance tuning
- ✅ Transaction management and ACID properties
- ✅ View creation for query simplification
- ✅ Stored procedures and triggers (where applicable)
Advanced Concepts Implemented:
- ✅ Polyglot Persistence: Strategic use of SQL, NoSQL, Graph, and In-Memory databases
- ✅ Graph Algorithms: Neo4j Cypher for shortest path with dynamic edge weights
- ✅ Real-time Processing: Cron-based ETL pipelines with intelligent caching
- ✅ Query Optimization: 481x performance improvement through indexing strategies
- ✅ Database Architecture: Hybrid SQL/NoSQL design with clear separation of concerns
- ✅ Spatial Queries: Location-based searches with distance calculations
- ✅ API Integration: Multi-source data aggregation with error handling
- ✅ Batch Processing: Efficient bulk operations with transaction control
Technical Documentation Depth:
- Database schema design rationale
- Performance benchmarking and optimization analysis
- Trade-off discussions (foreign keys vs. API flexibility)
- Cost-benefit analysis of different database technologies
- Comparative implementation approaches
- FastAPI (Python 3.13) - Main API framework for routing and traffic services
- Node.js (v18+) - SQL API server and cron job management
- Express.js - Node.js web framework
- Uvicorn - ASGI server for FastAPI
- MariaDB 12.0 - Primary relational database
- MongoDB Atlas - NoSQL document store for real-time data
- Neo4j 5.x - Graph database for routing algorithms
- Redis - In-memory cache and message broker
- Leaflet.js - Interactive mapping library
- Chart.js - Data visualization
- HTML5/CSS3 - Modern web standards
- Pandas - Data manipulation and analysis
- NumPy - Numerical computations
- Python-dateutil - Date/time handling
- LTA DataMall API - Singapore Land Transport Authority data
- Data.gov.sg - Singapore Government Open Data
- NEA API - National Environment Agency
- URA API - Urban Redevelopment Authority
- Python 3.13+ with pip
- Node.js 18+ with npm
- MariaDB 12.0+
- MongoDB Atlas account (or local MongoDB)
- Neo4j AuraDB account (or local Neo4j)
- Redis (local or cloud instance)
- LTA DataMall API Key (Register here)
- MongoDB Atlas connection string
- Neo4j AuraDB credentials
- Redis connection details
git clone https://github.com/yourusername/smart-city-framework.git
cd smart-city-frameworkcd BACKEND
# Create virtual environment
python3.13 -m venv venv
source venv/bin/activate # On Windows: venv\Scripts\activate
# Install Python dependencies
pip install -r requirements.txt
# For NoSQL-specific dependencies
pip install -r requirements_nosql.txt --break-system-packages# In BACKEND directory
npm install# Create database
mysql -u root -p
CREATE DATABASE smart_city;
USE smart_city;
SOURCE sql/smart_city.sql;- Create a MongoDB Atlas cluster
- Whitelist your IP address
- Create a database user with read/write permissions
- Note your connection string
# Build the road network graph (one-time setup)
cd BACKEND/nosql
node build_network.jsThis script will:
- Fetch all traffic speed band data from LTA
- Create Point nodes for road intersections
- Create ROAD_SEGMENT relationships with traffic data
- Takes ~5-10 minutes depending on network
- Set up Redis locally or use a cloud provider
- Note your connection details (host, port, password)
Create a .env file in the BACKEND directory:
# Application Settings
APP_NAME=Smart City Framework
VERSION=1.0.0
DEBUG=true
# MariaDB Configuration
MARIADB_HOST=localhost
MARIADB_USER=root
MARIADB_PASSWORD=your_password
MARIADB_DATABASE=smart_city
# MongoDB Configuration
MONGODB_USER=your_mongo_user
MONGODB_PASSWORD=your_mongo_password
MONGODB_CLUSTER=cluster0.xxxxx.mongodb.net
MONGODB_NAME=smart_city
# Neo4j Configuration
NEO4J_URI=neo4j+s://xxxxx.databases.neo4j.io
NEO4J_USER=neo4j
NEO4J_PASSWORD=your_neo4j_password
# Redis Configuration
REDIS_HOST=your-redis-host.com
REDIS_PORT=6379
REDIS_PASSWORD=your_redis_password
# LTA DataMall API
LTA_API_KEY=your_lta_api_key
# Server Ports
FASTAPI_PORT=8000
NODE_PORT=5000
FRONTEND_PORT=5500BACKEND/app/config/settings.py- Application settingsBACKEND/app/config/database.py- Database connection poolsBACKEND/package.json- Node.js dependenciesBACKEND/requirements.txt- Python dependencies
You'll need 4 terminal windows to run the complete system:
# Start MariaDB service
mysql.server start # macOS
# OR
sudo systemctl start mariadb # Linuxcd BACKEND
source venv/bin/activate
python nosql/main.py
# Server runs on http://localhost:8001cd BACKEND
node cron.js
# Background service - updates data every 2-5 minutescd FRONTEND
python -m http.server 5500
# Or use Live Server extension in VS Code
# Dashboard available at http://localhost:5500/html/dashboard.html- Open your browser to:
http://localhost:5500/html/dashboard.html - Wait for initial data load (~5-10 seconds)
- Explore the 🗺️ Layers tab to view transportation infrastructure
- Switch to 🧭 Routes tab to plan routes
- Toggle layers using checkboxes in the sidebar
- Click markers to see details (bus stop names, carpark availability, camera feeds)
- View statistics at the top of the Layers tab
- Monitor live alerts in the scrolling marquee at the top of the map
- Switch to the Routes tab
- Click the map to set your starting point (red marker appears)
- Click again to set your destination (green marker appears)
- Click "Find Route" button
- View:
- Traffic-aware route with color-coded segments
- Distance and estimated duration
- Turn-by-turn directions
- Nearby bus stops at destination
- 🟢 Green: Fast traffic (>60 km/h)
- 🟡 Yellow: Moderate traffic (40-60 km/h)
- 🟠 Orange: Slow traffic (20-40 km/h)
- 🔴 Red: Heavy congestion (<20 km/h)
- Refresh Data: Click the refresh button to reload all layers
- Clear Cache: Force fresh data fetch from external APIs (removes MongoDB/Redis cache)
cd BACKEND
source venv/bin/activate
python nosql/main.py
# Runs on http://localhost:8000cd BACKEND
node sql/smart_city_api.py
# Runs on http://localhost:5000cd BACKEND
node cron.js
# Background service for data updatescd FRONTEND
python -m http.server 5500
# Or use Live Server extension in VS Code
# Runs on http://localhost:5500# FastAPI with Gunicorn
gunicorn nosql.main:app -w 4 -k uvicorn.workers.UvicornWorker --bind 0.0.0.0:8000
# Node.js with PM2
pm2 start sql/smart_city_api.py --name smart-city-sql
pm2 start cron.js --name smart-city-cronWhen running in DEBUG mode, access comprehensive API documentation at:
- Swagger UI: http://localhost:8000/docs
- ReDoc: http://localhost:8000/redoc
GET /api/routing/optimize?start_lat=1.3521&start_lon=103.8198&end_lat=1.2966&end_lon=103.7764&vehicle_type=carReturns traffic-aware optimized routes with turn-by-turn directions.
GET /api/v1/traffic/incidentsReal-time traffic incidents and accidents.
GET /api/v1/traffic/camerasAll traffic camera feeds with image URLs.
GET /api/v1/traffic/vms-emasVariable Message Signs and EMAS alerts.
GET /api/bus-stops?limit=100&offset=0Paginated bus stop information.
GET /api/train-stationsAll MRT/LRT stations with locations.
GET /api/v1/parking/overviewComprehensive parking availability overview.
All API responses follow this structure:
{
"status": "success",
"data": { ... },
"metadata": {
"timestamp": "2025-11-30T10:00:00Z",
"source": "LTA DataMall"
}
}Error responses:
{
"error": "error_code",
"message": "Human-readable error message",
"detail": "Technical details (DEBUG mode only)"
}- bus_stops - 5,000+ bus stop locations
- bus_routes - 26,000+ route segments with schedules
- bus_services - 580 service definitions
- carparks - 382 carpark facilities
- train_stations - 165 MRT/LRT stations
- taxi_stops - 316 taxi stands
- traffic_cameras - 90+ camera locations
- erp_rates - Electronic Road Pricing rates
- bus_interchange - Major bus terminal info
- view_bus_stops_with_services - Bus stops joined with service counts
- view_carpark_summary - Aggregated parking statistics
- view_train_station_routes - Stations with line information
- view_traffic_overview - Combined traffic monitoring data
// traffic_incidents
{
Type: "Accident",
Latitude: 1.3521,
Longitude: 103.8198,
Message: "Accident on PIE...",
cachedAt: ISODate("2025-11-30T10:00:00Z")
}
// vms_emas
{
DeviceID: "VMS001",
Message: "Heavy traffic ahead",
Latitude: 1.3521,
Longitude: 103.8198,
cachedAt: ISODate("2025-11-30T10:00:00Z")
}
// train_service_alerts
{
Status: 1,
AffectedSegments: "NS1-NS5",
Message: "Train delay...",
cachedAt: ISODate("2025-11-30T10:00:00Z")
}
// traffic_speed_bands
{
LinkID: "12345",
RoadName: "Pan Island Expressway",
MinimumSpeed: 45,
MaximumSpeed: 65,
StartLat: 1.3521,
StartLon: 103.8198,
cachedAt: ISODate("2025-11-30T10:00:00Z")
}// Nodes
(:Point {lat: Float, lon: Float, id: String})
// Relationships
(:Point)-[:ROAD_SEGMENT {
link_id: String,
road_name: String,
road_category: String,
min_speed: Float,
max_speed: Float,
current_speed_kph: Float,
distance_km: Float,
duration_sec: Float
}]->(:Point)The frontend provides a comprehensive real-time visualization and routing interface built with modern web technologies:
- 🗺️ Layers Tab: Map layer controls and statistics
- 🧭 Routes Tab: Interactive route planning with turn-by-turn directions
- Interactive Leaflet Map: Centered on Singapore (1.3521°N, 103.8198°E)
- Layer Toggle System: Show/hide different transportation layers
- Live Traffic Alerts: Scrolling marquee with real-time incidents, VMS messages, and train alerts
- Color-Coded Markers:
- 🔵 Bus Stops (Blue)
- 🔴 Train Stations (Red)
- 🟢 Carparks (Green)
- 🟠 Taxi Stands (Orange)
- 🟣 Traffic Cameras (Purple)
- 💥 Traffic Incidents (Dynamic emojis)
- Click-to-Route: Click map to set start/end points
- Traffic-Aware Visualization: Color-coded route segments by speed:
- 🟢 Green: >60 km/h (Fast)
- 🟡 Yellow: 40-60 km/h (Moderate)
- 🟠 Orange: 20-40 km/h (Slow)
- 🔴 Red: <20 km/h (Congestion)
- Turn-by-Turn Directions: Detailed navigation instructions
- Nearby Bus Stops: Automatic discovery of 5 closest stops
- Auto-Refresh: Background cron jobs update data every 2-5 minutes
- Manual Refresh: User-triggered data reload
- Cache Management: Clear cache to force fresh data fetch
- Live Statistics: Real-time counters for all transportation assets
| Technology | Purpose | Version |
|---|---|---|
| Leaflet.js | Interactive mapping library | 1.9.4 |
| OpenStreetMap | Base map tiles | Latest |
| Vanilla JavaScript | Application logic | ES6+ |
| CSS3 | Modern styling with gradients | - |
| HTML5 | Semantic markup | - |
// Real-time statistics grid
- Bus Stops: 5,000+
- Train Stations: 165
- Carparks: 382
- Taxi Stands: 316
- Traffic Cameras: 90+
- Bus Routes: 26,000+Interactive checkboxes with live counts:
☑️ Bus Stops (5,042)
☑️ Train Stations (165)
☑️ Carparks (382)
☐ Taxi Stands (316)
☑️ Traffic Cameras (90)
☑️ Traffic Incidents (12)🚗 Route: Fastest Route
Distance: 15.2 km
Duration: 18.5 mins
Directions:
➡️ Continue onto Pan Island Expressway (2.3km)
⬅️ Turn left onto Central Expressway (5.1km)
➡️ Turn right onto Orchard Road (800m)
✓ You have arrived at your destination
The frontend communicates with dual backend APIs:
const CONFIG = {
API: {
SQL: 'http://localhost:8000', // Historical data
NOSQL: 'http://localhost:8001' // Real-time data & routing
}
};Key Frontend-Backend Interactions:
| Frontend Action | Backend API | Response |
|---|---|---|
| Load bus stops | GET /api/bus-stops?limit=5000 |
Array of bus stop objects |
| Load train stations | GET /api/train-stations |
Array of station objects |
| Load carparks | GET /api/carparks |
Array of carpark objects with availability |
| Load traffic cameras | GET /api/traffic-cameras/live |
Array with live image URLs |
| Load incidents | GET /api/v1/traffic/incidents |
Real-time incident data |
| Plan route | GET /api/routing/optimize?start_lat=... |
Optimized routes with turn-by-turn |
| Load statistics | GET /api/stats |
Aggregated counts |
| Clear cache | DELETE /api/cache/clear |
Cache deletion confirmation |
The dashboard uses a sophisticated SVG-based icon generation system:
// Dynamic icon generator
const icons = {
create: (color, size = [9, 9]) => L.icon({
iconUrl: `data:image/svg+xml;base64,...`,
iconSize: size,
iconAnchor: [size[0]/2, size[1]/2]
})
};
// Specialized route markers
icons.routeStart // Red pin marker for origin
icons.routeEnd // Green pin marker for destination
// Dynamic incident icons
icons.getIncident(type) // Returns emoji based on incident type:
// 💥 Accident, 🚗 Breakdown, 🚧 Roadwork, 🌊 Flood, 🚦 Traffic LightClient-side state is managed through a global state object:
const state = {
routeMode: 'driving', // Transportation mode
routingAlgo: 'neo4j', // Routing algorithm
routeMarkers: { from: null, to: null }, // Route start/end markers
neo4jRouteLines: [], // Active route polylines
allBusStops: [], // Cached bus stops for nearby search
liveAlerts: { // Real-time alert aggregation
train: [],
vms: [],
incidents: []
}
};// Parallel data fetching using Promise.allSettled
await Promise.allSettled([
loadBusStops(),
loadTrainStations(),
loadCarparks(),
loadCameras(),
loadIncidents()
]);- Limit parameters: Configurable limits prevent excessive marker rendering
- Layer groups: Efficient layer management with Leaflet LayerGroups
- On-demand loading: Taxi stops only load when checkbox is enabled
- Browser-side caching: Stores bus stop data for nearby searches
- Server-side caching: Redis/MongoDB cache with TTL policies
- Manual cache control: User-triggered cache clearing
smart-city-framework/
│
├── BACKEND/
│ ├── app/
│ │ ├── config/
│ │ │ ├── database.py # Database connection management
│ │ │ └── settings.py # Application configuration
│ │ ├── models/
│ │ │ ├── nosql_models.py # MongoDB/Neo4j data models
│ │ │ └── sql_models.py # SQLAlchemy ORM models
│ │ ├── services/
│ │ │ ├── traffic_service.py # Traffic data processing
│ │ │ └── parking_service.py # Parking availability logic
│ │ └── utils/
│ │ └── helpers.py # Utility functions
│ │
│ ├── nosql/
│ │ ├── build_network.js # One-time Neo4j graph builder
│ │ ├── lta.js # LTA API integration
│ │ ├── mongodb.js # MongoDB connection
│ │ ├── neo4j.js # Neo4j driver setup
│ │ ├── redis.js # Redis connection
│ │ └── main.py # FastAPI application entry point
│ │
│ ├── sql/
│ │ ├── smart_city.sql # Database schema and data dump
│ │ └── smart_city_api.py # Node.js SQL API server
│ │
│ ├── cron.js # Automated data refresh jobs
│ ├── package.json # Node.js dependencies
│ ├── requirements.txt # Python dependencies (SQL)
│ └── requirements_nosql.txt # Python dependencies (NoSQL/Routing)
│
└── FRONTEND/
├── html/
│ └── dashboard.html # Single-page application (619 lines)
│ # - Modular JavaScript architecture
│ # - Real-time map visualization
│ # - Interactive routing interface
│ # - Live traffic alerts marquee
└── style.css # Complete styling (623 lines)
# - Gradient purple theme
# - Responsive layout
# - Tab system
# - Map overlays
# - Loading states
Primary source for real-time transportation data:
- Bus arrivals, routes, and services
- Traffic speed bands and incidents
- Carpark availability
- Train service alerts
- Taxi availability
- Traffic camera images
API Endpoint: https://datamall2.mytransport.sg/ltaodataservice/
Government open data portal:
- Historical transportation statistics
- Infrastructure locations
- Urban planning data
- High-frequency (2 minutes): VMS/EMAS, Traffic Incidents
- Medium-frequency (5 minutes): Speed Bands, Train Alerts, Neo4j Updates
- Low-frequency (Daily/Static): Bus stops, Routes, Stations
-- Composite indexes for common queries
CREATE INDEX idx_service_direction ON bus_routes(ServiceNo, Direction);
CREATE INDEX idx_stop_lookup ON bus_stops(BusStopCode);
CREATE INDEX idx_carpark_area ON carparks(Area, Development);
-- Spatial indexes for location queries
CREATE SPATIAL INDEX idx_bus_location ON bus_stops(Latitude, Longitude);- Bus route lookup: 481x faster with optimized indexes
- Carpark search: ~0.05s average response time
- Database size: Reduced from 28 tables to 9 core tables + 4 views
db.traffic_speed_bands.createIndex({ LinkID: 1 })
db.traffic_incidents.createIndex({ cachedAt: -1 })
db.vms_emas.createIndex({ Location: "2dsphere" })// Index on Point coordinates for fast nearest-neighbor search
CREATE INDEX point_location FOR (p:Point) ON (p.lat, p.lon)
// Composite property index for route queries
CREATE INDEX road_segment_props FOR ()-[r:ROAD_SEGMENT]-() ON (r.current_speed_kph, r.distance_km)Redis TTL Policies:
- Traffic incidents: 120 seconds
- Speed bands: 300 seconds
- Static data (bus stops): 24 hours
- API responses: Conditional based on data volatility
// Batch processing for bulk operations
const batchSize = 500;
for (let i = 0; i < data.length; i += batchSize) {
const batch = data.slice(i, i + batchSize);
await processBatch(batch);
}Problem: Dashboard shows "Loading data..." indefinitely
Solution:
1. Check that all backend services are running
2. Verify API endpoints in browser console (F12)
3. Check CORS settings in backend configuration
4. Ensure ports 8000, 8001, 5000 are accessibleProblem: Map markers not appearing
Solution:
1. Check browser console for JavaScript errors
2. Verify data is loading (check Network tab in DevTools)
3. Ensure layer checkboxes are enabled
4. Try refreshing the page (Ctrl+R)Problem: Route planning returns "No route found"
Solution:
1. Ensure Neo4j road network is built (run build_network.js)
2. Check that cron job is updating traffic speeds
3. Verify Neo4j connection in FastAPI logs
4. Try clicking points within Singapore boundariesProblem: FastAPI server won't start
Solution:
1. Check Python version: python --version (requires 3.13+)
2. Verify all dependencies: pip list
3. Check MongoDB/Neo4j connectivity
4. Review .env file configurationProblem: Cron job not updating data
Solution:
1. Check LTA API key is valid
2. Verify MongoDB connection string
3. Check Node.js version: node --version (requires 18+)
4. Review cron.js logs for error messagesProblem: Neo4j routing errors
Solution:
1. Verify Neo4j credentials in .env
2. Run build_network.js to populate graph database
3. Check Neo4j Aura instance is running
4. Ensure traffic speed updates are occurring (check cron logs)Problem: MariaDB connection refused
Solution:
1. Start MariaDB: mysql.server start (macOS) or systemctl start mariadb (Linux)
2. Verify credentials in .env file
3. Check database exists: SHOW DATABASES;
4. Import schema if missing: SOURCE sql/smart_city.sql;Problem: MongoDB cache not updating
Solution:
1. Verify MongoDB Atlas connection string
2. Check IP whitelist in MongoDB Atlas
3. Test connection: mongo "your-connection-string"
4. Use "Clear All Cache" button in dashboardProblem: Slow map loading
Solution:
1. Reduce BUS_STOPS limit in CONFIG (dashboard.html line 161)
2. Disable unnecessary layers
3. Check network speed to APIs
4. Consider using Redis cacheProblem: Routing takes too long
Solution:
1. Ensure Neo4j indexes are created (check build_network.js)
2. Verify cron job is updating graph weights
3. Check Neo4j instance resources (RAM/CPU)
4. Consider shorter routes (< 30 km works best)Enable debug logging in FastAPI:
# In BACKEND/app/config/settings.py
DEBUG = TrueCheck browser console for frontend errors:
F12 (Chrome/Firefox) → Console tab
Look for red error messages
Check backend logs:
# FastAPI logs
tail -f backend.log
# Cron job logs
tail -f cron.log
# Check specific endpoint
curl http://localhost:8001/api/v1/traffic/incidentsWe welcome contributions! Please follow these guidelines:
- Fork the repository
- Create a feature branch (
git checkout -b feature/amazing-feature) - Commit your changes (
git commit -m 'Add amazing feature') - Push to the branch (
git push origin feature/amazing-feature) - Open a Pull Request
- Follow PEP 8 for Python code
- Use ESLint for JavaScript
- Write unit tests for new features
- Update documentation for API changes
- Ensure all tests pass before submitting PR
This project is licensed under the MIT License - see the LICENSE file for details.
- Land Transport Authority (LTA) - For comprehensive DataMall API providing real-time transportation data
- Data.gov.sg - Singapore Government Open Data Portal for infrastructure datasets
- Singapore Institute of Technology - INF2003 Database Systems Module
- Team 11 - Project development and implementation
- OpenStreetMap Contributors - Base map tiles for visualization
- Leaflet.js Community - Excellent mapping library
- Backend: ~2,500 lines (Python + Node.js)
- Frontend: ~1,200 lines (HTML + CSS + JavaScript)
- Database Schema: 33,000+ lines (with data)
- Total Files: 25+ source files
- Bus Infrastructure: 5,042 stops, 26,000+ routes, 580 services
- Rail Network: 165 stations across MRT/LRT lines
- Road Network: 15,000+ segments in Neo4j graph
- Parking: 382 carparks with real-time availability
- Monitoring: 90+ traffic cameras, real-time incident tracking
- Query Optimization: 481x speedup on key queries
- Data Update Frequency: 2-5 minute intervals
- Average API Response: <200ms for most endpoints
- Route Calculation: ~1-2 seconds for typical Singapore routes
- Database Size: ~180MB MariaDB + dynamic NoSQL collections
- Polyglot Persistence: Strategic use of 4 different database technologies
- Query Optimization: From 28 tables down to 9 core tables + 4 views
- Hybrid Architecture: SQL for historical, NoSQL for real-time
- Graph Algorithms: Traffic-aware shortest path implementation
- Multi-API Integration: LTA DataMall, Data.gov.sg, NEA, URA
- Real-time Processing: Automated cron jobs with smart caching
- Dual Backend: Python FastAPI + Node.js Express for optimal performance
- Cross-browser Compatibility: Tested on Chrome, Firefox, Safari, Edge
- Turn-by-Turn Navigation: Direction generation with bearing calculations
- Traffic-Aware Routing: Real-time speed band integration in pathfinding
- Live Visualization: Color-coded routes, dynamic markers, scrolling alerts
- Intelligent Caching: Multi-tier caching with differentiated TTL policies
- ✅ Real-time traffic monitoring
- ✅ Graph-based intelligent routing
- ✅ Comprehensive transportation data
- ✅ Interactive dashboard
- 🔄 Machine learning traffic prediction
- 🔄 Multi-modal journey planning
- 🔄 Mobile app integration
- 🔄 User authentication and personalization
- 🔄 Historical analytics dashboard
- 🔄 Weather integration for route planning
# Start all services
cd BACKEND && source venv/bin/activate && python nosql/main.py &
cd BACKEND && node cron.js &
cd FRONTEND && python -m http.server 5500 &
# Stop all services
pkill -f "python nosql/main.py"
pkill -f "node cron.js"
pkill -f "http.server 5500"
# Check service status
lsof -i :8001 # FastAPI
lsof -i :8000 # Node.js SQL API
lsof -i :5500 # Frontend
# View logs
tail -f BACKEND/nosql/app.log
tail -f BACKEND/cron.log
# Database operations
mysql -u root -p smart_city # Access MariaDB
mongo "your-mongodb-uri" # Access MongoDB
cypher-shell -u neo4j # Access Neo4j
# Clear caches (via API)
curl -X DELETE http://localhost:8001/api/cache/clear
# Test endpoints
curl http://localhost:8001/api/v1/traffic/incidents
curl http://localhost:8000/api/bus-stops?limit=10
curl "http://localhost:8001/api/routing/optimize?start_lat=1.3521&start_lon=103.8198&end_lat=1.2966&end_lon=103.7764"# Required for basic operation
MARIADB_HOST=localhost
MARIADB_DATABASE=smart_city
MONGODB_CLUSTER=yourcluster.mongodb.net
NEO4J_URI=neo4j+s://xxxxx.databases.neo4j.io
LTA_API_KEY=your_lta_key_here
# Optional (defaults exist)
REDIS_HOST=localhost
DEBUG=true
FASTAPI_PORT=8001| Port | Service | Purpose |
|---|---|---|
| 3306 | MariaDB | Database server |
| 8001 | FastAPI | NoSQL API & Routing |
| 8000 | Node.js | SQL API & Historical data |
| 5500 | Frontend | Dashboard UI |
| 27017 | MongoDB | NoSQL database (if local) |
| 7687 | Neo4j | Graph database (if local) |
| 6379 | Redis | Cache server (if local) |
Important Files:
├── Configuration
│ ├── .env # All credentials
│ ├── BACKEND/app/config/settings.py # App settings
│ └── BACKEND/app/config/database.py # DB connections
│
├── Entry Points
│ ├── BACKEND/nosql/main.py # FastAPI server
│ ├── BACKEND/sql/smart_city_api.py # Node.js server
│ ├── BACKEND/cron.js # Background jobs
│ └── FRONTEND/html/dashboard.html # UI
│
├── Database
│ ├── BACKEND/sql/smart_city.sql # Schema + data
│ └── BACKEND/nosql/build_network.js # Neo4j setup
│
└── Documentation
├── README.md # This file
└── BACKEND/app/ # API documentation
Traffic & Routing (FastAPI - Port 8001)
GET /api/v1/traffic/incidents # Real-time incidents
GET /api/v1/traffic/cameras # Traffic cameras
GET /api/v1/traffic/vms-emas # VMS alerts
GET /api/routing/optimize # Get optimized route
GET /api/v1/parking/overview # Parking availability
DELETE /api/cache/clear # Clear all caches
Transportation Data (Node.js - Port 8000)
GET /api/bus-stops?limit=100 # Bus stops (paginated)
GET /api/train-stations # Train stations
GET /api/carparks # Carpark data
GET /api/taxi-stops # Taxi stands
GET /api/traffic-cameras/live # Live camera feeds
GET /api/stats # System statistics
| Key | Action |
|---|---|
1 |
Switch to Layers tab |
2 |
Switch to Routes tab |
R |
Refresh all data |
C |
Clear route |
ESC |
Close popups |
F5 |
Reload page |
Map Markers
- 🔵 Blue: Bus Stops
- 🔴 Red: Train Stations
- 🟢 Green: Carparks
- 🟠 Orange: Taxi Stands
- 🟣 Purple: Traffic Cameras
Route Traffic Speeds
- 🟢 Green: >60 km/h (Fast)
- 🟡 Yellow: 40-60 km/h (Moderate)
- 🟠 Orange: 20-40 km/h (Slow)
- 🔴 Red: <20 km/h (Congested)
Incident Icons
- 💥 Accident
- 🚗 Vehicle Breakdown
- 🚧 Roadwork
- 🌊 Flood
- 🚦 Traffic Light Fault
⚠️ Other Incidents
Last Updated: November 30, 2025