Sonavi

---

A Sound Detection System for the Deaf and Hard of Hearing using On-Device Machine Learning

📖 Summary

This repository serves as the compiled package of our capstone project for Bulacan State University - Sarmiento Campus entitled: "AI-Driven Mobile Platform with IoT-Enabled Haptic Feedback for Real-Time Sound Recognition and Emergency Alerts for Deaf Individuals"

📑 Contents

• 📖 About
• ⭐ Features
• 📷 Screenshots
• ⚙️ How It Works
• 🧰 Tech Stack
• 📥 Installation
• 📋 Requirements
• 👥 Team
• 🏆 Acknowledgements

📖 About

Sonavi is a sound detection system that alerts Deaf and Hard of Hearing users to environmental sounds through customizable smartwatch vibrations. We harness the power of on-device machine learning to detect and classify sounds captured by a Wear OS smartwatch, process them on an Android mobile device, and deliver haptic feedback—all without requiring an internet connection.

We implemented the YAMNet audio classification model using LiteRT (formerly TensorFlow Lite) for real-time sound detection. The system allows users to detect pre-trained sounds as well as create custom sound profiles by recording or uploading their own audio samples, making it highly personalized and adaptable to individual needs.

⭐ Features

• 🎧 Real-time Sound Detection: Captures audio from your Wear OS smartwatch and processes it instantly on your Android phone
• 🤖 On-Device Machine Learning: Uses YAMNet model via LiteRT (TensorFlow Lite) for accurate sound classification
• 📳 Customizable Vibration Patterns: Set unique vibration alerts for different sound types
• 🎵 Custom Sound Training: Create personalized sound profiles by recording or uploading 3+ samples of specific sounds you want to detect
• 🔒 Privacy-First: All processing happens on-device; no data leaves your phone
• ⚡ Low Latency: Optimized communication between watch and phone for quick notifications

📷 Screenshots

Screenshot 1	Screenshot 2

⚙️ How It Works

1. Capture: The Wear OS smartwatch continuously listens for ambient sounds
2. Transmit: Audio data is sent to the paired Android mobile device
3. Process: The mobile app uses the YAMNet ML model to classify the sound
4. Notify: If a registered sound is detected, a vibration pattern is sent back to the smartwatch

🧰 Tech Stack

• Languages: Kotlin, Java
• Machine Learning: LiteRT (formerly TensorFlow Lite)
• ML Model: YAMNet for audio event classification
• Platform: Android 8.1+ (API 27), Wear OS 3+
• Architecture: MVVM pattern with offline-first approach

📥 Installation

Option 1: Download APK

1. Download the latest release from the Releases page
2. Install the mobile APK on your Android phone
3. Install the Wear OS APK on your smartwatch
4. Pair your devices if not already paired

Option 2: Build from Source

# Clone the repository
git clone https://github.com/xyugen/sonavi.git
cd sonavi

# Build mobile app
./gradlew :mobile:assembleDebug

# Build wear app
./gradlew :wear:assembleDebug

# Install to connected devices
./gradlew installDebug

📋 Requirements

• Mobile Device: Android 8.1 (Oreo, API 27) or higher
• Wearable Device: Wear OS 3 or higher

👥 Team

Capstone Project Team

Role	Name
Project Leader & Lead Developer	Renz Arias
Researcher	Ara Garong
Researcher	Angel Estonina
Quality Assurance	Jeric Gonzales
UI/UX Designer	Jomel Mislos

🏆 Acknowledgements

Name	Contributions
Dr. Mary Grace G. Hermogenes	Our ever-supportive Capstone Professor
Dr. Marlon D.P. Hernandez	Our ever-supportive Capstone Adviser

This README was inspired by ScolioVis.

---

Made with ❤️ for the Deaf and Hard of Hearing community