HavenCore

Self-Hosted AI Smart Home Assistant with Voice Control

A comprehensive AI-powered smart home system designed to run entirely on your own hardware. HavenCore provides voice-activated control, natural language processing, and integrates with popular smart home platforms like Home Assistant.

⚠️ Work in Progress: This project currently works well for the creator but may require configuration adjustments for other environments. Documentation and templates are being improved for broader compatibility.

🚀 Key Features

🎤 Voice Activation: Wake-word detection and real-time speech processing
🗣️ Natural Conversations: Advanced LLM-powered responses with tool calling
🏠 Smart Home Integration: Direct Home Assistant control and monitoring
🔊 High-Quality TTS: Kokoro TTS for natural-sounding voice responses
📡 OpenAI-Compatible APIs: Drop-in replacement for OpenAI services
🔍 Web Search & Knowledge: Brave Search and WolframAlpha integration
🐳 Self-Contained: Fully containerized with Docker Compose
💻 Hardware Flexible: Support for multiple GPU configurations and LLM backends

📚 Comprehensive Documentation

For detailed guides, tutorials, and advanced configuration, visit our HavenCore Wiki:

Essential Guides

Getting Started - Complete setup walkthrough
Configuration Guide - Environment variables and settings
Architecture Overview - System design and components
API Reference - Complete API documentation

Integration & Development

Home Assistant Integration - Smart home setup
Tool Development - Creating custom tools and integrations
Development Guide - Contributing and extending HavenCore
Service Documentation - Individual service details

Support & Troubleshooting

Troubleshooting Guide - Common issues and solutions
FAQ - Frequently asked questions

💡 New to HavenCore? Start with the Getting Started Guide for a comprehensive walkthrough.

🚀 Quick Start

For experienced users who want to get started immediately:

# Clone the repository
git clone https://github.com/ThatMattCat/havencore.git
cd havencore

# Copy and configure environment
cp .env.tmpl .env
# Edit .env with your specific settings

# Start the services
docker compose up -d

# Access the web interface
open http://localhost:6002  # Agent UI

🏗️ Architecture

HavenCore is built as a microservices architecture using Docker containers. Each service handles a specific aspect of the AI assistant functionality:

Core Services

Service	Port	Purpose	API Endpoints
Nginx	80	API Gateway & Load Balancer	Routes to other services
Agent	6002, 6006	LLM Logic & Tool Calling	`/v1/chat/completions`
Speech-to-Text	6000, 6001, 5999	Audio Transcription	`/v1/audio/transcriptions`
Text-to-Speech	6003, 6004, 6005	Audio Generation	`/v1/audio/speech`
PostgreSQL	5432	Database & Conversation Storage	N/A (internal)
vLLM	8000	LLM Inference Backend	OpenAI-compatible API
LlamaCPP	8000*	Alternative LLM Backend	OpenAI-compatible API

*Only one LLM backend runs at a time

Data Flow

Voice Input: Edge device captures wake word and audio
Transcription: Speech-to-text converts audio to text
Processing: Agent processes text and determines actions
Tool Execution: Agent calls appropriate tools (Home Assistant, web search, etc.)
Response Generation: LLM generates natural language response
Audio Output: Text-to-speech converts response to audio
Playback: Edge device plays audio response

Supported Integrations

🏠 Home Assistant - Device control and state monitoring
🔍 Brave Search - Web search capabilities
🧮 WolframAlpha - Computational queries and factual data
🌤️ Weather API - Weather forecasts and conditions
📊 Grafana Loki - Centralized logging (optional)

📋 Prerequisites

Hardware Requirements

NVIDIA GPU: At least one CUDA-compatible GPU (RTX 3090 or better recommended)
RAM: 16GB+ system RAM (32GB+ recommended for larger models)
Storage: 50GB+ free space for models and data
Network: Stable internet connection for initial model downloads

Software Requirements

Operating System: Ubuntu 22.04 LTS (tested) or compatible Linux distribution
Docker: Version 20.10+ with Docker Compose V2
NVIDIA Container Toolkit: For GPU support in containers
Git: For cloning the repository

Optional Hardware

Edge Device: For voice activation (see HavenCore Edge)
ESP32-Box-3: Alternative edge device with built-in OpenAI integration

Installation of Prerequisites

Install Docker and Docker Compose

# Remove old Docker versions
sudo apt-get remove docker docker-engine docker.io containerd runc

# Install Docker
curl -fsSL https://get.docker.com -o get-docker.sh
sudo sh get-docker.sh
sudo usermod -aG docker $USER

# Verify installation
docker --version
docker compose version

Install NVIDIA Container Toolkit

# Configure production repository
curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg
curl -s -L https://nvidia.github.io/libnvidia-container/stable/deb/nvidia-container-toolkit.list | \
  sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' | \
  sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list

# Install the toolkit
sudo apt-get update
sudo apt-get install -y nvidia-container-toolkit

# Configure Docker daemon
sudo nvidia-ctk runtime configure --runtime=docker
sudo systemctl restart docker

# Test GPU access
docker run --rm --runtime=nvidia --gpus all nvidia/cuda:11.0.3-base-ubuntu20.04 nvidia-smi

🔧 Installation

1. Clone the Repository

git clone https://github.com/ThatMattCat/havencore.git
cd havencore

2. Configure Environment Variables

# Copy the template
cp .env.tmpl .env

# Edit the configuration file
nano .env  # or your preferred editor

3. Key Configuration Items

Edit the .env file with your specific settings:

Variable	Description	Example
`HOST_IP_ADDRESS`	IP address of your Docker host	`192.168.1.100`
`AGENT_NAME`	Name for your AI assistant	`Selene`
`HAOS_URL`	Home Assistant URL	`https://homeassistant.local:8123/api`
`HAOS_TOKEN`	Home Assistant long-lived token	`eyJ0eXAiOiJKV1QiLCJhbGc...`
`WEATHER_API_KEY`	WeatherAPI.com API key	`abc123def456...`
`BRAVE_SEARCH_API_KEY`	Brave Search API key (optional)	`BSA...`
`WOLFRAM_ALPHA_API_KEY`	WolframAlpha API key (optional)	`ABC123...`

4. Start the Services

# Start all services in detached mode
docker compose up -d

# View logs to monitor startup
docker compose logs -f

# Check service health
docker compose ps

5. Verify Installation

Once all services are running, you can verify the installation:

# Check service health
curl http://localhost/health

# Test text-to-speech
curl -X POST http://localhost/v1/audio/speech \
  -H "Content-Type: application/json" \
  -d '{"input": "Hello, HavenCore is working!", "model": "tts-1", "voice": "alloy"}'

# Test chat completion
curl -X POST http://localhost/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer 1234" \
  -d '{"model": "gpt-3.5-turbo", "messages": [{"role": "user", "content": "Hello!"}]}'

⚙️ Configuration

LLM Backend Selection

HavenCore supports multiple LLM backends. Choose one by modifying compose.yaml:

Option 1: vLLM (Default)

Best for high-throughput inference
Supports tensor parallelism across multiple GPUs
Current model: TechxGenus/Mistral-Large-Instruct-2411-AWQ

Option 2: LlamaCPP

More memory efficient
Better for smaller deployments
Supports draft model speculative decoding

To switch backends, comment out the unwanted service in compose.yaml.

GPU Configuration

Edit these environment variables in .env to configure GPU usage:

TTS_DEVICE="cuda:0"  # GPU for text-to-speech
STT_DEVICE="0"       # GPU for speech-to-text

For multi-GPU setups, modify the LLM service configuration in compose.yaml:

vLLM: Adjust the -tp parameter for tensor parallelism
LlamaCPP: Modify the -dev parameter for device selection

Service-Specific Configuration

Home Assistant Integration

Generate a long-lived access token in Home Assistant:
- Profile → Security → Long-lived access tokens

Add the token to your .env file:

HAOS_URL="https://your-homeassistant.local:8123/api"
HAOS_TOKEN="your_long_lived_token_here"

External API Keys

Configure optional services by adding API keys to .env:

# Weather forecasts
WEATHER_API_KEY="your_weatherapi_key"

# Web search capabilities
BRAVE_SEARCH_API_KEY="your_brave_search_key"

# Computational queries
WOLFRAM_ALPHA_API_KEY="your_wolfram_key"

📱 Usage

Web Interface Access

Agent Interface: http://localhost:6002 - Interactive chat interface
Text-to-Speech: http://localhost:6004 - TTS testing interface
System Health: http://localhost - Nginx status page

Edge Device Integration

ESP32-Box-3 Setup

Flash with Espressif's ChatGPT example code

Configure WiFi and endpoint:

API Endpoint: http://YOUR_HOST_IP/v1/
API Key: 1234 (or your configured key)

Custom Edge Device

Use the HavenCore Edge project for Raspberry Pi integration.

Voice Interaction Workflow

Wake Word: Edge device detects activation phrase
Recording: Captures audio until silence detection
Transcription: Converts speech to text via Whisper
Processing: Agent analyzes request and executes tools
Response: Generates natural language response
Synthesis: Converts text to speech via Kokoro TTS
Playback: Edge device plays audio response

Example Voice Commands

"Turn on the living room lights"
"What's the weather like tomorrow?"
"Set the thermostat to 72 degrees"
"What time is it in Tokyo?"
"Search for local restaurants"

📚 API Reference

HavenCore provides OpenAI-compatible APIs accessible through the Nginx gateway.

Chat Completions

Endpoint: POST /v1/chat/completions

curl -X POST http://localhost/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer 1234" \
  -d '{
    "model": "gpt-3.5-turbo",
    "messages": [
      {"role": "user", "content": "Turn on the kitchen lights"}
    ]
  }'

Speech Synthesis

Endpoint: POST /v1/audio/speech

curl -X POST http://localhost/v1/audio/speech \
  -H "Content-Type: application/json" \
  -d '{
    "input": "Hello, this is a test message",
    "model": "tts-1",
    "voice": "alloy",
    "response_format": "wav"
  }' \
  --output speech.wav

Audio Transcription

Endpoint: POST /v1/audio/transcriptions

curl -X POST http://localhost/v1/audio/transcriptions \
  -H "Content-Type: multipart/form-data" \
  -F "[email protected]" \
  -F "model=whisper-1"

Service Health Checks

Individual service health endpoints:

Agent: GET http://localhost:6002/
Text-to-Speech: GET http://localhost:6005/health
vLLM: GET http://localhost:8000/v1/models

🔧 Troubleshooting

Quick Diagnostics

Services Not Starting

# Check Docker and NVIDIA runtime
docker run --rm --gpus all nvidia/cuda:11.0.3-base-ubuntu20.04 nvidia-smi

# Verify GPU access
nvidia-smi

# Check service logs
docker compose logs [service_name]

Out of Memory Errors

Reduce model size in vLLM configuration
Adjust --gpu-memory-utilization parameter
Switch to LlamaCPP backend for lower memory usage

Model Download Issues

# Pre-download models manually
huggingface-cli download TechxGenus/Mistral-Large-Instruct-2411-AWQ

# Check Hugging Face token
echo $HF_HUB_TOKEN

Comprehensive Troubleshooting

For detailed troubleshooting guides covering installation issues, runtime problems, performance optimization, and more, see our Troubleshooting Guide.

Also helpful:

FAQ - Common questions and answers
Configuration Guide - Environment variable issues
Service Documentation - Service-specific debugging

🤝 Contributing

We welcome contributions from the community! Whether you're fixing bugs, adding features, improving documentation, or sharing ideas, your help makes HavenCore better for everyone.

Quick Contributing Guide

Fork and clone the repository
Set up development environment - see Development Guide
Create a feature branch for your changes
Make your improvements with proper testing
Submit a pull request with clear description

Areas for Contribution

📝 Documentation: Improve setup guides and API documentation
🐛 Bug Fixes: Fix issues and improve stability
✨ Features: Add new integrations and capabilities
🧪 Testing: Add automated tests and validation
🎨 UI/UX: Improve web interfaces and user experience
🔧 DevOps: Improve deployment and configuration management

Detailed Contributing Information

For comprehensive development setup, coding standards, testing guidelines, and contribution workflows, see our Development Guide.

Key Resources:

We appreciate all contributions and will review pull requests promptly!

Submitting Changes

Test your changes thoroughly
Update documentation if needed
Commit with clear, descriptive messages
Push to your fork and create a pull request

Reporting Issues

When reporting bugs, please include:

System specifications (OS, GPU, RAM)
Docker and NVIDIA runtime versions
Complete error logs
Steps to reproduce the issue

📄 License

This project is licensed under the GNU Lesser General Public License v2.1 (LGPL-2.1). See the LICENSE file for details.

Key Points

✅ Use: Free to use for personal and commercial projects
✅ Modify: Modify the source code for your needs
✅ Distribute: Share your modifications under the same license
⚠️ Copyleft: Derivative works must use LGPL-compatible licenses

🙏 Acknowledgments

Kokoro TTS - High-quality text-to-speech synthesis
Faster Whisper - Efficient speech recognition
vLLM - High-performance LLM inference
Home Assistant - Open-source smart home platform
Mistral AI - Advanced language models

📞 Support

📚 Documentation: Check this README and service-specific docs
🐛 Issues: GitHub Issues
💬 Discussions: GitHub Discussions
🔗 Edge Device: HavenCore Edge Repository

Made with ❤️ for the self-hosting and smart home communities

Name		Name	Last commit message	Last commit date
Latest commit History 101 Commits
.github		.github
docs		docs
services		services
shared		shared
.env.tmpl		.env.tmpl
.gitignore		.gitignore
LICENSE		LICENSE
README.md		README.md
compose.yaml		compose.yaml

License

ThatMattCat/havencore

Folders and files

Latest commit

History

Repository files navigation