-
Notifications
You must be signed in to change notification settings - Fork 3k
Blaze Livekit plugin integration #5050
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
HoangPN711
wants to merge
13
commits into
livekit:main
Choose a base branch
from
Actable-AI:feat/blaze-voicebot-plugin
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
13 commits
Select commit
Hold shift + click to select a range
d837f02
add blaze voicebot plugin
fpt-hoangpn2 1ec3883
fix comment review
fpt-hoangpn2 ebd97e9
blaze: align python LLM route and query params with voicebot-call API
fpt-hoangpn2 6c99127
blaze: register plugin and fix system role check in LLM conversion
fpt-hoangpn2 9360552
blaze(py): fix python compatibility, timeout precedence, and tool-mod…
fpt-hoangpn2 d55cfd9
fix(blaze-python): align retry/error handling with framework and reso…
fpt-hoangpn2 a0955d9
fix(blaze-python): handle developer role in LLM context and correct T…
fpt-hoangpn2 855972e
fix(blaze-python): ignore null SSE content chunks in llm stream extra…
fpt-hoangpn2 1b3ef2f
fix(blaze-python): flush output emitter after streaming tts chunks
fpt-hoangpn2 17b68d4
fix(blaze-python): use BLAZE_API_TOKEN via api_token config field
fpt-hoangpn2 2c1493d
fix(blaze-python): add postponed annotations in _utils for Python 3.9…
fpt-hoangpn2 71adf3a
blaze fix: initialize emitter before empty-text early return in tts s…
fpt-hoangpn2 19e0fcc
blaze-fix: wrap unexpected httpx failures as APIConnectionError for r…
fpt-hoangpn2 File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,30 @@ | ||
| # livekit-plugins-blaze | ||
|
|
||
| Agent Framework plugin for [Blaze AI](https://blaze.ai) services: | ||
|
|
||
| - **STT** (Speech-to-Text) via `POST /v1/stt/transcribe` | ||
| - **TTS** (Text-to-Speech) via `POST /v1/tts/realtime` (streaming PCM) | ||
| - **LLM** (Conversational AI) via `POST /voicebot/{bot_id}/chat-conversion?stream=true` (SSE) | ||
|
|
||
| ## Installation | ||
|
|
||
| ```bash | ||
| pip install livekit-plugins-blaze | ||
| ``` | ||
|
|
||
| ## Usage | ||
|
|
||
| ```python | ||
| from livekit.plugins import blaze | ||
|
|
||
| stt = blaze.STT(language="vi") | ||
| tts = blaze.TTS(speaker_id="speaker-1") | ||
| llm = blaze.LLM(bot_id="my-chatbot") | ||
| ``` | ||
|
|
||
| ## Environment Variables | ||
|
|
||
| | Variable | Description | Default | | ||
| |---|---|---| | ||
| | `BLAZE_API_URL` | Base URL for Blaze API gateway | `https://api.blaze.vn` | | ||
| | `BLAZE_API_TOKEN` | Bearer token for authentication | | |
67 changes: 67 additions & 0 deletions
67
livekit-plugins/livekit-plugins-blaze/livekit/plugins/blaze/__init__.py
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,67 @@ | ||
| """ | ||
| livekit-plugins-blaze | ||
|
|
||
| LiveKit Agent Framework plugin for Blaze AI services (STT, TTS, LLM). | ||
|
|
||
| This package provides LiveKit agent plugins that interface with Blaze's | ||
| speech-to-text, text-to-speech, and conversational AI services. | ||
|
|
||
| Example: | ||
| >>> from livekit.plugins import blaze | ||
| >>> | ||
| >>> # Create plugins with environment variables (BLAZE_*) | ||
| >>> stt = blaze.STT(language="vi") | ||
| >>> tts = blaze.TTS(speaker_id="speaker-1") | ||
| >>> llm = blaze.LLM(bot_id="my-chatbot") | ||
| >>> | ||
| >>> # Or use shared configuration | ||
| >>> config = blaze.BlazeConfig( | ||
| ... api_url="https://api.example.com", | ||
| ... auth_token="my-token", | ||
| ... ) | ||
| >>> stt = blaze.STT(config=config) | ||
| >>> tts = blaze.TTS(config=config, speaker_id="custom-voice") | ||
| >>> llm = blaze.LLM(config=config, bot_id="enterprise-bot") | ||
|
|
||
| Environment Variables: | ||
| BLAZE_API_URL: Base URL for Blaze API gateway | ||
| BLAZE_API_TOKEN: Bearer token for API authentication | ||
| """ | ||
|
|
||
| from livekit.agents import Plugin | ||
|
|
||
| from ._config import BlazeConfig | ||
| from .llm import LLM, LLMStream | ||
| from .log import logger | ||
| from .stt import STT | ||
| from .tts import TTS | ||
| from .version import __version__ | ||
|
|
||
| __all__ = [ | ||
| # Version | ||
| "__version__", | ||
| # Configuration | ||
| "BlazeConfig", | ||
| # Plugins | ||
| "STT", | ||
| "TTS", | ||
| "LLM", | ||
| "LLMStream", | ||
| ] | ||
|
|
||
|
|
||
| class BlazePlugin(Plugin): | ||
| def __init__(self) -> None: | ||
| super().__init__(__name__, __version__, __package__, logger) | ||
|
|
||
|
|
||
| Plugin.register_plugin(BlazePlugin()) | ||
|
|
||
| # Cleanup docs of unexported modules | ||
| _module = dir() | ||
| NOT_IN_ALL = [m for m in _module if m not in __all__] | ||
|
|
||
| __pdoc__ = {} | ||
|
|
||
| for n in NOT_IN_ALL: | ||
| __pdoc__[n] = False | ||
70 changes: 70 additions & 0 deletions
70
livekit-plugins/livekit-plugins-blaze/livekit/plugins/blaze/_config.py
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,70 @@ | ||
| """ | ||
| Blaze Configuration Module | ||
|
|
||
| Provides centralized configuration for Blaze services using Pydantic settings. | ||
| Configuration can be provided via environment variables with BLAZE_ prefix. | ||
| """ | ||
|
|
||
| from pydantic import Field | ||
| from pydantic_settings import BaseSettings | ||
|
|
||
|
|
||
| class BlazeConfig(BaseSettings): | ||
| """ | ||
| Configuration for Blaze AI services. | ||
|
|
||
| All services (STT, TTS, LLM) route through a single gateway URL. | ||
| Service-specific configuration (language, speaker, etc.) comes from the | ||
| voicebot ID and is passed as constructor arguments to each plugin. | ||
|
|
||
| Environment Variables: | ||
| BLAZE_API_URL: Base URL for Blaze API gateway | ||
| BLAZE_API_TOKEN: Bearer token for API authentication | ||
| BLAZE_STT_TIMEOUT: STT request timeout in seconds | ||
| BLAZE_TTS_TIMEOUT: TTS request timeout in seconds | ||
| BLAZE_LLM_TIMEOUT: LLM request timeout in seconds | ||
|
|
||
| Example: | ||
| >>> from livekit.plugins.blaze import BlazeConfig | ||
| >>> | ||
| >>> # Load from environment variables | ||
| >>> config = BlazeConfig() | ||
| >>> | ||
| >>> # Or provide explicit values | ||
| >>> config = BlazeConfig( | ||
| ... api_url="https://api.blaze.vn", | ||
| ... api_token="my-token", | ||
| ... ) | ||
| """ | ||
|
|
||
| # Service URL | ||
| api_url: str = Field( | ||
| default="https://api.blaze.vn", | ||
| description="Base URL for Blaze API gateway", | ||
| ) | ||
|
|
||
| # Authentication | ||
| api_token: str = Field( | ||
| default="", | ||
| description="Bearer token for API authentication", | ||
| ) | ||
|
|
||
| # Timeouts | ||
| stt_timeout: float = Field( | ||
| default=30.0, | ||
| description="STT request timeout in seconds", | ||
| ) | ||
| tts_timeout: float = Field( | ||
| default=60.0, | ||
| description="TTS request timeout in seconds", | ||
| ) | ||
| llm_timeout: float = Field( | ||
| default=60.0, | ||
| description="LLM request timeout in seconds", | ||
| ) | ||
|
|
||
| model_config = { | ||
| "env_prefix": "BLAZE_", | ||
| "env_file": ".env", | ||
| "extra": "ignore", | ||
| } |
92 changes: 92 additions & 0 deletions
92
livekit-plugins/livekit-plugins-blaze/livekit/plugins/blaze/_utils.py
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,92 @@ | ||
| """ | ||
| Internal utilities for livekit-plugins-blaze. | ||
|
|
||
| This module contains helper functions used by the plugin implementations. | ||
| """ | ||
|
|
||
| from __future__ import annotations | ||
|
|
||
| import io | ||
| import struct | ||
|
|
||
|
|
||
| def convert_pcm_to_wav( | ||
| pcm_data: bytes, | ||
| sample_rate: int = 16000, | ||
| channels: int = 1, | ||
| bits_per_sample: int = 16, | ||
| ) -> bytes: | ||
| """ | ||
| Convert raw PCM audio data to WAV format with proper headers. | ||
|
|
||
| Args: | ||
| pcm_data: Raw PCM audio data (16-bit signed integers, little-endian) | ||
| sample_rate: Audio sample rate in Hz (default: 16000) | ||
| channels: Number of audio channels (default: 1 for mono) | ||
| bits_per_sample: Bits per sample (default: 16) | ||
|
|
||
| Returns: | ||
| WAV format audio data with RIFF header, fmt chunk, and data chunk | ||
| """ | ||
| pcm_size = len(pcm_data) | ||
|
|
||
| # Calculate WAV file structure | ||
| # "WAVE" + fmt chunk (24 bytes) + data chunk header (8 bytes) + data | ||
| file_size = 4 + 24 + 8 + pcm_size | ||
|
|
||
| wav_buffer = io.BytesIO() | ||
|
|
||
| # RIFF header | ||
| wav_buffer.write(b"RIFF") | ||
| wav_buffer.write(struct.pack("<I", file_size)) | ||
| wav_buffer.write(b"WAVE") | ||
|
|
||
| # fmt chunk | ||
| wav_buffer.write(b"fmt ") | ||
| wav_buffer.write(struct.pack("<I", 16)) # Chunk size (16 bytes for PCM) | ||
| wav_buffer.write(struct.pack("<H", 1)) # Audio format (1 = PCM) | ||
| wav_buffer.write(struct.pack("<H", channels)) | ||
| wav_buffer.write(struct.pack("<I", sample_rate)) | ||
| byte_rate = sample_rate * channels * (bits_per_sample // 8) | ||
| wav_buffer.write(struct.pack("<I", byte_rate)) | ||
| block_align = channels * (bits_per_sample // 8) | ||
| wav_buffer.write(struct.pack("<H", block_align)) | ||
| wav_buffer.write(struct.pack("<H", bits_per_sample)) | ||
|
|
||
| # data chunk | ||
| wav_buffer.write(b"data") | ||
| wav_buffer.write(struct.pack("<I", pcm_size)) | ||
| wav_buffer.write(pcm_data) | ||
|
|
||
| return wav_buffer.getvalue() | ||
|
|
||
|
|
||
| def apply_normalization_rules( | ||
| text: str, | ||
| rules: dict[str, str] | None, | ||
HoangPN711 marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| ) -> str: | ||
| """ | ||
| Apply text normalization rules. | ||
|
|
||
| Performs simple string replacement based on the provided rules dictionary. | ||
| Matching is case-sensitive. | ||
|
|
||
| Args: | ||
| text: Input text to normalize | ||
| rules: Dictionary mapping patterns to their replacements. | ||
| If None or empty, returns original text. | ||
|
|
||
| Returns: | ||
| Text with all matching patterns replaced | ||
| """ | ||
| if not rules: | ||
| return text | ||
|
|
||
| # Apply longer patterns first for more deterministic results. | ||
| # Example: {"USD": "...", "$": "..."} should replace "USD" before "$". | ||
| result = text | ||
| for pattern, replacement in sorted(rules.items(), key=lambda kv: len(kv[0]), reverse=True): | ||
| if not pattern: | ||
| continue | ||
| result = result.replace(pattern, replacement) | ||
| return result | ||
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.