feat(azure): Add Azure Voice Live Realtime API support

[007DXR]

Description

This PR adds support for Azure Voice Live, enabling real-time speech-to-speech conversations through the Azure plugin.

What's New

Azure Voice Live Integration

• New RealtimeModel class providing end-to-end speech-to-speech capabilities
• Full bidirectional audio streaming
• Server-side VAD (Voice Activity Detection) with configurable thresholds
• Automatic reconnection handling for connection resilience

Features

• Speech-to-Speech: Direct audio input/output without separate STT/TTS pipeline
• Function Calling: Built-in tool use for agentic workflows
• Multilingual Support: Works with Azure's multilingual neural voices (e.g., en-US-AvaMultilingualNeural)
• Interruption Handling: Graceful handling of user interruptions during responses
• Metrics Collection: Token usage and TTFT (Time to First Token) tracking
• Debug Mode: Optional audio saving per turn for debugging (save_audio_per_turn=True)

Usage

from livekit.agents import Agent, AgentSession
from livekit.plugins import azure

session = AgentSession(
    llm=azure.realtime.RealtimeModel(
        voice="en-US-AvaMultilingualNeural",
    )
)

await session.start(room=ctx.room, agent=Agent(instructions="You are helpful."))

Environment Variables

export AZURE_VOICELIVE_ENDPOINT=https://<region>.api.cognitive.microsoft.com/
export AZURE_VOICELIVE_API_KEY=<your-speech-key>

New Dependencies

• azure-ai-voicelive[aiohttp]>=1.0.0
• azure-identity>=1.15.0

---

[devin-ai-integration]

Devin Review found 2 potential issues.

View issues and 6 additional flags in Devin Review.

[devin-ai-integration]

Devin Review found 1 new potential issue.

View issue and 8 additional flags in Devin Review.

[devin-ai-integration]

Devin Review found 2 new potential issues.

View issues and 9 additional flags in Devin Review.

[devin-ai-integration]

🔴 Error event handling assumes event.error is a dict when it's likely an object

The error handling code will crash with AttributeError if event.error is an object rather than a dictionary.

Click to expand

Issue

At line 502, the code does:

error_msg = getattr(event, "error", {}).get("message", "Unknown error")

This assumes event.error is a dictionary with a get() method. However, looking at how other Azure SDK event attributes are accessed throughout the codebase, they are objects with attributes (e.g., event.response.id at line 514, event.session.id at line 440). If event.error is similarly an object (not a dict), calling .get() on it will raise an AttributeError.

Impact

When Azure returns an error event, the code will crash with AttributeError: 'SomeErrorClass' object has no attribute 'get', causing the event loop to fail and potentially disconnecting the session.

Expected Behavior

The error message should be extracted using getattr(event.error, "message", "Unknown error") to be consistent with how other event attributes are accessed.

Recommendation: Change to: error_msg = getattr(getattr(event, "error", None), "message", None) or "Unknown error" or use a consistent pattern like other event handlers: error_obj = getattr(event, "error", None); error_msg = getattr(error_obj, "message", "Unknown error") if error_obj else "Unknown error"

---

Was this helpful? React with 👍 or 👎 to provide feedback.

[devin-ai-integration]

🟡 Breaking change: responses module removed from package exports without deprecation

The responses module was replaced with realtime in the package exports, but the responses directory still exists with functional code.

Click to expand

Issue

In __init__.py, the import changed from:

from . import responses
__all__ = ["STT", "SpeechStream", "TTS", "responses", "__version__"]

to:

from . import realtime
__all__ = ["STT", "SpeechStream", "TTS", "realtime", "__version__"]

The responses directory (livekit-plugins/livekit-plugins-azure/livekit/plugins/azure/responses/) still exists with __init__.py and llm.py, but is no longer exported.

Impact

Existing users who import from livekit.plugins import azure; azure.responses or from livekit.plugins.azure import responses will get an AttributeError or ImportError after upgrading. This is a breaking API change that should either be documented or the responses module should continue to be exported alongside realtime.

Recommendation: Either: (1) Export both modules: from . import realtime, responses and update __all__ to include both, or (2) Remove the responses directory if it's deprecated, or (3) Document this as a breaking change in the PR description.

---

Was this helpful? React with 👍 or 👎 to provide feedback.