Add Intelligent interruption handling

[Mishikasardana]

feat: Add intelligent interruption handling

Fixes an issue where the agent stops speaking when the user says simple listening words like “yeah” or “ok” during agent speech.

Changes

Added an InterruptionFilter to ignore backchanneling words

Integrated the filter into the agent’s interruption logic

Added configuration options to:

Enable or disable the filter

Customize the list of ignored words

All four test cases are passing

How it works

When the user speaks while the agent is talking, the filter checks:

Whether the agent is currently speaking

Whether the user’s input contains only backchanneling words

If both conditions are true, the interruption is ignored and the agent continues speaking.
Otherwise, the agent is interrupted as usual.

Summary by CodeRabbit
New Features

Interruption filter for voice agents that ignores common backchannel utterances while the agent is speaking; enabled by default.
Configuration

Custom ignore-word lists, environment variable support, case-sensitivity toggle, and runtime enable/disable controls; session option to enable/disable filter.
Documentation

New README and proof-of-test documenting behavior, examples, and usage.
Tests

Extensive unit tests covering behavior, edge cases, punctuation, case sensitivity, and runtime updates.
Public API

Filter and related voice exports added for integration.

Summary by CodeRabbit

Release Notes

• New Features

  • Introduced an interruption filter for voice agents to intelligently handle backchannel responses (common interjections like "yeah" and "ok") when the agent is speaking, preventing unintended interruptions.
  • Added configuration options to enable/disable the filter and customize ignore words via parameters or environment variables.

• Documentation

  • New guide documenting the interruption filter, usage examples, and customization options.

• Tests

  • Added comprehensive test coverage for the interruption filter functionality.

_{✏️ Tip: You can customize this high-level summary in your review settings.}

[coderabbitai]

Caution

Review failed

The pull request is closed.

📝 Walkthrough

Walkthrough

Introduces an InterruptionFilter feature to distinguish backchannel utterances from meaningful interruptions in voice agents. When the agent is speaking, common filler words like "yeah" and "ok" are ignored as interruptions while genuine interruptions are processed. Includes configuration options, type annotation modernization, documentation, tests, and integration with AgentSession and AgentActivity.

Changes

Cohort / File(s)	Summary
InterruptionFilter Core Implementation livekit-agents/livekit/agents/voice/filter.py, tests/filter.py	Introduces new InterruptionFilter class supporting customizable ignore words, case sensitivity toggle, enable/disable control, dynamic word management, and backchannel detection via normalization and word-set matching. Default ignore words include common conversational fillers.
Agent Session Configuration livekit-agents/livekit/agents/voice/agent_session.py	Extends AgentSessionOptions and AgentSession.__init__() with two new public fields: interruption_filter_enabled (bool, default True) and interruption_ignore_words (list[str] | None).
Interruption Handling Integration livekit-agents/livekit/agents/voice/agent_activity.py	Initializes _interruption_filter from session options and integrates filter logic into _interrupt_by_audio_activity() to consult should_ignore_interruption() before processing interruptions.
Public API Exports livekit-agents/livekit/agents/voice/__init__.py	Adds InterruptionFilter and TranscriptSynchronizer to module-level exports.
Type Annotation Modernization examples/bank-ivr/ivr_system_agent.py, examples/bank-ivr/mock_bank_service.py, livekit-agents/livekit/agents/cli/cli.py, livekit-agents/livekit/agents/voice/ivr/ivr_activity.py	Replaces Optional[T] with Python 3.10+ union syntax T | None across type hints and removes corresponding Optional imports.
Example Usage examples/voice_agents/basic_agent.py	Adds interruption_filter_enabled=True parameter to AgentSession instantiation.
Example Refactoring examples/voice_agents/llamaindex-rag/retrieval.py, examples/voice_agents/session_close_callback.py	Extracts string normalization and formatting into intermediate variables for clarity.
Documentation & Testing FILTER_README.md, test.py, proof.md	Introduces comprehensive filter documentation, unit tests covering edge cases (case sensitivity, punctuation, dynamic word management), and test execution proof.
Build Configuration pyproject.toml	Adds Ruff per-file linter ignore rules for examples/** and livekit-agents/livekit/agents/cli/** to suppress union-type and optional-related rule violations.

Sequence Diagram(s)

sequenceDiagram
    participant User
    participant AgentActivity
    participant InterruptionFilter
    participant Logger
    participant Decision

    User->>AgentActivity: Audio input detected
    AgentActivity->>AgentActivity: Extract transcript
    AgentActivity->>AgentActivity: Determine agent_is_speaking state
    AgentActivity->>InterruptionFilter: should_ignore_interruption(transcribed_text, agent_is_speaking)
    
    alt Filter Enabled & Agent Speaking
        InterruptionFilter->>InterruptionFilter: Normalize text (lowercase, remove punctuation)
        InterruptionFilter->>InterruptionFilter: Check if all words in ignore_words set
        alt All words are backchannels
            InterruptionFilter->>Logger: Log debug "Backchannel detected"
            InterruptionFilter-->>AgentActivity: return True
            AgentActivity->>Decision: Skip interruption handling
        else Contains non-backchannel words
            InterruptionFilter->>Logger: Log info "Real interruption"
            InterruptionFilter-->>AgentActivity: return False
            AgentActivity->>Decision: Process interruption normally
        end
    else Filter Disabled or Agent Silent
        InterruptionFilter-->>AgentActivity: return False
        AgentActivity->>Decision: Process interruption normally
    end

Loading

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Possibly related PRs

• interrupt the same speech handle #4536: Also modifies interruption handling in agent_activity.py via a local speech handle to avoid races—complementary to this PR's filter-based approach.

Suggested reviewers

• longcw

Poem

🐰 A whisper of "yeah" now filters clean,
No more false alarms when we convene!
Backchannel words slip softly by,
While real intent grabs Agent's eye. ✨
With union types and types so neat,
This interruption tale is quite complete!

✨ Finishing touches

• 📝 Generate docstrings

---

📜 Recent review details

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 80f2e33 and 008bde1.

📒 Files selected for processing (16)

• FILTER_README.md
• examples/bank-ivr/ivr_system_agent.py
• examples/bank-ivr/mock_bank_service.py
• examples/voice_agents/basic_agent.py
• examples/voice_agents/llamaindex-rag/retrieval.py
• examples/voice_agents/session_close_callback.py
• livekit-agents/livekit/agents/cli/cli.py
• livekit-agents/livekit/agents/voice/__init__.py
• livekit-agents/livekit/agents/voice/agent_activity.py
• livekit-agents/livekit/agents/voice/agent_session.py
• livekit-agents/livekit/agents/voice/filter.py
• livekit-agents/livekit/agents/voice/ivr/ivr_activity.py
• proof.md
• pyproject.toml
• test.py
• tests/filter.py

_{✏️ Tip: You can disable this entire section by setting review_details to false in your review settings.}

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}