Question: Fixed participants with toggleable audio/video

[Viktoriagrg]

Feature Type

I cannot use LiveKit without it

Feature Description

Hi LiveKit team,

I’m exploring a participant management pattern and want to check whether it’s supported natively.

Goal / Use Case

I’d like to maintain a fixed set of participants in a room, each identified by a known identity, with the following behavior:

1. Participants exist in the room in a passive state (camera and microphone off)
2. When a user “joins,” their connection is mapped to one of these predefined participants, and their audio/video can be switched on
3. When the user “leaves” or disconnects, the participant remains in the room with audio/video switched off, ready for reuseThis approach would allow reusing participant identities rather than constantly creating/destroying participants.

Context

I noticed the set_participant / unset_participant methods in the room_io class and wondered if they could be used to implement this lifecycle. I’m not fully clear on their intended usage.

Questions

• Is this fixed-participant + toggleable media pattern possible with LiveKit today?
• Are set_participant / unset_participant intended for this kind of participant reuse?
• Or is the expected design that participant identities are always tied to a single active connection?

Looking forward to your guidance — thanks for any insights 🙏

Workarounds / Alternatives

No response

Additional Context

No response

[Viktoriagrg]

@chenghao-mou do you have any info on this?

[darshankparmar]

Hi @Viktoriagrg, yes this pattern is fully supported! The set_participant/unset_participant methods in RoomIO are designed exactly for dynamic participant switching.

Implementation

# User joins - activate their media
room_io.set_participant("user-001")

# User leaves - return to passive state
room_io.unset_participant()

The _ParticipantInputStream classes handle seamless transitions between participants. This allows reusing participant identities without creating/destroying them, exactly matching your use case.

[longcw]

here is an example to toggle the audio/video input.

When the user “leaves” or disconnects, the participant remains in the room with audio/video switched off, ready for reuse

you can make the user to join the room in the same participant identity, but I don't think there is a way to keep the participant remains in the room the the user disconnects.

[darshankparmar]

I don't think there is a way to keep the participant remains in the room the the user disconnects.

Thanks @longcw for the clarification! You're right

[davidzhao]

do you have a specific use case for this pattern? why would you want to build it this way?

[Viktoriagrg]

@davidzhao
Thanks for the question.

Our use case is to have multiple AI agents with distinct personalities, but only run one at a time for cost optimization. We want fixed “slot” identities (agent-1, agent-2, etc.) that stay connected to the room—similar to the Meet example—so the UI and participant list remain stable. Then we toggle the avatar, audio/video, and AgentSession on/off per slot. This creates the illusion that all agents are present, while only the active one is actually running.

We tried implementing this by creating separate remote participants for each switch, each with its own token for connecting to the room, but that leads to jarring transitions and doesn’t feel like the right approach for smooth switching. We can’t connect an avatar by identity; we can only disconnect a given participant slot and then attach an avatar. If we try to keep the avatar session alive with restricted participant permissions (false), it disconnects this slot after some time.

Looking at the Meet repo and existing examples, it seems this pattern may not be realized yet on the agents side, even though it feels like a very common and important multi-agent use case.

[Viktoriagrg]

@davidzhao @longcw @darshankparmar
I’ve prepared a minimal reproducible example, and it is not behaving as expected. I’ve reviewed the code and tried to debug it, but I’m unable to identify the cause. Any guidance or insights would be greatly appreciated.

import os
import asyncio
import json
from dataclasses import dataclass
from dotenv import load_dotenv

from livekit import rtc, agents
from livekit.api import (
    LiveKitAPI,
    AccessToken,
    VideoGrants,
    MuteRoomTrackRequest,
    UpdateParticipantRequest,
)
from livekit.plugins import bey


# -----------------------
# Env / Config
# -----------------------

load_dotenv(".env.local")

LIVEKIT_URL = os.environ["LIVEKIT_URL"]
API_KEY = os.environ["LIVEKIT_API_KEY"]
API_SECRET = os.environ["LIVEKIT_API_SECRET"]
ROOM_NAME = "demo-room"

EMPTY_SLOTS = ["user-slot-1", "user-slot-2", "user-slot-3"]


# -----------------------
# Models
# -----------------------

@dataclass
class SlotState:
    room: rtc.Room
    identity: str
    audio_source: rtc.AudioSource | None = None
    video_source: rtc.VideoSource | None = None
    audio_track_sid: str | None = None
    video_track_sid: str | None = None


@dataclass
class PersonaConfig:
    key: str
    display_name: str
    avatar_id: str


# -----------------------
# Avatar
# -----------------------

def create_bey_avatar(persona: PersonaConfig, slot_identity: str) -> bey.AvatarSession:
    return bey.AvatarSession(
        avatar_id=persona.avatar_id,
        avatar_participant_identity=slot_identity,
        avatar_participant_name=persona.display_name,
    )


# -----------------------
# Slot Manager
# -----------------------

class SlotManager:
    def __init__(self):
        self.slots: dict[str, SlotState] = {}
        self.active_avatars: dict[str, bey.AvatarSession] = {}
        self.lkapi = LiveKitAPI(LIVEKIT_URL, API_KEY, API_SECRET)

    async def register_slot(self, slot: SlotState):
        self.slots[slot.identity] = slot

    async def start_slot(self, slot_identity: str, avatar: bey.AvatarSession):
        slot = self.slots.get(slot_identity)
        if not slot:
            return

        agent_session = agents.AgentSession()

        await avatar.start(agent_session, room=slot.room)
        await agent_session.start(
            room=slot.room,
            agent=agents.Agent(instructions="Helper avatar assistant"),
        )

        self.active_avatars[slot_identity] = avatar

        await update_participant_metadata(
            ROOM_NAME,
            slot_identity,
            {
                "state": "active",
                "avatar_id": avatar.avatar_id,
            },
        )

    async def stop_slot(self, slot_identity: str):
        avatar = self.active_avatars.pop(slot_identity, None)
        if avatar:
            await avatar.stop()

        await update_participant_metadata(
            ROOM_NAME,
            slot_identity,
            {"state": "idle"},
        )


# -----------------------
# LiveKit helpers
# -----------------------

async def mute_track(
    room_name: str,
    identity: str,
    track_sid: str,
    muted: bool,
):
    lkapi = LiveKitAPI(LIVEKIT_URL, API_KEY, API_SECRET)
    await lkapi.room.mute_published_track(
        MuteRoomTrackRequest(
            room=room_name,
            identity=identity,
            track_sid=track_sid,
            muted=muted,
        )
    )
    await lkapi.aclose()


async def update_participant_metadata(
    room_name: str,
    identity: str,
    metadata: dict,
):
    lkapi = LiveKitAPI(LIVEKIT_URL, API_KEY, API_SECRET)
    await lkapi.room.update_participant(
        UpdateParticipantRequest(
            room=room_name,
            identity=identity,
            metadata=json.dumps(metadata),
        )
    )
    await lkapi.aclose()


# -----------------------
# Token
# -----------------------

def create_token(identity: str) -> str:
    return (
        AccessToken(API_KEY, API_SECRET)
        .with_identity(identity)
        .with_grants(
            VideoGrants(
                room_join=True,
                room=ROOM_NAME,
                can_publish=True,
                can_subscribe=True,
            )
        )
        .to_jwt()
    )


# -----------------------
# Command loop
# -----------------------

async def handle_commands(slot_manager: SlotManager):
    while True:
        cmd = input("> ").lower()

        slot = next((p for p in cmd.split() if "user-slot" in p), None)
        if not slot:
            continue

        if "start" in cmd:
            persona = PersonaConfig(
                key="helper",
                display_name="Helper Bot",
                avatar_id="7c9ca52f-d4f7-46e1-a4b8-0c8655857cc3",
            )
            avatar = create_bey_avatar(persona, slot)
            await slot_manager.start_slot(slot, avatar)

        elif "stop" in cmd:
            await slot_manager.stop_slot(slot)


# -----------------------
# Main
# -----------------------

async def main():
    slot_manager = SlotManager()

    for slot_identity in EMPTY_SLOTS:
        token = create_token(slot_identity)

        room = rtc.Room()
        await room.connect(LIVEKIT_URL, token)

        await slot_manager.register_slot(
            SlotState(
                room=room,
                identity=slot_identity,
            )
        )

    await handle_commands(slot_manager)


asyncio.run(main())

[darshankparmar]

@Viktoriagrg
Thanks for sharing the detailed use case and code! The issue is that you're creating multiple rtc.Room instances - one for each slot. LiveKit agents are designed to use a single room per session.

The Problem

# Creating separate rooms for each slot ❌
for slot_identity in EMPTY_SLOTS:
    room = rtc.Room()  # Separate room per slot
    await room.connect(LIVEKIT_URL, token)

Recommended Solution

Use a single room with RoomIO to manage participant switching:

# Single room with RoomIO ✅
room = rtc.Room()
await room.connect(LIVEKIT_URL, token)

session = AgentSession()
room_io = RoomIO(session, room=room)
await room_io.start()

# Switch between agent slots
room_io.set_participant("agent-1")  # Activate agent 1
room_io.set_participant("agent-2")  # Switch to agent 2

This approach maintains stable participant identities while only the active one publishes media, achieving your "fixed slots with toggleable media" goal.

[davidzhao]

thank you for sharing your use case.. if the main reason to do this is for UI purposes, then I'd recommend you to just use placeholders on the UI itself.. i.e. even tho the agent isn't in the room, use a placeholder slot for agent-1, 2, etc.. and when the actual agents show up, they'll replace the placeholders.

the framework itself isn't designed to support the backend switching use case you've described.