Feature: Session‑based Code Interpreter with Persistent State

[JakobStadlhuber]

Title: Feature: Session‑based Code Interpreter with Persistent State

Is your feature request related to a problem? Please describe.

• Multi‑step workflows: Today, each call to the interpreter is stateless, so variables, installed packages, and files are lost between calls.
• Re-initialization cost: Re-importing libraries, re-downloading datasets, and re-establishing context slows iteration.
• Brittle context passing: Manually re-sending prior outputs or code increases token usage and error risk.

Describe the solution you'd like

• Durable sessions: A session ID that pins execution to an isolated runtime where variables, files, and package state persist across calls.
• Session lifecycle: Create, resume, pause, checkpoint, restore, reset, and close; configurable TTL and resource limits.
• State persistence: Persist working directory, installed packages/cache, environment variables, and execution history.
• Observability & control: Streaming stdout/stderr, per-call metadata, resource usage, and session‑scoped logs.
• Portability: Export session as archive (code + env + files) for reproducibility; optional snapshot/restore.
• Runtimes: Python first; R/JS as follow‑ups, matching repository goals; kernel-like interactive behavior.

Describe alternatives you've considered

• Pass all context every call: Increases token usage, latency, and failure risk; doesn’t persist file system or package state.
• External storage + reload: Requires manual synchronization, is brittle, and still redoes environment setup each call.
• Long-running single job: Resource‑heavy, harder to isolate/secure, and fragile across timeouts or outages.

Additional context

• Alignment with open issues: Related to Feature Request: Interactive Python Kernel #46 (Interactive Python Kernel), Feature Request: API Server for Other Languages with Stateful Container Manager #47 (API Server with stateful manager), Feature Request: Security Presets #45 (Security Presets), Feature Request: Timeout Sandbox #43 (Firecracker backend), and Feature Request: Add Apple Container Backend Support for Enhanced Security on macOS #65 (Apple Container Backend on macOS).
• Security model: Per‑session isolation, quotas, timeouts, network egress policy, secrets via session‑scoped mounts, and audit trail.
• Implementation sketch: Session Manager service + backend orchestrator (k8s pods with PVCs; Firecracker/macos backends), checkpointing/snapshots, token‑scoped auth, idle cleanup.
• UX ideas: “New Session” button, session list with status/TTL/resources, quick actions (Pause/Restore/Reset), and export/archive.

Acceptance criteria

• Persistence: Variables, files, and installed packages remain available across multiple execute calls within the same session.
• Lifecycle controls: Create, resume, pause, reset, checkpoint, restore, and close function reliably with clear statuses.
• Reproducibility: Export/import yields the same environment and results on a fresh backend.
• Performance: Time‑to‑first‑execute ≤ N seconds; subsequent executes are faster due to cached state.
• Safety: Enforced isolation, quotas, timeouts, and configurable egress by preset; sessions auto‑clean after TTL.

[github-actions]

This GitHub issue proposes a feature to enable session-based code interpretation with persistent state in order to streamline multi-step workflows, address re-initialization costs, and reduce brittle context passing in current stateless execution. The suggested solution involves durable sessions tied to a session ID, which would preserve variables, files, installed packages, and execution context across multiple calls. Users would have access to session lifecycle management (e.g., create, resume, checkpoint, reset) and additional features such as state persistence, observability, exportability, and support for Python initially (with potential extensions to R and JavaScript). The feature aligns with multiple open issues and includes considerations for security, resource management, and user experience, such as session-specific isolation, configurable quotas, and a user-friendly interface. Acceptance criteria focus on persistence, lifecycle reliability, reproducibility across environments, performance optimization, and adherence to security measures.

[vndee]

Hey @JakobStadlhuber, thank you for this detailed feature request! I wanted to let you know that llm-sandbox already has this core functionality implemented through the InteractiveSandboxSession class.

The InteractiveSandboxSession provides persistent state across multiple code executions using an IPython kernel. Here's what's currently supported:

• Persistent variables, functions, and classes across multiple run() calls
• Persistent file system within the session working directory
• Persistent installed packages using %pip install within the session
• Multi-backend support: Docker, Podman, and Kubernetes
• IPython magic commands: %who, %pwd, !shell, %%timeit, etc.
• Execution history with configurable history size
• Timeout management at both cell and session levels
• Memory limits via max_memory configuration
• Streaming stdout/stderr with exit codes

Example Usage

from llm_sandbox import InteractiveSandboxSession

with InteractiveSandboxSession(lang="python") as session:
    # Variables persist across calls
    session.run("x = 42")
    result = session.run("print(x)")  # Outputs: 42
    
    # Install and use packages
    session.run("%pip install pandas numpy")
    session.run("import pandas as pd")
    
    # Define functions that persist
    session.run("def add(a, b): return a + b")
    result = session.run("print(add(10, 20))")  # Outputs: 30
    
    # Use IPython magic
    result = session.run("%who")  # List all variables

Documentation & Examples

• Full Documentation: docs/interactive-sessions.md
• Working Examples:
  • examples/interactive_session_demo.py
  • examples/interactive_session_kubernetes.py
  • examples/interactive_session_podman.py

While the core persistent execution is available, some advanced features from your request are not yet supported:

• Session registry with IDs: Currently sessions only exist within context manager lifetime; no way to list/reconnect to sessions
• Pause/Resume: Container pause capability not implemented
• Checkpoint/Restore: State snapshots not available
• Reset API: Can manually reset via %reset -f IPython magic, but no dedicated API method
• Export/Import: No session archive export for reproducibility
• Multi-language support: Currently Python/IPython only (R and JavaScript kernels not implemented)
• Advanced monitoring: Per-session resource usage tracking not exposed
• Network egress policies: Session-scoped network controls not implemented
• Audit trail: Session operation logging not implemented

Next Steps

We will consider adding the advanced features (session IDs, pause/resume, checkpoint/restore, export/import, multi-language support) in future releases when time permits. These would be valuable additions, but the core stateful execution capability you requested is already functional and available today.

Please try out InteractiveSandboxSession and let us know if it meets your needs or if you have specific use cases that require the advanced features listed above. Your feedback would help us prioritize which enhancements to implement first.

Thanks again for the detailed proposal!