Skip to content

fix: revert agent-server Docker image to Python 3.12#1910

Open
neubig wants to merge 1 commit intomainfrom
fix/agent-server-python312-libtmux
Open

fix: revert agent-server Docker image to Python 3.12#1910
neubig wants to merge 1 commit intomainfrom
fix/agent-server-python312-libtmux

Conversation

@neubig
Copy link
Contributor

@neubig neubig commented Feb 4, 2026

Summary

This PR fixes the Docker/Apptainer sandboxed server example tests failing in CI (issue #1886).

Root Cause

The CI failures were caused by the Python 3.12 → 3.13 upgrade in commit 42cbff3 (merged Jan 27, 2026).

When running the agent-server Docker image built with Python 3.13:

  • PyInstaller + libtmux + Python 3.13 has a compatibility issue
  • tmux sessions ARE created successfully (verified via tmux list-sessions)
  • But libtmux fails to find them immediately after creation
  • Results in: TmuxObjectDoesNotExist: Could not find object

Evidence

I tested two Docker images side-by-side:

  • Working: 10fff69-python (built Jan 14, 2026 with Python 3.12)
  • Broken: latest-python (built recently with Python 3.13)

Both images had identical PyInstaller environments (LD_LIBRARY_PATH, _MEI* directories), but:

  • Python 3.12 container: tmux session creation works every time
  • Python 3.13 container: fails with "Failed to create tmux session after 3 attempts"

Changes

  1. .github/workflows/server.yml - Changed base_image from python3.13-nodejs22 to python3.12-nodejs22
  2. openhands-agent-server/openhands/agent_server/docker/Dockerfile - Changed FROM and uv python version to 3.12
  3. openhands-agent-server/openhands/agent_server/docker/build.py - Updated default base_image

Testing

Please tag this PR with test-examples to trigger the example tests and verify the fix.

Fixes #1886


Agent Server images for this PR

GHCR package: https://github.com/OpenHands/agent-sdk/pkgs/container/agent-server

Variants & Base Images

Variant Architectures Base Image Docs / Tags
java amd64, arm64 eclipse-temurin:17-jdk Link
python amd64, arm64 nikolaik/python-nodejs:python3.12-nodejs22 Link
golang amd64, arm64 golang:1.21-bookworm Link

Pull (multi-arch manifest)

# Each variant is a multi-arch manifest supporting both amd64 and arm64
docker pull ghcr.io/openhands/agent-server:2344553-python

Run

docker run -it --rm \
  -p 8000:8000 \
  --name agent-server-2344553-python \
  ghcr.io/openhands/agent-server:2344553-python

All tags pushed for this build

ghcr.io/openhands/agent-server:2344553-golang-amd64
ghcr.io/openhands/agent-server:2344553-golang_tag_1.21-bookworm-amd64
ghcr.io/openhands/agent-server:2344553-golang-arm64
ghcr.io/openhands/agent-server:2344553-golang_tag_1.21-bookworm-arm64
ghcr.io/openhands/agent-server:2344553-java-amd64
ghcr.io/openhands/agent-server:2344553-eclipse-temurin_tag_17-jdk-amd64
ghcr.io/openhands/agent-server:2344553-java-arm64
ghcr.io/openhands/agent-server:2344553-eclipse-temurin_tag_17-jdk-arm64
ghcr.io/openhands/agent-server:2344553-python-amd64
ghcr.io/openhands/agent-server:2344553-nikolaik_s_python-nodejs_tag_python3.12-nodejs22-amd64
ghcr.io/openhands/agent-server:2344553-python-arm64
ghcr.io/openhands/agent-server:2344553-nikolaik_s_python-nodejs_tag_python3.12-nodejs22-arm64
ghcr.io/openhands/agent-server:2344553-golang
ghcr.io/openhands/agent-server:2344553-java
ghcr.io/openhands/agent-server:2344553-python

About Multi-Architecture Support

  • Each variant tag (e.g., 2344553-python) is a multi-arch manifest supporting both amd64 and arm64
  • Docker automatically pulls the correct architecture for your platform
  • Individual architecture tags (e.g., 2344553-python-amd64) are also available if needed

Python 3.13 + PyInstaller + libtmux has a compatibility issue where
tmux sessions are created but libtmux cannot find them immediately
after creation, causing 'TmuxObjectDoesNotExist' errors.

This was introduced in commit 42cbff3 which upgraded from Python 3.12
to Python 3.13 for all CI workflows and Docker images.

Testing confirmed:
- Python 3.12 container (10fff69-python): Works correctly
- Python 3.13 container (latest-python): Fails with tmux session errors

Fixes #1886

Co-authored-by: openhands <[email protected]>
@neubig neubig added the test-examples Run all applicable "examples/" files. Expensive operation. label Feb 4, 2026
@github-actions
Copy link
Contributor

github-actions bot commented Feb 4, 2026

🔄 Running Examples with openhands/claude-haiku-4-5-20251001

Generated: 2026-02-04 19:57:17 UTC

Example Status Duration Cost
01_standalone_sdk/02_custom_tools.py ✅ PASS 22.2s $0.03
01_standalone_sdk/03_activate_skill.py ✅ PASS 18.5s $0.03
01_standalone_sdk/05_use_llm_registry.py ✅ PASS 11.8s $0.01
01_standalone_sdk/07_mcp_integration.py ✅ PASS 42.0s $0.02
01_standalone_sdk/09_pause_example.py ✅ PASS 15.9s $0.01
01_standalone_sdk/10_persistence.py ✅ PASS 30.9s $0.02
01_standalone_sdk/11_async.py ✅ PASS 31.1s $0.03
01_standalone_sdk/12_custom_secrets.py ✅ PASS 18.2s $0.01
01_standalone_sdk/13_get_llm_metrics.py ✅ PASS 20.0s $0.01
01_standalone_sdk/14_context_condenser.py ✅ PASS 6m 18s $0.77
01_standalone_sdk/17_image_input.py ✅ PASS 16.0s $0.02
01_standalone_sdk/18_send_message_while_processing.py ✅ PASS 22.8s $0.01
01_standalone_sdk/19_llm_routing.py ✅ PASS 15.3s $0.02
01_standalone_sdk/20_stuck_detector.py ✅ PASS 20.7s $0.02
01_standalone_sdk/21_generate_extraneous_conversation_costs.py ✅ PASS 9.5s $0.00
01_standalone_sdk/22_anthropic_thinking.py ✅ PASS 20.7s $0.01
01_standalone_sdk/23_responses_reasoning.py ✅ PASS 1m 39s $0.01
01_standalone_sdk/24_planning_agent_workflow.py ✅ PASS 51.8s $0.05
01_standalone_sdk/25_agent_delegation.py ✅ PASS 2m 11s $0.18
01_standalone_sdk/26_custom_visualizer.py ✅ PASS 27.0s $0.02
01_standalone_sdk/28_ask_agent_example.py ❌ FAIL
Exit code 1
16.5s --
01_standalone_sdk/29_llm_streaming.py ✅ PASS 38.5s $0.03
01_standalone_sdk/30_tom_agent.py ❌ FAIL
Exit code 1
2.6s --
01_standalone_sdk/31_iterative_refinement.py ✅ PASS 7m 0s $0.50
01_standalone_sdk/32_configurable_security_policy.py ✅ PASS 18.5s $0.02
01_standalone_sdk/34_critic_example.py ✅ PASS 2m 3s $0.00
02_remote_agent_server/01_convo_with_local_agent_server.py ✅ PASS 47.1s $0.03
02_remote_agent_server/02_convo_with_docker_sandboxed_server.py ✅ PASS 1m 27s $0.05
02_remote_agent_server/03_browser_use_with_docker_sandboxed_server.py ✅ PASS 44.9s $0.06
02_remote_agent_server/04_convo_with_api_sandboxed_server.py ✅ PASS 1m 34s $0.02
02_remote_agent_server/05_vscode_with_docker_sandboxed_server.py ❌ FAIL
Exit code 1
21.9s --
02_remote_agent_server/07_convo_with_cloud_workspace.py ✅ PASS 32.9s $0.02
02_remote_agent_server/08_convo_with_apptainer_sandboxed_server.py ✅ PASS 3m 10s $0.02
04_llm_specific_tools/01_gpt5_apply_patch_preset.py ✅ PASS 30.7s $0.04
04_llm_specific_tools/02_gemini_file_tools.py ✅ PASS 53.9s $0.07
05_skills_and_plugins/01_loading_agentskills/main.py ✅ PASS 15.6s $0.02
05_skills_and_plugins/02_loading_plugins/main.py ✅ PASS 10.4s $0.01

❌ Some tests failed

Total: 37 | Passed: 34 | Failed: 3 | Total Cost: $2.20

Failed examples:

  • examples/01_standalone_sdk/28_ask_agent_example.py: Exit code 1
  • examples/01_standalone_sdk/30_tom_agent.py: Exit code 1
  • examples/02_remote_agent_server/05_vscode_with_docker_sandboxed_server.py: Exit code 1

View full workflow run

@openhands-ai
Copy link

openhands-ai bot commented Feb 4, 2026

Looks like there are a few issues preventing this PR from being merged!

  • GitHub Actions are failing:
    • Run Examples Scripts

If you'd like me to help, just leave a comment, like

@OpenHands please fix the failing actions on PR #1910 at branch `fix/agent-server-python312-libtmux`

Feel free to include any additional details that might help me get this PR into a better state.

You can manage your notification settings

Copy link
Collaborator

@all-hands-bot all-hands-bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a reasonable workaround for the PyInstaller+libtmux+Python 3.13 compatibility issue. The changes are consistent and well-documented. I have a few suggestions for improvement around maintenance and future planning.

include:
# Python variant
# NOTE: Using Python 3.12 due to PyInstaller+libtmux compatibility
# issue with Python 3.13. See issue #1886 for details.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Suggestion: Consider adding a TODO or tracking link to monitor when this can be reverted.

Suggested change
# issue with Python 3.13. See issue #1886 for details.
# NOTE: Using Python 3.12 due to PyInstaller+libtmux compatibility
# issue with Python 3.13. See issue #1886 for details.
# TODO: Re-evaluate upgrading to Python 3.13 once upstream issue is resolved

This helps prevent the workaround from becoming permanent.


ARG BASE_IMAGE=nikolaik/python-nodejs:python3.13-nodejs22
# NOTE: Using Python 3.12 due to PyInstaller+libtmux compatibility issue
# with Python 3.13. See issue #1886 for details.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟠 Important: This NOTE comment is duplicated across 3 files (Dockerfile, build.py, and server.yml). Consider maintaining a single source of truth to prevent comment drift.

Options:

  1. Keep detailed explanation only in the issue tracker and use brief comments: # Python 3.12 required (issue #1886)
  2. Document the workaround in AGENTS.md and reference it from all files
  3. Add a linter rule to ensure consistency

Duplicate documentation often becomes stale when one location is updated but others are forgotten.

# We copy source + build a venv here for local dev and debugging.
####################################################################################
FROM python:3.13-bullseye AS builder
FROM python:3.12-bullseye AS builder
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Nit: Consider documenting Python version compatibility in a central location (e.g., README or AGENTS.md) for visibility. Users might wonder why 3.12 is used when 3.13 is available.

class BuildOptions(BaseModel):
base_image: str = Field(default="nikolaik/python-nodejs:python3.13-nodejs22")
# NOTE: Using Python 3.12 due to PyInstaller+libtmux compatibility issue
# with Python 3.13. See issue #1886 for details.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Suggestion: Minor inconsistency in comment format. In server.yml you wrote "See issue #1886 for details." but here just "See issue #1886.". Consider standardizing.

"--base-image",
default=_env("BASE_IMAGE", "nikolaik/python-nodejs:python3.13-nodejs22"),
# NOTE: Using Python 3.12 due to PyInstaller+libtmux compatibility issue
# with Python 3.13. See issue #1886.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Suggestion: Consider adding a comment explaining what Python 3.13 features/improvements are being sacrificed by this downgrade, or link to upstream issues tracking the PyInstaller/libtmux compatibility. This helps future maintainers understand the trade-offs and when to prioritize re-upgrading.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

test-examples Run all applicable "examples/" files. Expensive operation.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Docker/Apptainer sandboxed server example tests failing in CI

2 participants