Add stop hook for pre-commit, pytest, and CI validation

[xingyaoww]

Summary

This PR adds a project-level stop hook that validates code quality and CI status before allowing the agent to finish. The hook runs automatically when the agent attempts to stop/finish and can block the stop if any checks fail, providing detailed feedback to the agent about what went wrong.

Features

1. Pre-commit Validation

• Runs pre-commit run --all-files to check formatting, linting, type checking
• Captures and reports any issues found

2. Smart Test Running

• Detects changed files using git status --porcelain
• Maps changed files to appropriate test directories:
  • openhands-sdk/* → tests/sdk/
  • openhands-tools/* → tests/tools/
  • openhands-workspace/* → tests/workspace/
  • openhands-agent-server/* → tests/agent_server/
  • And more...
• Runs pytest only for affected packages

3. GitHub CI Status Checking

• Detects if the current commit has been pushed to GitHub
• Uses gh CLI (with fallback to curl API) to check CI status
• Waits up to 5 minutes for in-progress CI runs to complete
• Reports detailed failure information including:
  • Which checks failed
  • Link to GitHub Actions
  • Job step-level failure details

4. Blocking with Feedback

• Blocks agent from stopping (exit code 2) when any check fails
• Outputs structured JSON feedback:

{
  "decision": "deny",
  "reason": "Checks failed",
  "additionalContext": "## CI Failed\n\nThe following CI checks failed:\n- build-and-evaluate: failure\n..."
}

• Feedback is passed back to the agent via additionalContext field

Files Added

• .openhands/hooks.json - Hook configuration for the stop event
• .openhands/hooks/on_stop.sh - Shell script with validation logic

Checklist

• If the PR is changing/adding functionality, are there tests to reflect this?
  • The hook is configuration/tooling, not SDK functionality. The existing hooks test suite (tests/sdk/hooks/) validates the hook loading mechanism.
• If there is an example, have you run the example to make sure that it works?
  • Tested the hook manually with various scenarios (pre-commit pass/fail, CI checking with GITHUB_TOKEN)
• If there are instructions on how to run the code, have you followed the instructions and made sure that it works?
  • Yes, verified the hook loads correctly via HookConfig.load(working_dir='.')
• If the feature is significant enough to require documentation, is there a PR open on the OpenHands/docs repository with the same branch name?
  • This is internal tooling for the agent-sdk repo, not user-facing SDK functionality
• Is the github CI passing?
  • Waiting for CI to run on this PR

@xingyaoww can click here to continue refining the PR

---

Agent Server images for this PR

• GHCR package: https://github.com/OpenHands/agent-sdk/pkgs/container/agent-server

Variants & Base Images

Variant	Architectures	Base Image	Docs / Tags
java	amd64, arm64	eclipse-temurin:17-jdk	Link
python	amd64, arm64	nikolaik/python-nodejs:python3.13-nodejs22	Link
golang	amd64, arm64	golang:1.21-bookworm	Link

Pull (multi-arch manifest)

# Each variant is a multi-arch manifest supporting both amd64 and arm64
docker pull ghcr.io/openhands/agent-server:cf0c6c7-python

Run

docker run -it --rm \
  -p 8000:8000 \
  --name agent-server-cf0c6c7-python \
  ghcr.io/openhands/agent-server:cf0c6c7-python

All tags pushed for this build

ghcr.io/openhands/agent-server:cf0c6c7-golang-amd64
ghcr.io/openhands/agent-server:cf0c6c7-golang_tag_1.21-bookworm-amd64
ghcr.io/openhands/agent-server:cf0c6c7-golang-arm64
ghcr.io/openhands/agent-server:cf0c6c7-golang_tag_1.21-bookworm-arm64
ghcr.io/openhands/agent-server:cf0c6c7-java-amd64
ghcr.io/openhands/agent-server:cf0c6c7-eclipse-temurin_tag_17-jdk-amd64
ghcr.io/openhands/agent-server:cf0c6c7-java-arm64
ghcr.io/openhands/agent-server:cf0c6c7-eclipse-temurin_tag_17-jdk-arm64
ghcr.io/openhands/agent-server:cf0c6c7-python-amd64
ghcr.io/openhands/agent-server:cf0c6c7-nikolaik_s_python-nodejs_tag_python3.13-nodejs22-amd64
ghcr.io/openhands/agent-server:cf0c6c7-python-arm64
ghcr.io/openhands/agent-server:cf0c6c7-nikolaik_s_python-nodejs_tag_python3.13-nodejs22-arm64
ghcr.io/openhands/agent-server:cf0c6c7-golang
ghcr.io/openhands/agent-server:cf0c6c7-java
ghcr.io/openhands/agent-server:cf0c6c7-python

About Multi-Architecture Support

• Each variant tag (e.g., cf0c6c7-python) is a multi-arch manifest supporting both amd64 and arm64
• Docker automatically pulls the correct architecture for your platform
• Individual architecture tags (e.g., cf0c6c7-python-amd64) are also available if needed

[xingyaoww]

@OpenHands /codereview-roasted and tell me if the project hook is supported in OpenHands/OpenHands-CLI and OpenHands/OpenHands

[openhands-ai]

I'm on it! xingyaoww can track my progress at all-hands.dev

[openhands-ai]

Summary

I performed a roasted code review of PR #1878 "Add stop hook for pre-commit, pytest, and CI validation" and investigated project hook support across OpenHands repositories.

---

Code Review (Linus Torvalds Style)

🔴 Taste Rating: Needs Improvement

Critical Issues:

1. Fragile set implementation (Lines 56-60): Using regex string matching for deduplication will fail on similar paths
2. Hardcoded directory mappings (Lines 63-82): The case statement mapping source→test directories is a maintenance nightmare
3. Deep nesting (Lines 170-220): CI status checking has >3 levels of nesting - needs refactoring into functions
4. Inconsistent error handling (Line 14): Using set -o pipefail without set -e creates unpredictable behavior

Improvement Opportunities:

• Extract repeated uv detection into a reusable function
• Replace global string concatenation for issues with temp file or array
• Make curl fallback behavior consistent with gh CLI (both should wait for CI)
• Use gh repo view instead of brittle sed regex for parsing remote URLs

---

Project Hook Support Status

Repository	Support	Details
OpenHands/software-agent-sdk	✅ Fully Supported	HookConfig class supports .openhands/hooks.json with stop, pre_tool_use, post_tool_use, session_start, session_end, user_prompt_submit events
OpenHands/OpenHands-CLI	✅ Supported via SDK	Depends on openhands-sdk==1.10.0 which includes the hooks system
OpenHands/OpenHands	⚠️ Not Yet Implemented	Main repo has .openhands/ but only contains microagents/, pre-commit.sh, setup.sh. Issue #11943 discusses hooks system design. Agent controller is marked "LEGACY V0"

Bottom line: The stop hook will work when running via SDK or CLI, but won't be recognized by the legacy V0 OpenHands main application until V1 migration is complete.

---

No code changes were made - this was a review-only request.

View full conversation