feat: Add .pr/ directory convention and auto-cleanup workflow #1764

jpshackelford · 2026-01-20T14:30:33Z

Summary

Adds a convention and automation for PR-specific artifacts (design docs, analysis notes, etc.) that should not be merged to main.

Closes #1763

Changes

GitHub Actions workflow (.github/workflows/pr-artifacts.yml):
- Posts a single notification comment when .pr/ directory is detected
- Auto-removes .pr/ directory when a reviewer approves (non-fork PRs)
- Check passes (green ✅) during development - no blocking errors
AGENTS.md documentation for the .pr/ directory convention

Convention

Use a .pr/ directory at the repository root for PR-specific artifacts:

.pr/
├── design.md
├── analysis.md
└── notes.md

How It Works

Step	What Happens
PR created with `.pr/`	Notification comment posted, checks pass (green)
PR approved	`.pr/` directory auto-removed via commit
PR merged	Clean - no `.pr/` artifacts in main

Fork PR Handling

PR Type	On Approval	Notes
Internal (same repo)	Auto-removes `.pr/`	Fully automated
Fork	Skips (no push access)	Manual removal required

Benefits

Clear convention for PR-specific artifacts
Non-blocking workflow - checks stay green during development
Single notification comment (not spammy)
Automatic cleanup for internal PRs
Graceful fallback for forks

Agent Server images for this PR

• GHCR package: https://github.com/OpenHands/agent-sdk/pkgs/container/agent-server

Variants & Base Images

Variant	Architectures	Base Image	Docs / Tags
java	amd64, arm64	`eclipse-temurin:17-jdk`	Link
python	amd64, arm64	`nikolaik/python-nodejs:python3.12-nodejs22`	Link
golang	amd64, arm64	`golang:1.21-bookworm`	Link

Pull (multi-arch manifest)

# Each variant is a multi-arch manifest supporting both amd64 and arm64
docker pull ghcr.io/openhands/agent-server:1cac8fc-python

Run

docker run -it --rm \
  -p 8000:8000 \
  --name agent-server-1cac8fc-python \
  ghcr.io/openhands/agent-server:1cac8fc-python

All tags pushed for this build

ghcr.io/openhands/agent-server:1cac8fc-golang-amd64
ghcr.io/openhands/agent-server:1cac8fc-golang_tag_1.21-bookworm-amd64
ghcr.io/openhands/agent-server:1cac8fc-golang-arm64
ghcr.io/openhands/agent-server:1cac8fc-golang_tag_1.21-bookworm-arm64
ghcr.io/openhands/agent-server:1cac8fc-java-amd64
ghcr.io/openhands/agent-server:1cac8fc-eclipse-temurin_tag_17-jdk-amd64
ghcr.io/openhands/agent-server:1cac8fc-java-arm64
ghcr.io/openhands/agent-server:1cac8fc-eclipse-temurin_tag_17-jdk-arm64
ghcr.io/openhands/agent-server:1cac8fc-python-amd64
ghcr.io/openhands/agent-server:1cac8fc-nikolaik_s_python-nodejs_tag_python3.12-nodejs22-amd64
ghcr.io/openhands/agent-server:1cac8fc-python-arm64
ghcr.io/openhands/agent-server:1cac8fc-nikolaik_s_python-nodejs_tag_python3.12-nodejs22-arm64
ghcr.io/openhands/agent-server:1cac8fc-golang
ghcr.io/openhands/agent-server:1cac8fc-java
ghcr.io/openhands/agent-server:1cac8fc-python

About Multi-Architecture Support

Each variant tag (e.g., 1cac8fc-python) is a multi-arch manifest supporting both amd64 and arm64
Docker automatically pulls the correct architecture for your platform
Individual architecture tags (e.g., 1cac8fc-python-amd64) are also available if needed

Adds a GitHub Actions workflow that: 1. Auto-removes .pr/ directory when a reviewer approves (non-fork PRs) 2. Blocks merge if .pr/ directory still exists (catches fork PRs) This establishes a convention for PR-specific artifacts (design docs, analysis notes) that should not be merged to main. Closes #1763 Co-authored-by: openhands <[email protected]>

- Add workflow_dispatch trigger for manual testing - Add .pr/test-artifact.md to verify blocking behavior Co-authored-by: openhands <[email protected]>

Co-authored-by: openhands <[email protected]>

all-hands-bot

Found several issues that should be addressed before merging. Most importantly: missing concurrency control and the test artifact will block merge until approval.

.github/workflows/pr-artifacts.yml

.pr/test-artifact.md

Co-authored-by: OpenHands Bot <[email protected]>

.github/workflows/pr-artifacts.yml

Co-authored-by: OpenHands Bot <[email protected]>

Document the .pr/ directory convention for storing PR-specific design docs and analysis notes that should not be merged to main. Co-authored-by: openhands <[email protected]>

Co-authored-by: openhands <[email protected]>

jpshackelford · 2026-01-20T15:11:36Z

Just realized this will mark every commit with failing check just because we have .pr and that isn't nice. Looking into merge queue or just adding notation as warning instead...

- Rename job to check-pr-artifacts - Emit warning instead of error when .pr/ exists - Remove exit 1 so checks pass (green) during development - Trust auto-cleanup on approval to remove .pr/ before merge Co-authored-by: openhands <[email protected]>

- Posts a comment to the PR conversation when .pr/ exists - Uses a hidden marker to ensure only ONE comment is ever created - Subsequent runs skip if comment already exists - Added pull-requests: write permission for commenting Co-authored-by: openhands <[email protected]>

github-actions · 2026-01-20T15:16:46Z

✅ PR Artifacts Cleaned Up

The .pr/ directory has been automatically removed.

- Add 'How It Works' section explaining the 3-step flow - Clarify that checks pass (green) during development - Remove reference to blocking check Co-authored-by: openhands <[email protected]>

all-hands-bot

Found several critical inconsistencies between the PR description and actual implementation. The workflow does not block merges as claimed.

.github/workflows/pr-artifacts.yml

.pr/test-artifact.md

AGENTS.md

.github/workflows/pr-artifacts.yml

.pr/test-artifact.md

- After auto-cleanup, update the notice comment to show cleanup completed - Change icon from 📁 to ✅ and update message - Added pull-requests: write permission to cleanup job Co-authored-by: openhands <[email protected]>

Co-authored-by: OpenHands Bot <[email protected]>

Co-authored-by: openhands <[email protected]>

openhands-ai · 2026-01-20T18:58:21Z

Looks like there are a few issues preventing this PR from being merged!

GitHub Actions are failing:
- PR Artifacts

If you'd like me to help, just leave a comment, like

@OpenHands please fix the failing actions on PR #1764 at branch `feat/pr-artifacts-workflow`

Feel free to include any additional details that might help me get this PR into a better state.

_{^{You can manage your notification settings}}

jpshackelford · 2026-01-20T19:01:44Z

@OpenHands -- looks like maybe we have bug in our GitHub workflow... Lets fix the workflow itself and test again with another approval rather than resolving the issue manually.

Run if [ -d ".pr" ]; then
rm '.pr/test-artifact.md'
[detached HEAD 31f5ef8] chore: Remove PR-only artifacts [automated]
1 file changed, 13 deletions(-)
delete mode 100644 .pr/test-artifact.md
fatal: You are not currently on a branch.
To push the history leading to the current (detached HEAD)
state now, use

git push origin HEAD:<name-of-remote-branch>

Error: Failed to push cleanup commit. Check branch protection rules.
Error: Process completed with exit code 1.

openhands-ai · 2026-01-20T19:01:54Z

I'm on it! jpshackelford can track my progress at all-hands.dev

The workflow was checking out the PR head SHA which puts Git in a detached HEAD state. This caused 'git push' to fail with: fatal: You are not currently on a branch. Changed to checkout the branch ref instead, which allows normal git push operations. Co-authored-by: openhands <[email protected]>

openhands-ai · 2026-01-20T19:03:35Z

Summary

I fixed the bug in the PR artifacts workflow that was causing the push to fail with "fatal: You are not currently on a branch."

Root Cause

The workflow was checking out the PR head SHA (github.event.pull_request.head.sha), which puts Git in a detached HEAD state. When in detached HEAD, git push doesn't know which branch to push to.

Fix

Changed the checkout to use the branch ref instead (github.event.pull_request.head.ref):

-                  ref: ${{ github.event.pull_request.head.sha }}
+                  ref: ${{ github.event.pull_request.head.ref }}

This ensures Git is on the actual branch, allowing normal git push operations.

Checklist

Fixed the workflow bug as requested
Change is minimal and focused (1 line change)
Committed and pushed to feat/pr-artifacts-workflow branch

The PR is now ready for another approval to test the fix.

View full conversation

Retest the approval flow automation

Commits made with GITHUB_TOKEN don't trigger subsequent workflow runs (to prevent infinite loops). This caused the PR checks UI to get stuck showing only the Vercel check after the automated cleanup commit. Switch to ALLHANDS_BOT_GITHUB_PAT (already used in todo-management.yml) so that the cleanup commit properly triggers pre-commit, tests, and other CI workflows. Co-authored-by: openhands <[email protected]>

Since the PR is already approved when cleanup runs, there's no need to re-run CI checks. Using [skip ci] in the commit message: 1. Skips all GitHub Actions workflows for the cleanup commit 2. Avoids the 'stuck checks' UI issue (no pending checks to wait for) 3. Doesn't require a PAT (simpler, more secure) 4. Is the standard convention supported by GitHub Actions Co-authored-by: openhands <[email protected]>

The Vercel GitHub App creates check suites for this repo even though there's nothing to deploy. These checks get stuck in 'queued' forever. With [skip ci], GitHub Actions are skipped but Vercel still creates its stuck check, which blocks merging. Using the PAT triggers real CI workflows that complete successfully, which satisfies branch protection requirements regardless of Vercel. Note: The Vercel integration should be reconfigured at the org level to exclude this repo, but that's outside the scope of this workflow. Co-authored-by: openhands <[email protected]>

Adding back the test artifact so the workflow can be properly tested when the PR receives another approval. Co-authored-by: openhands <[email protected]>

We need to test automation again.

AGENTS.md

enyst

I love this!

I suggested to add temporary scripts to it, but it's not a big deal if you feel there are concerns with it. It's because quite many times, I've done something like
"pick x example that is closest to the PR fix, modify a bit to show our fix, then run it, let's see"

That made edits into the examples files themselves, that were not meant for merge.

Maybe instead this allows something like "make a script in .pr like x example that is closest... etc"

Co-authored-by: Engel Nyst <[email protected]>

jpshackelford added the automation label Jan 20, 2026

openhands-agent added 2 commits January 20, 2026 14:37

test: Add workflow_dispatch and test artifact for PR workflow testing

dbb7f9d

- Add workflow_dispatch trigger for manual testing - Add .pr/test-artifact.md to verify blocking behavior Co-authored-by: openhands <[email protected]>

style: Fix YAML formatting for pre-commit

276af82

Co-authored-by: openhands <[email protected]>

jpshackelford marked this pull request as ready for review January 20, 2026 14:45

all-hands-bot reviewed Jan 20, 2026

View reviewed changes

.github/workflows/pr-artifacts.yml Show resolved Hide resolved

.github/workflows/pr-artifacts.yml Show resolved Hide resolved

.github/workflows/pr-artifacts.yml Outdated Show resolved Hide resolved

.pr/test-artifact.md Outdated Show resolved Hide resolved

Update .github/workflows/pr-artifacts.yml

89eab7a

Co-authored-by: OpenHands Bot <[email protected]>

xingyaoww reviewed Jan 20, 2026

View reviewed changes

.github/workflows/pr-artifacts.yml Show resolved Hide resolved

jpshackelford and others added 3 commits January 20, 2026 09:53

Update .github/workflows/pr-artifacts.yml

af737b3

Co-authored-by: OpenHands Bot <[email protected]>

docs: Add PR artifacts convention to AGENTS.md

efb8877

Document the .pr/ directory convention for storing PR-specific design docs and analysis notes that should not be merged to main. Co-authored-by: openhands <[email protected]>

fix: Remove duplicate YAML keys from workflow

5899859

Co-authored-by: openhands <[email protected]>

jpshackelford marked this pull request as draft January 20, 2026 15:11

openhands-agent added 2 commits January 20, 2026 15:15

docs: Update AGENTS.md with revised .pr/ workflow

91cc45b

- Add 'How It Works' section explaining the 3-step flow - Clarify that checks pass (green) during development - Remove reference to blocking check Co-authored-by: openhands <[email protected]>

jpshackelford force-pushed the feat/pr-artifacts-workflow branch from 922a71b to 91cc45b Compare January 20, 2026 15:21

jpshackelford marked this pull request as ready for review January 20, 2026 15:22

all-hands-bot reviewed Jan 20, 2026

View reviewed changes

openhands-agent and others added 6 commits January 20, 2026 15:31

feat: Update PR comment when .pr/ directory is cleaned up

60ad8f7

- After auto-cleanup, update the notice comment to show cleanup completed - Change icon from 📁 to ✅ and update message - Added pull-requests: write permission to cleanup job Co-authored-by: openhands <[email protected]>

Update .github/workflows/pr-artifacts.yml

d2eb864

Co-authored-by: OpenHands Bot <[email protected]>

Add missing triggers

ce11218

Co-authored-by: OpenHands Bot <[email protected]>

use sha instead of ref for forked repos

28f0c88

Co-authored-by: OpenHands Bot <[email protected]>

added error handling for failed push

97f6a6c

Co-authored-by: OpenHands Bot <[email protected]>

fix: Remove duplicate permissions key in YAML

cebef0b

Co-authored-by: openhands <[email protected]>

jpshackelford requested a review from xingyaoww January 20, 2026 15:59

jpshackelford mentioned this pull request Jan 20, 2026

feat(agent-server): Support plugin loading when starting conversations #1651

Merged

xingyaoww previously approved these changes Jan 20, 2026

View reviewed changes

jpshackelford requested a review from xingyaoww January 20, 2026 19:10

Merge branch 'main' into feat/pr-artifacts-workflow

d540c3d

xingyaoww previously approved these changes Jan 20, 2026

View reviewed changes

github-actions bot and others added 5 commits January 20, 2026 19:19

chore: Remove PR-only artifacts [automated]

ac95458

test: Re-add .pr/ directory to test cleanup workflow

cba8bee

Adding back the test artifact so the workflow can be properly tested when the PR receives another approval. Co-authored-by: openhands <[email protected]>

jpshackelford requested a review from xingyaoww January 20, 2026 19:51

xingyaoww approved these changes Jan 20, 2026

View reviewed changes

chore: Remove PR-only artifacts [automated]

f610ff6

enyst reviewed Jan 20, 2026

View reviewed changes

AGENTS.md Outdated Show resolved Hide resolved

enyst reviewed Jan 20, 2026

View reviewed changes

AGENTS.md Show resolved Hide resolved

enyst approved these changes Jan 20, 2026

View reviewed changes

jpshackelford and others added 2 commits January 20, 2026 15:33

AGENTS.md should mention scripts can go in .pr/ too

ddb826c

Co-authored-by: Engel Nyst <[email protected]>

Metion scripts in AGENTS.md

058ec74

Co-authored-by: Engel Nyst <[email protected]>

jpshackelford enabled auto-merge (squash) January 20, 2026 20:35

Merge branch 'main' into feat/pr-artifacts-workflow

85d3e4b

jpshackelford merged commit b471909 into main Jan 20, 2026
18 checks passed

jpshackelford deleted the feat/pr-artifacts-workflow branch January 20, 2026 20:38

feat: Add .pr/ directory convention and auto-cleanup workflow #1764

feat: Add .pr/ directory convention and auto-cleanup workflow #1764

Uh oh!

Conversation

jpshackelford commented Jan 20, 2026 • edited by github-actions bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Changes

Convention

How It Works

Fork PR Handling

Benefits

Uh oh!

all-hands-bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

jpshackelford commented Jan 20, 2026

Uh oh!

github-actions bot commented Jan 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

all-hands-bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

openhands-ai bot commented Jan 20, 2026

Uh oh!

jpshackelford commented Jan 20, 2026

Uh oh!

openhands-ai bot commented Jan 20, 2026

Uh oh!

openhands-ai bot commented Jan 20, 2026

Summary

Root Cause

Fix

Checklist

Uh oh!

Uh oh!

Uh oh!

enyst left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

jpshackelford commented Jan 20, 2026 •

edited by github-actions bot

Loading

github-actions bot commented Jan 20, 2026 •

edited

Loading