Design: Design and implement the operator-in-the-loop end-to-end orchestration model

[github-actions[bot]]

Design Discussion: Design and implement the operator-in-the-loop end-to-end orchestration model

Source issue: #33
Created by orchestration automation

Summary

This design defines a GitHub-native orchestration model where setting an issue to Ready starts autonomous delivery, produces a reviewable deployment, iterates on operator feedback until approval, then merges, verifies, captures follow-ups, and closes out. Project Status stays coarse and operator-readable; detailed execution state is tracked via GitHub artifacts.

Problem

The current flow does not enforce one deterministic, resumable path from Ready to Done. Missing pieces include a strict review-ready contract, machine-detectable feedback/approval triggers, explicit retry/blocked behavior, and a concrete finalization sequence.

Goals

• Define one canonical state machine across build, review-loop, and finalize lanes.
• Keep project Status coarse and meaningful to operators.
• Require explicit review-ready artifacts before review begins.
• Make feedback and approval machine-detectable and automation-triggering.
• Preserve resumability and stage-level retry semantics using GitHub artifacts.
• Make finalization explicit: merge, verification, diagnostics/retries, follow-up capture, closeout.

Non-goals

• Adding an external state store.
• Mapping every internal stage to project Status.
• Multi-repo orchestration expansion.
• Broad infrastructure redesign beyond minimum end-to-end support in this repository.

Proposed Approach

1) Canonical state model

Project Status values:

• Backlog
• Ready
• In Progress
• In Review
• Approved
• Done
• Blocked

Internal stages:

Build lane:

• kickoff
• clarification
• design
• plan
• execution
• deploy-review

Review loop lane:

• review-intake
• feedback-implementation
• redeploy-review

Finalize lane:

• merge
• post-merge-verify
• follow-up-capture
• closeout

2) Status/stage transition contract

• Operator sets issue Status=Ready to start orchestration.
• kickoff transitions Status to In Progress.
• Build lane executes in order through deploy-review.
• deploy-review success transitions Status=In Review and enters review-intake.
• In review-intake:
  • feedback command enters feedback-implementation -> redeploy-review -> review-intake
  • approval command transitions Status=Approved and enters finalize lane
• closeout success transitions Status=Done.
• Retry-exhausted or unrecoverable failure sets Status=Blocked with diagnostics.

3) GitHub artifact contract (system of record)

Required persisted artifacts:

• Issue comment: single orchestration checkpoint thread (append-only stage events).
• PR linked to issue for implementation work.
• Deployment/status artifact containing review URL.
• Structured issue comments for review-ready, feedback, approval, finalization summary.
• Project Status for coarse operator-visible state.

Each stage event records at minimum:

• lane, stage, state (started|succeeded|failed|blocked)
• attempt, max_attempts
• issue_number, pr_number (or none)
• deployment_url (or none)
• timestamp_utc
• next_stage (or none)

Resumability rule:

• Orchestration resumes from latest non-terminal stage event.
• Re-running a completed stage event is idempotent/no-op unless explicitly retried.

4) Review-ready deployment contract

deploy-review/redeploy-review can succeed only if all are true:

• PR exists and is linked to the source issue.
• Required review-readiness checks are passing.
• A reachable review URL exists in GitHub deployment/status.
• A structured review-ready issue comment is posted containing:
  • PR link
  • deployment URL
  • concise change summary
  • exact feedback/approval command format

5) Operator feedback + approval contract

Primary machine triggers are structured issue comments from authorized operators:

• Feedback: gpa:feedback <instructions>
• Approval: gpa:approve (optional trailing note)

Parsing rules:

• Only comments newer than the most recent review-ready marker are eligible.
• Only authorized operator identities are actionable.
• Same comment ID processed twice is ignored (idempotent).
• Latest valid actionable command wins.

PR reviews and unstructured comments remain contextual input, not orchestration triggers.

6) Finalization contract

• merge: merge PR according to repository policy.
• post-merge-verify: run post-merge verification; retry recoverable failures within stage budget.
• follow-up-capture: create/link follow-up issues for unresolved non-blocking work.
• closeout: publish final artifact summary, close parent issue, set Status=Done.

If verification remains failing after retries:

• set Status=Blocked
• publish diagnostics and explicit operator recovery actions
• keep parent issue open.

Open Questions

• What is the smallest acceptable “deployed somewhere” target for first proof? (DEFERRED-TO-PLAN)
  Answer: A GitHub Deployment/Environment status with a reachable URL that the operator can interact with is sufficient for v1.
• Should operator feedback be modeled via PR review, structured PR comments, issue comments, or a combination? (DEFERRED-TO-PLAN)
  Answer: Structured issue comments are the authoritative trigger channel; PR reviews/comments are supporting context.
• Should Approved be a project Status value or only an internal signal? (DEFERRED-TO-PLAN)
  Answer: Keep Approved as a coarse project Status value and use it as the gate into finalize stages.

Exit Criteria

• Relationship between coarse project Status and internal stage progression is explicit.
• Operator review artifact set is explicit (PR + deployment URL + structured review-ready summary).
• Feedback trigger is concrete and machine-detectable.
• Approval signal is concrete and machine-detectable.
• Finalization path explicitly covers merge, verification, diagnostics/retries, follow-up capture, and cleanup/closeout.
• Design is specific enough for the next plan stage without additional operator clarification.

[project-automation-gateway[bot]]

gpa:design-review:auto

Verdict: Ready for planning. The design resolves every open question flagged in the source issue and satisfies all five acceptance criteria for the design artifact itself. A few specifics worth tracking before or during planning:

Gaps and risks

1. Out-of-order command handling is unspecified. The design says gpa:feedback and gpa:approve are only valid at review-intake, but does not say what happens if they appear earlier (e.g., posted during execution) or if multiple gpa:feedback comments accumulate. The implementation will need a clear rule — likely "only the latest command at the moment review-intake polls" — and the review-ready summary should state when commands become active.

2. "Required repository checks for reviewability" is vague. The deploy-review contract gates on passing checks but does not name them or say how they are configured. This is fine to defer, but the plan stage needs to produce a concrete, enumerable list or a discovery mechanism; otherwise the exit condition for deploy-review is not deterministic.

3. Approved is not a stable resting state. The design correctly keeps it as a coarse status, but operators will see Approved while the finalize lane is actively running. This is operationally fine as long as the review-ready summary and checkpoint comments make the running stage visible. If post-merge-verify fails and the status flips to Blocked, operators may not understand why Approved disappeared. The plan should document this transition explicitly.

4. Checkpoint comment schema is fully deferred. This is the right call — it belongs in the plan — but resume logic, retry counting, and Blocked diagnostics all depend on it. The plan stage should treat this as its highest-priority deliverable; slipping it further would leave the orchestration model only partially implementable.

What works well

The status-to-stage mapping rules are unambiguous and cover every lane. Choosing structured issue comments as the sole machine-detectable feedback/approval channel is a clean, auditable decision. The finalization lane covers every required element from the acceptance criteria. The "smallest acceptable deployment" answer (GitHub deployment artifact + reachable URL) is appropriately scoped for an end-to-end proof without over-engineering.