Skip to content

Design and implement the operator-in-the-loop end-to-end orchestration model #33

@jflamb

Description

@jflamb

Summary

Design and implement the next evolution of the orchestration system so a single issue can move from Ready through autonomous implementation, deployment to a reviewable environment, operator feedback loops, final approval, merge, verification, and closeout with minimal operator intervention.

The target model is:

  • operator initiates work by transitioning an issue to Ready
  • autonomous agent workflows carry the work through clarification, design, planning, implementation, and review-ready deployment
  • operator reviews the deployed implementation and leaves feedback
  • autonomous agent workflows address feedback, redeploy, and notify the operator when an updated version is ready
  • this feedback loop repeats until operator approval
  • autonomous agent workflows then merge, verify, diagnose failures, capture follow-ups, and perform housekeeping/cleanup

Scope

  • Define the canonical state machine for this operator/agent collaboration model
  • Distinguish coarse project Status values from finer-grained internal orchestration stages
  • Define the deployment contract required before operator review begins
  • Define the structured operator feedback and approval contract
  • Define how feedback-triggered iteration re-enters the orchestration flow
  • Define the merge/finalization/cleanup path after approval
  • Implement the minimum workflow and contract changes in this repository to support the model end-to-end

Acceptance Criteria

  • An issue can move from Ready into an autonomous build path without additional operator edits
  • The system can produce a review-ready implementation that is deployed somewhere the operator can interact with
  • Operator feedback is captured in a machine-detectable format that can re-trigger autonomous implementation work
  • The system can iterate through at least one operator-feedback cycle without losing state
  • Operator approval is machine-detectable and transitions the work into merge/finalization
  • Finalization covers merge, verification, diagnostics/retries as appropriate, follow-up capture, and cleanup/closeout
  • The project board and issue/PR artifacts make the current overall process understandable at a glance

Constraints

  • Keep the GitHub Project Status field coarse and operator-meaningful rather than overloading it with every internal stage
  • Use GitHub artifacts as the primary system of record: issue comments, PRs, discussions, deployment comments/status, and project fields
  • Preserve resumability and stage-level retry semantics
  • Prefer contracts and workflow changes that can propagate later to repo-template
  • Avoid inventing an external state store unless GitHub artifacts prove insufficient

Proposed State Model

Coarse project statuses

  • Backlog
  • Ready
  • In Progress
  • In Review
  • Approved
  • Done
  • Blocked

Internal orchestration stages

Build lane

  • kickoff
  • clarification
  • design
  • plan
  • execution
  • deploy-review

Review loop lane

  • review-intake
  • feedback-implementation
  • redeploy-review

Finalize lane

  • merge
  • post-merge-verify
  • follow-up-capture
  • closeout

Acceptance Criteria For The Design Itself

  • The relationship between coarse project status and internal stage progression is explicit
  • The operator review artifact set is explicit (PR + deployment URL + issue summary, or similar)
  • The feedback loop has a concrete machine-detectable trigger
  • The approval signal has a concrete machine-detectable trigger
  • The finalization path is explicit about verification, diagnostics, retries, and cleanup

Open Questions

  • What is the smallest acceptable "deployed somewhere" target for the first end-to-end implementation proof? (DEFERRED-TO-DESIGN)
  • Should operator feedback be modeled primarily via PR review state, structured PR comments, issue comments, or a combination? (DEFERRED-TO-DESIGN)
  • Should Approved be a project Status value, or should approval remain an internal signal while the card stays In Review? (DEFERRED-TO-DESIGN)

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions