Add OSOP workflow example — automated code review pipeline#13727
Add OSOP workflow example — automated code review pipeline#13727Archie0125 wants to merge 4 commits intomicrosoft:mainfrom
Conversation
Adds a portable YAML workflow definition describing an automated PR code review pipeline (fetch diff, parallel analysis, generate review, post comments) using the OSOP open standard. Additive only — no existing files modified. OSOP (Open Standard for Orchestration Protocols) is a framework-agnostic format for describing AI workflows, similar to how OpenAPI standardizes REST API descriptions.
Adds documentation explaining OSOP format and how it maps to Semantic Kernel concepts.
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
There was a problem hiding this comment.
Automated Code Review
Reviewers: 4 | Confidence: 91%
✓ Correctness
This PR adds two new documentation-only files (a README and a YAML workflow definition) for OSOP (Open Standard for Orchestration Protocols) under python/samples/concepts/osop/. There is no executable Python code. The only correctness issue is a filename mismatch: the YAML file's internal comment on line 7 references
semantic-kernel-code-review.osop.yamlbut the actual file is namedcode-review-pipeline.osop.yaml. The README correctly references the filename. Additionally, the concepts README.md is not updated to list this new sample directory, though that's more of a completeness concern than a correctness bug.
✓ Security Reliability
This PR adds two documentation-only files (a README and a YAML workflow spec) under a new
python/samples/concepts/osop/directory. No executable code is introduced—these are purely declarative/descriptive files. From a security and reliability standpoint, the files themselves pose no direct risk (no code execution, no secrets, no deserialization). The only notable concern is that the README directs users to install a third-party package (pip install osop) from PyPI, which introduces a supply-chain trust dependency that users should be aware of, but this is standard practice for sample documentation.
✓ Test Coverage
This PR adds two purely documentation/sample files: a README.md and a YAML workflow definition describing an OSOP (Open Standard for Orchestration Protocols) code-review pipeline. No Python code, no library changes, and no behavioral modifications are introduced. Since there is no executable code to test, the lack of accompanying tests is appropriate. Other sample concept directories (agents, mcp, chat_history, etc.) follow the same pattern of including README files without dedicated tests. No test coverage concerns arise from this change.
✗ Design Approach
This PR adds a new
osop/directory underpython/samples/concepts/containing only a README and a declarative YAML file — no runable Python code. Every other directory underpython/samples/concepts/contains actual.pyfiles that exercise Semantic Kernel APIs; this PR fundamentally breaks that contract. The YAML is a description of a workflow in a third-party format (OSOP) and cannot be executed by Semantic Kernel. The README frames OSOP as a peer to OpenAPI but the PyPIosoppackage has zero metadata (no author, no summary, no home page, no project URLs), making thepip install osopinstruction untrustworthy. Additionally, the inline comment in the YAML references a wrong filename, and thecheck_testsagent node is missing aconfig.modelfield that all other agent nodes define.
Flagged Issues
- The
python/samples/concepts/directory is for runable Python code demonstrating Semantic Kernel features. This PR adds only a YAML description file and README with no Python code that invokes any SK API. The correct approach is to either (a) add a working Python sample that uses SK to implement this workflow and optionally include the YAML as a supplementary artifact, or (b) place purely-declarative format examples under aresources/ordocs/directory rather thanconcepts/. - The
pip install osopinstruction in README.md points to a PyPI package (osop) that has no author, no summary, no home page, and no project URLs — hallmarks of an unclaimed or placeholder package. Recommending installation of an unveted third-party package in an official Microsoft repository is a security risk.
Suggestions
- Fix the filename mismatch in the YAML comment on line 7:
osop validate semantic-kernel-code-review.osop.yamlshould beosop validate code-review-pipeline.osop.yamlto match the actual filename. - Consider updating python/samples/concepts/README.md to include the new OSOP section for discoverability, consistent with other sample directories.
- The README instructs users to run
pip install osop— consider adding a note that this is a third-party package not maintained by Microsoft, so users can make an informed trust decision before installing it. - The YAML file references
https://api.github.com/repos/{owner}/{repo}/pulls/{number}as API endpoints. If this sample is ever used as a basis for actual execution, ensure authentication tokens are handled securely and not embedded in the workflow definition. - The
check_testsagent node (line 43–47 of the YAML) is the only agent node with noconfigsection specifying a model, which is inconsistent withanalyze_complexityandcheck_patterns. Add aconfig.modelfield for consistency. - The inline comment on line 7 of the YAML says
osop validate semantic-kernel-code-review.osop.yamlbut the actual filename iscode-review-pipeline.osop.yaml. Update the comment to match.
Automated review by Archie0125's agents
| # in parallel, generate a structured review, post comments on GitHub, | ||
| # and request changes if critical issues are found. | ||
| # | ||
| # Run with Semantic Kernel or validate: osop validate semantic-kernel-code-review.osop.yaml |
There was a problem hiding this comment.
Filename mismatch: the comment references semantic-kernel-code-review.osop.yaml but the actual file is named code-review-pipeline.osop.yaml.
| # Run with Semantic Kernel or validate: osop validate semantic-kernel-code-review.osop.yaml | |
| # Run with Semantic Kernel or validate: osop validate code-review-pipeline.osop.yaml |
| # in parallel, generate a structured review, post comments on GitHub, | ||
| # and request changes if critical issues are found. | ||
| # | ||
| # Run with Semantic Kernel or validate: osop validate semantic-kernel-code-review.osop.yaml |
There was a problem hiding this comment.
Filename in the comment doesn't match the actual file. Should be code-review-pipeline.osop.yaml.
| # Run with Semantic Kernel or validate: osop validate semantic-kernel-code-review.osop.yaml | |
| # Run with Semantic Kernel or validate: osop validate code-review-pipeline.osop.yaml |
| - id: "check_tests" | ||
| type: "agent" | ||
| subtype: "llm" | ||
| name: "Check Test Coverage" | ||
| description: "Verify new code paths have corresponding test cases." |
There was a problem hiding this comment.
check_tests is the only agent node without a config.model field. All other type: agent nodes specify a model; omitting it here creates an inconsistent and incomplete workflow definition.
| - id: "check_tests" | |
| type: "agent" | |
| subtype: "llm" | |
| name: "Check Test Coverage" | |
| description: "Verify new code paths have corresponding test cases." | |
| - id: "check_tests" | |
| type: "agent" | |
| subtype: "llm" | |
| name: "Check Test Coverage" | |
| description: "Verify new code paths have corresponding test cases." | |
| config: | |
| model: "gpt-4o" |
| ## Quick Start | ||
|
|
||
| ```bash | ||
| # Validate the workflow |
There was a problem hiding this comment.
The osop package on PyPI has no author, summary, homepage, or project URLs. Recommending pip install osop in an official Microsoft repo without vetting this package is a trust and supply-chain risk. At minimum, pin to a specific version and verify the package provenance before publishing this.
| # Validate the workflow | |
| pip install osop==0.5 # verify package provenance before use |
|
@microsoft-github-policy-service agree |
- Fix filename mismatch in YAML comment (line 7) - Add config.model to check_tests node for consistency - Add model field to generate_review node - Remove pip install osop from README (these are declarative docs, not executable) - Add note that OSOP files are declarative descriptions - Fix GitHub links to correct repo URLs
- Add osop_workflow_reader.py: reads .osop.yaml and creates SK ChatCompletionAgents from agent nodes (addresses design review concern that concepts/ needs runnable Python code) - Remove pip install osop from README (address supply-chain concern) - Update README to describe the Python script usage
|
Thanks for the thorough review! Addressed the actionable items:
Regarding the YAML-only concern — the .osop.yaml file is a self-contained workflow definition that works standalone without any OSOP tooling. It's meant as documentation showing how Semantic Kernel concepts (planners, plugins, kernel functions) map to a portable format. Happy to move it to |
1 participant
Summary
python/samples/concepts/osop/code-review-pipeline.osop.yaml) describing an automated PR code review pipeline with parallel analysis stagesChatCompletionAgent, plugins,KernelFunction, etc.)What is OSOP?
OSOP (Open Standard for Orchestration Protocols) is a portable YAML format for describing AI agent workflows — think OpenAPI, but for agent orchestration. It lets you document, validate, and visualize multi-agent patterns in a framework-agnostic way.
The
.osop.yamlfile is self-documenting and human-readable. It works standalone without any OSOP tooling installed.Files Added
python/samples/concepts/osop/README.mdpython/samples/concepts/osop/code-review-pipeline.osop.yamlTest Plan
python -c "import yaml; yaml.safe_load(open('python/samples/concepts/osop/code-review-pipeline.osop.yaml'))"