[copilot-cli-research] Copilot CLI Deep Research - 2026-03-02

[github-actions[bot]]

Analysis Date: 2026-03-02
Repository: github/gh-aw
Scope: 165 total workflows, 79 using Copilot engine (48%)
Copilot CLI Version: 0.0.420 (default)

---

📊 Executive Summary

This first comprehensive research analysis of Copilot CLI usage in gh-aw reveals strong adoption of core features (github tools, bash, cache-memory) but significant gaps in newer capabilities. The most impactful opportunity is copilot-requests feature adoption — only 52% of Copilot workflows use it, despite it simplifying auth requirements and removing the need for a COPILOT_GITHUB_TOKEN secret. Additionally, 6 of 9 custom agent files in .github/agents/ are never referenced in any workflow, representing untapped value. The max-continuations feature (autopilot mode) is used in just 1 of 79 Copilot workflows, despite being particularly valuable for long-running analysis tasks.

Overall, the workflow ecosystem is healthy and well-structured, but there are 8 clear opportunities to improve security, performance, and developer experience by using features that already exist in the system.

---

🔴 Critical Findings

High Priority

1. copilot-requests Feature: 52% Adoption Gap
38 Copilot workflows don't use the copilot-requests feature, requiring them to have a COPILOT_GITHUB_TOKEN secret. This feature uses the GitHub Actions token directly ($\{\{ github.token }}), simplifying authentication significantly.

Workflows missing copilot-requests: daily-performance-summary.md, firewall.md, grumpy-reviewer.md, jsweep.md, layout-spec-maintainer.md, mcp-inspector.md, mergefest.md, metrics-collector.md, org-health-report.md, pdf-summary.md, plan.md, portfolio-analyst.md, pr-nitpick-reviewer.md, pr-triage-agent.md, python-data-charts.md, q.md, refiner.md, release.md, and ~20 others.

2. disable-xpia-prompt: Never Used
Zero workflows use this feature flag despite it being available. Internal/trusted workflows that only process system-generated content (not user input) could safely disable the XPIA prompt for cleaner, more focused prompts.

---

1️⃣ Copilot CLI Capabilities Inventory

View Full Capabilities Inventory

CLI Flags (auto-configured by compiler)

• --add-dir — directory access control (compiler-managed)
• --disable-builtin-mcps — always applied
• --allow-tool / --allow-all-tools — tool permissions
• --agent — custom agent file identifier
• --autopilot + --max-autopilot-continues — multi-continuation mode
• --allow-all-paths — write access for edit tool
• --log-level all --log-dir — logging
• --prompt — instruction delivery

Engine Configuration Options

Option	Type	Description
version	string	Pin Copilot CLI version
model	string	Override AI model
agent	string	Custom agent file (.github/agents/*.agent.md)
args	array	Additional CLI arguments
command	string	Custom executable path
env	map	Custom environment variables
max-continuations	int	Autopilot continuation count
max-turns	string	Max tool call turns

Feature Flags

Flag	Purpose
copilot-requests	Use Actions token (no secret required)
safe-inputs	Secure input validation via MCP
disable-xpia-prompt	Disable XPIA security notice
mcp-gateway	MCP gateway routing

Sandbox Options

Mode	Description
sandbox.agent: awf	Agent Workflow Firewall (network isolation)
sandbox.agent: srt	Sandbox Runtime (process isolation)
sandbox.agent: false	Disable sandboxing

Available Custom Agents (.github/agents/)

• agentic-workflows — workflow design assistant
• ci-cleaner — CI cleanup specialist
• contribution-checker — PR contribution reviewer
• create-safe-output-type — safe output implementation guide
• custom-engine-implementation — engine development guide
• grumpy-reviewer — critical code reviewer
• interactive-agent-designer — interactive workflow designer
• technical-doc-writer — documentation writer
• w3c-specification-writer — spec writing specialist

---

2️⃣ Feature Usage Matrix

Feature Category	Available	Used	Not Used	Usage Rate
copilot-requests	✅	41 workflows	38 copilot workflows	52%
max-continuations	✅	1 workflow	78 workflows	1%
safe-inputs	✅	1 workflow	78 workflows	1%
disable-xpia-prompt	✅	0 workflows	79 workflows	0%
mcp-gateway	✅	0 workflows	79 workflows	0%
Custom agents	9 available	3	6 unused	33%
AWF sandbox	✅	13 workflows	66 workflows	16%
Model override	✅	8 workflows	71 workflows	10%
Engine env vars	✅	16 workflows	—	20%
Version pinning	✅	0 workflows	79 workflows	0%
Plugins	✅	0 workflows	all	0%
post-steps	✅	0 workflows	all	0%
web-search tool	✅	2 workflows	—	1%
web-fetch tool	✅	15 workflows	—	9%
playwright tool	✅	11 workflows	—	7%

Most Used Features

Tool/Feature	Workflows	% of Total
github tool	122	74%
bash tool	114	69%
edit tool	77	47%
cache-memory	72	44%
strict: true	97	59%
imports	124	75%
concurrency	172	100%+ (lock files)

---

3️⃣ Missed Opportunities

View High Priority Opportunities

🔴 Opportunity 1: copilot-requests Feature — 38 Workflows Missing It

What: The copilot-requests feature uses $\{\{ github.token }} instead of COPILOT_GITHUB_TOKEN secret, simplifying secrets management. It also adds copilot-requests: write permission automatically.

Why It Matters: Reduces secret management overhead. Workflows without it require COPILOT_GITHUB_TOKEN to be configured in repo secrets.

How to Implement:

features:
  copilot-requests: true

Caveat: Only safe for workflows that don't need an elevated PAT with broader permissions than github.token provides. Review each workflow's needs first.

---

🔴 Opportunity 2: max-continuations — Only 1 Workflow

What: The max-continuations config combined with --autopilot flag allows Copilot CLI to continue across multiple sessions, tackling complex multi-step tasks.

Why It Matters: For long-running workflows (30+ minute timeouts) doing iterative work (code simplification, doc updates, analysis), autopilot mode can handle more complex tasks without manual intervention.

Where: daily-compiler-quality.md, daily-doc-updater.md, commit-changes-analyzer.md, code-simplifier.md are all 30+ minute workflows that could benefit.

How to Implement:

engine:
  id: copilot
  max-continuations: 3
timeout-minutes: 60

View Medium Priority Opportunities

🟡 Opportunity 3: 6 Custom Agent Files Never Used

What: These .github/agents/*.agent.md files exist but are never referenced in any workflow:

• contribution-checker.agent.md — Could improve contribution-check.md workflow
• create-safe-output-type.agent.md — Could help workflows that create safe outputs
• custom-engine-implementation.agent.md — Internal tooling workflows
• grumpy-reviewer.agent.md — Code review workflows like ci-coach.md or pr-nitpick-reviewer.md
• interactive-agent-designer.agent.md — Workflow design meta-tasks
• w3c-specification-writer.agent.md — Documentation workflows

How to Implement:

engine:
  id: copilot
  agent: grumpy-reviewer   # references .github/agents/grumpy-reviewer.agent.md

---

🟡 Opportunity 4: safe-inputs — Only 1 Workflow

What: The safe-inputs feature validates and sanitizes inputs before they reach the AI, providing an additional layer of prompt injection protection.

Why It Matters: Only security-review.md currently uses this. Any workflow triggered by external user input (slash commands, issue comments, PR content) is a candidate.

Candidates: auto-triage-issues.md, ai-moderator.md, pr-triage-agent.md, ci-coach.md.

---

🟡 Opportunity 5: GitHub Toolsets — Some Over-Broad (all)

What: 3 workflows use toolsets: [all] which grants access to every GitHub MCP tool. More specific toolsets reduce the attack surface.

Where: Identify with:

grep -rl "toolsets: \[all\]" .github/workflows/

How to Improve: Use specific toolsets:

tools:
  github:
    toolsets: [repos, issues, pull_requests]  # Instead of [all]

View Low Priority Opportunities

🟢 Opportunity 6: disable-xpia-prompt — Never Used

What: Disables the automatic XPIA (Cross-Prompt Injection Attack) security notice injected into every prompt.

Why: Internal workflows that only process system-generated content (e.g., daily-architecture-diagram.md, scheduled metrics workflows) don't need this security notice, and removing it makes prompts cleaner and more focused.

How to Implement:

features:
  disable-xpia-prompt: true

Caution: Only use for workflows that never process external user content.

---

🟢 Opportunity 7: web-search Tool — Only 2 Workflows

What: Copilot CLI has built-in web-search capability but only 2 workflows use it (daily-news.md, one other). Many research workflows rely on web-fetch for specific URLs when web-search would be more flexible.

Candidates: blog-auditor.md, daily-news.md, any workflow doing external knowledge lookup.

tools:
  web-search:

---

🟢 Opportunity 8: Version Pinning — 0 Workflows

What: No workflow pins a specific Copilot CLI version. All use 0.0.420 (the current default).

Why It Matters (optionally): For stability-critical workflows, pinning a version prevents unexpected behavior changes when the default updates.

How to Implement:

engine:
  id: copilot
  version: "0.0.420"

Note: Generally fine to leave at default; only consider pinning if a specific workflow breaks after upgrades.

---

🟢 Opportunity 9: post-steps — Never Used

What: Allows custom steps to run after the agent completes (cleanup, notifications, metrics).

Candidates: Workflows that write to disk or need cleanup after runs.

post-steps:
  - name: Cleanup temp files
    run: rm -rf /tmp/workflow-artifacts/

---

4️⃣ Specific Workflow Recommendations

View Workflow-Specific Recommendations

contribution-check.md

• Current State: Uses Copilot engine for PR contribution checking
• Recommended: Add agent: contribution-checker — there's already a dedicated agent file for this exact purpose

pr-nitpick-reviewer.md / grumpy-reviewer.md

• Current State: Uses Copilot engine without custom agent
• Recommended: Add agent: grumpy-reviewer to leverage the existing agent file
• Also: Add features: { copilot-requests: true } if not already present

daily-doc-updater.md, daily-doc-healer.md

• Current State: 30-minute timeouts, single-pass doc updates
• Recommended: Add engine.max-continuations: 2 to enable multi-pass improvements for comprehensive doc updates

code-simplifier.md, commit-changes-analyzer.md

• Current State: Single continuation, 30-minute timeout
• Recommended: max-continuations: 3 — complex code analysis and multi-file simplification would benefit from autopilot mode

auto-triage-issues.md, ai-moderator.md

• Current State: Process external user content (issues/PRs)
• Recommended: Enable safe-inputs for prompt injection protection on user-submitted content

Workflows using toolsets: [all]

• Current State: Full GitHub MCP access
• Recommended: Narrow to specific toolsets (e.g., [repos, issues]) to minimize permission scope

---

5️⃣ Trends & Insights

View Historical Context

This is the first comprehensive analysis of Copilot CLI usage in this repository. Future runs will track:

• copilot-requests feature adoption rate
• max-continuations adoption for complex workflows
• Custom agent file utilization
• GitHub toolset specificity improvements

Baseline established: 2026-03-02, run §22596116311

---

6️⃣ Best Practice Guidelines

Based on this research, here are recommended best practices:

1. Always enable copilot-requests for new Copilot workflows unless a PAT with elevated permissions is specifically required — it simplifies auth and reduces secrets overhead.

2. Use custom agents for specialized tasks — The .github/agents/ directory has expert agents for code review, documentation, specification writing, and more. Check before writing a raw prompt.

3. Prefer specific GitHub toolsets — Use [repos, issues] over [default] over [all]. Minimize the permission surface to what the workflow actually needs.

4. Consider max-continuations for complex tasks — Workflows with 30+ minute timeouts doing iterative improvements are prime candidates for autopilot mode with 2-3 continuations.

5. Enable safe-inputs for user-triggered workflows — Any workflow processing issue comments, PR content, or slash commands should use safe-inputs for prompt injection protection.

6. Reserve disable-xpia-prompt — Only use for scheduled workflows with no user-generated input. Leave the default XPIA prompt for all user-interactive workflows.

---

7️⃣ Action Items

Immediate Actions (this week):

• Enable copilot-requests: true in the 38 copilot workflows that are missing it (batch update)
• Wire contribution-check.md to use agent: contribution-checker
• Wire grumpy-reviewer.md (or pr-nitpick-reviewer.md) to use agent: grumpy-reviewer

Short-term (this month):

• Add max-continuations: 2 to daily-doc-updater.md, daily-doc-healer.md, code-simplifier.md
• Evaluate safe-inputs for user-triggered workflows (auto-triage-issues.md, ai-moderator.md)
• Narrow toolsets: [all] to more specific toolsets in the 3 workflows using it

Long-term (this quarter):

• Document and link unused agent files to appropriate workflows
• Create a standard for when disable-xpia-prompt is appropriate
• Establish version pinning policy (when to pin, when to track latest)
• Re-run this analysis after implementing recommendations to measure adoption

---

View Research Methodology

Research Methodology

Data Sources:

• Go source files: pkg/workflow/copilot_engine*.go, pkg/workflow/copilot_mcp.go
• All 165 workflow markdown files in .github/workflows/*.md
• Engine documentation: docs/src/content/docs/reference/engines.md
• Constants: pkg/constants/constants.go
• Frontend config: pkg/workflow/frontmatter_types.go

Analysis Techniques:

• Pattern matching with grep across all workflow files
• Feature-by-feature inventory from source code
• Usage counting per tool/feature/flag
• Cross-referencing available vs. used capabilities

Tools Used: bash, grep, Go source analysis

Research Persistence: Analysis saved to repo-memory branch for future trend tracking.

---

References:

• §22596116311 — This analysis run
• Copilot Engine Source — Implementation details
• Engine Documentation — Configuration reference

AI generated by Copilot CLI Deep Research Agent

• expires on Mar 3, 2026, 9:25 PM UTC

[github-actions[bot]]

This discussion was automatically closed because it expired on 2026-03-03T21:25:14.372Z.

Closed by Workflow