Skip to content

docs(agents): new-item-type onboarding agent#193

Merged
ayeshurun merged 13 commits intomicrosoft:mainfrom
ayeshurun:copilot/update-custom-agent-instructions
Mar 22, 2026
Merged

docs(agents): new-item-type onboarding agent#193
ayeshurun merged 13 commits intomicrosoft:mainfrom
ayeshurun:copilot/update-custom-agent-instructions

Conversation

@ayeshurun
Copy link
Collaborator

@ayeshurun ayeshurun commented Mar 19, 2026

📥 Pull Request

✨ Description of new changes

  • Summary: Adds a new GitHub Copilot agent (.github/agents/new-item-type.agent.md) that guides contributors through the end-to-end process of onboarding a new Microsoft Fabric item type into the CLI.
  • Context: Adding a new item type to Fabric CLI currently requires changes across 7+ files (enum registration, API/portal mappings, definition formats, command support YAML, import payloads, test parametrization lists, changelog, and docs). This agent codifies all integration points into a structured 13-step checklist with code snippets, validation rules, and real examples — making the process faster, less error-prone, and accessible to new contributors.
  • Dependencies: None. This is a documentation/agent-only change with no runtime code modifications.

What the agent covers

The new-item-type onboarding agent walks contributors through:

  1. Step 1–4 — Register the item type enum, API format mapping, portal URI mapping, and definition format mapping in fab_types.py
  2. Step 5–6 — Add OneLake folders and job type mappings (agent asks the requestor for input on these — does not guess)
  3. Step 7–8 — Add creation parameters and payload logic in fab_cmd_mkdir_utils.py
  4. Step 9 — Add import payload handling in fab_item.py
  5. Step 10 — Update command_support.yaml for export/import/mv/cp
  6. Step 11 — Add to test parametrization lists in tests/test_commands/conftest.py (ALL_ITEM_TYPES, basic_item_parametrize, mv params, export params, set metadata params)
  7. Step 12 — Add changelog entry
  8. Step 13 — Update documentation pages (resource types + item examples)

Key design decisions

  • "When in doubt, ask" — The agent is instructed to always ask the requestor for clarification rather than guessing, particularly for OneLake folders (Step 5) and job types (Step 6)
  • Complexity-based patterns — Includes guidance for simple items, definition-supporting items, items with creation parameters, OneLake folders, job support, and full-featured items
  • Real reference example — Uses the Map item type (from PR feat(types): add support for Map item type #191) as a complete worked example showing all 7 files changed
  • Validation checklist — Provides a comprehensive post-implementation checklist to verify completeness

ayeshurun and others added 7 commits March 17, 2026 12:14
This script benchmarks the startup performance of the CLI by measuring module import times, CLI invocation times, and heavy dependency loading. It allows comparisons against a baseline branch or tag.
…h complete integration checklist

Co-authored-by: ayeshurun <98805507+ayeshurun@users.noreply.github.com>
…changes

PR microsoft#191 added Map as a simple item (no definition formats, no jobs,
no OneLake folders, no creation parameters). The agent file previously
overclaimed that Map needed command_support.yaml, definition_format_mapping,
changelog, and documentation changes. This corrects the reference example,
common patterns classification, and reference table to match reality.

Co-authored-by: ayeshurun <98805507+ayeshurun@users.noreply.github.com>
…r PR microsoft#191 new commits

PR microsoft#191 new commits expanded Map integration to include export, import,
mv, cp support. Updated: Map reference example now lists all 7 changed
files, added Step 12e (export test params) and 12f (set metadata params),
reclassified Map as "Standard with definitions" in patterns and reference
table, expanded validation checklist.

Co-authored-by: ayeshurun <98805507+ayeshurun@users.noreply.github.com>
Co-authored-by: ayeshurun <98805507+ayeshurun@users.noreply.github.com>
…uidance to Steps 5-6, renumber steps

- Removed Step 7 (Add Mutable Properties) entirely
- Added ⚠️ Ask the requestor prompts to Step 5 (OneLake Folders) and Step 6 (Job Types)
- Added general "when in doubt, ask" instruction at the top
- Renumbered all subsequent steps (old 8-14 → new 7-13)
- Updated Validation Checklist, Common Patterns, and Reference sections
- Removed all mutable properties references

Co-authored-by: ayeshurun <98805507+ayeshurun@users.noreply.github.com>
@ayeshurun ayeshurun requested a review from a team as a code owner March 19, 2026 13:06
@ayeshurun ayeshurun changed the title docs(agents): rewrite new-item-type onboarding agent with complete integration checklist docs(agents):new-item-type onboarding agent Mar 19, 2026
@ayeshurun ayeshurun changed the title docs(agents):new-item-type onboarding agent docs(agents): new-item-type onboarding agent Mar 19, 2026
@jeremydhoover-blip
Copy link
Contributor

These are all about making the agent more reliable when the LLM processes the file, not about rewriting it. The content itself is solid — the steps, code snippets, and rules are all correct and complete.

1. Frontmatter already handles persona and "when to use"

The description field is what the LLM reads to decide when to activate the agent. The "You are an expert..." line and the "When to Use This Agent" section repeat what's already in description and argument-hint. Removing them saves ~8 lines of context that provide no additional signal to the model.

2. "Key Files Quick Reference" table is redundant

Every step already says File: src/fabric_cli/core/fab_types.py. The quick reference at the bottom repeats all 12 file paths. The LLM already has them — it just has them twice now, which costs tokens without improving accuracy.

3. Safety rules should be consolidated at the top

Currently the "never guess" rule is in a callout at line 12, and the safety rules (no hardcoded secrets, deterministic test data, etc.) are at line 591. The LLM processes the file top-to-bottom — rules it sees first have the strongest influence on behavior. Moving safety rules to the top (or grouping them with the existing "never guess" callout) means they apply from the start of generation, not after 590 lines of context.

4. Complexity/decision table should appear before the steps

This is the highest-impact change. Right now the "Common Patterns by Item Complexity" section is at line 498 — after all 13 steps. That means the LLM reads through all 13 steps before learning that a simple item (Dashboard) only needs steps 1-3 and 9. If the complexity table comes first, the LLM knows what to skip before it starts reading steps. Result: fewer unnecessary steps executed, fewer tokens wasted.

5. "Existing Item Types to Study" should be next to the complexity table

These two tables answer the same question: "what kind of item am I dealing with?" Having them adjacent lets the LLM cross-reference (e.g., "this is like Map, which is definition-support complexity, which means steps 1-4, 9, 10-11, 12-13"). Currently one is near the end and one would need to be near the beginning.

6. Phase structure is optional but helps the LLM sequence correctly

The 4-phase framing (gather → classify → execute → validate) gives the LLM a clear mental model: "don't write code until you've collected prerequisites and classified complexity." Without it, there's no signal telling the LLM to stop and ask questions before jumping to Step 1. But this is a lighter suggestion — even just reordering the sections (complexity table before steps) captures most of the value without adding phase labels.

7. Replace the verbose validation checklist with a runnable verification table

The current validation checklist is 19 lines of checkboxes with prose descriptions. Since the LLM can't "check a checkbox" it has to read each file and reason about whether the item type is present. That's expensive and error-prone. Instead, give it a verification table with actual commands it can execute via runInTerminal:

Step Verify with When
1. Enum grep -n 'NEW_ITEM.*=.*"NewItem"' src/fabric_cli/core/fab_types.py Always
2. format_mapping grep -n 'ItemType.NEW_ITEM' src/fabric_cli/core/fab_types.py | grep -i format_mapping Always
3. uri_mapping grep -n 'ItemType.NEW_ITEM' src/fabric_cli/core/fab_types.py | grep -i uri_mapping Always
4. definition_format grep -n 'ItemType.NEW_ITEM' src/fabric_cli/core/fab_types.py | grep -i definition Has definitions
5. OneLake folders grep -n 'ItemType.NEW_ITEM' src/fabric_cli/core/fab_types.py | grep -i folder Has OneLake folders
6. Job mapping grep -n 'ItemType.NEW_ITEM' src/fabric_cli/core/fab_types.py | grep -i job Has jobs
7. Creation params grep -n 'ItemType.NEW_ITEM' src/fabric_cli/utils/fab_cmd_mkdir_utils.py | grep -i param Has creation params
8. Creation payload grep -n 'ItemType.NEW_ITEM' src/fabric_cli/utils/fab_cmd_mkdir_utils.py | grep -i payload Has creation payload
9. Import payload grep -n 'ItemType.NEW_ITEM' src/fabric_cli/core/hiearchy/fab_item.py Always
10. Command support grep -n 'new_item' src/fabric_cli/core/fab_config/command_support.yaml Has definitions
11a. ALL_ITEM_TYPES grep -n 'NEW_ITEM' tests/test_commands/conftest.py | grep -i all_item Always
11b. basic_item grep -n 'NEW_ITEM' tests/test_commands/conftest.py | grep -i basic Basic items only
11c. mv params grep -n 'NEW_ITEM' tests/test_commands/conftest.py | grep -i mv Has mv support
11e. export params grep -n 'NEW_ITEM' tests/test_commands/conftest.py | grep -i export Has export support
11f. set metadata grep -n 'NEW_ITEM' tests/test_commands/conftest.py | grep -i metadata Has set support
12. Changelog ls .changes/unreleased/ | grep -i newitem Always
13a. Resource types grep -n 'NewItem' docs/essentials/resource_types.md Always
13b. Item examples grep -n 'NewItem' docs/examples/item_examples.md Has cp or export
Tests pass python -m pytest tests/ -q Always

Copilot AI added a commit to ayeshurun/fabric-cli that referenced this pull request Mar 22, 2026
Copy link
Contributor

@jeremydhoover-blip jeremydhoover-blip left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks amazing, just one small note. I think the language quoted below is not explicit enough and too up for interpretation. I think it can just be removed because it should be clear enough from the first part of this rule plus the other rules/instructions we have throughout.

"Do not guess or assume. It is better to pause and confirm than to generate incorrect code."

@ayeshurun
Copy link
Collaborator Author

Looks amazing, just one small note. I think the language quoted below is not explicit enough and too up for interpretation. I think it can just be removed because it should be clear enough from the first part of this rule plus the other rules/instructions we have throughout.

"Do not guess or assume. It is better to pause and confirm than to generate incorrect code."

resolved. :)

shirasassoon
shirasassoon previously approved these changes Mar 22, 2026
@ayeshurun ayeshurun merged commit 89db746 into microsoft:main Mar 22, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants