Skip to content

Conversation

@haolingdong-msft
Copy link
Member

@haolingdong-msft haolingdong-msft commented Jan 23, 2026

This PR is a PoC to try the design approach based on @chunyu3's custom agent PR : use skill only to author TypeSpec. See the detailed design approach here for more details.

Try out Steps:

  1. checkout on this branch feature/azsdk-qa-bot branch in Crystal's PR [typespec authoring] azsdk_typespec_consult MCP tool azure-sdk-tools#13122 and run dotnet build
  2. update mcp.json to use the latest built azsdk
{
  "servers": {
    "azure-sdk-mcp": {
      "type": "stdio",
      "command": "C:\\workspace\\azure-sdk-tools\\artifacts\\bin\\Azure.Sdk.Tools.Cli\\Debug\\net8.0\\azsdk",
      "args": [
        "start"
      ]
    }
  }
}

Sample user prompt:

use azure-typespec-author skill to add a new preview version "2025-10-01-preview" to widget resource manager

PS: Have not tuned the instructions, just try to call the skill directly in user prompt

Agent output

image image image

@github-actions
Copy link

github-actions bot commented Jan 23, 2026

Next Steps to Merge

Next steps that must be taken to merge this PR:
  • ❌ The required check named Protected Files has failed. Refer to the check in the PR's 'Checks' tab for details on how to fix it and consult the aka.ms/ci-fix guide


Comment generated by summarize-checks workflow run.

@haolingdong-msft haolingdong-msft changed the title [PoC]Add TypeSpec Authoring Skill [PoC] Add TypeSpec Authoring Skill Jan 23, 2026
@@ -0,0 +1,50 @@
---
name: azure-typespec-author
description: 'Author and update Azure TypeSpec (.tsp) safely by retrieving authoritative solution with azsdk_typespec_retrieve_solution, then applying minimal changes and validating.'
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this description is not enough for LLM to decide when to trigger the skill. I think you need to include more details of this skill as well as some sample prompt that would trigger this skill.

Copy link
Member Author

@haolingdong-msft haolingdong-msft Jan 26, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yea, thanks for pointing out. as I wrote in the PR's description (paste in below), I did not get time to tune the triggerring part yet. Just first use the prompt use azure-typespec-author skill to trigger the skill directly and make sure 'typespec authoring' skill works.

I will tune the instructions/descriptions about the triggering part.

Sample user prompt:
use azure-typespec-author skill to add a new preview version "2025-10-01-preview" to widget resource manager

PS: Have not tuned the instructions, just try to call the skill directly in user prompt

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Previously I'm planning to tune the instructions of invocation/triggering part once we decide to go with the Skills only approach (I'm hoping we can decide on Thursday's sync up meeting between Shanghai and Redmond), because

  1. If we choose custom agent approach to go with first, we may not need to tune the instructions.
  2. The user can use the prompt use azure-typespec-author skill to xxx to trigger our typespec authoring solution, so it does not block users from trying our solution.
  3. To me, tuning the instruction/description of triggering skill may not be the major concern of choosing the design approach. To me, the major difference of the two approaches are:
    • user experience: whether we leave it to the agent to trigger the skill, or let users choose to use our custom agent. The former is more transparent to user. The latter is more determinstic.
    • whether we need a clean context window or not. More details can be found in this slides (Thanks @maririos for sharing!!).
    image

@lirenhe what do you think? I'm happy to tune the description/instruction parts, but just I might prefer not blocking our design approach choice.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants