-
Notifications
You must be signed in to change notification settings - Fork 5.6k
[PoC] Add TypeSpec Authoring Skill #39869
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
…into typespec-authoring-agent
…/azure-rest-api-specs into typespec-authoring-agent
Next Steps to MergeNext steps that must be taken to merge this PR:
Comment generated by summarize-checks workflow run. |
| @@ -0,0 +1,50 @@ | |||
| --- | |||
| name: azure-typespec-author | |||
| description: 'Author and update Azure TypeSpec (.tsp) safely by retrieving authoritative solution with azsdk_typespec_retrieve_solution, then applying minimal changes and validating.' | |||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think this description is not enough for LLM to decide when to trigger the skill. I think you need to include more details of this skill as well as some sample prompt that would trigger this skill.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yea, thanks for pointing out. as I wrote in the PR's description (paste in below), I did not get time to tune the triggerring part yet. Just first use the prompt use azure-typespec-author skill to trigger the skill directly and make sure 'typespec authoring' skill works.
I will tune the instructions/descriptions about the triggering part.
Sample user prompt:
use azure-typespec-author skill to add a new preview version "2025-10-01-preview" to widget resource managerPS: Have not tuned the instructions, just try to call the skill directly in user prompt
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Previously I'm planning to tune the instructions of invocation/triggering part once we decide to go with the Skills only approach (I'm hoping we can decide on Thursday's sync up meeting between Shanghai and Redmond), because
- If we choose custom agent approach to go with first, we may not need to tune the instructions.
- The user can use the prompt
use azure-typespec-author skill to xxxto trigger our typespec authoring solution, so it does not block users from trying our solution. - To me, tuning the instruction/description of triggering skill may not be the major concern of choosing the design approach. To me, the major difference of the two approaches are:
- user experience: whether we leave it to the agent to trigger the skill, or let users choose to use our custom agent. The former is more transparent to user. The latter is more determinstic.
- whether we need a clean context window or not. More details can be found in this slides (Thanks @maririos for sharing!!).
@lirenhe what do you think? I'm happy to tune the description/instruction parts, but just I might prefer not blocking our design approach choice.
This PR is a PoC to try the design approach based on @chunyu3's custom agent PR : use skill only to author TypeSpec. See the detailed design approach here for more details.
Try out Steps:
feature/azsdk-qa-bot branchin Crystal's PR [typespec authoring] azsdk_typespec_consult MCP tool azure-sdk-tools#13122 and rundotnet build{ "servers": { "azure-sdk-mcp": { "type": "stdio", "command": "C:\\workspace\\azure-sdk-tools\\artifacts\\bin\\Azure.Sdk.Tools.Cli\\Debug\\net8.0\\azsdk", "args": [ "start" ] } } }Sample user prompt:
PS: Have not tuned the instructions, just try to call the skill directly in user prompt
Agent output