Reframe model evaluation ops and pipelines & Fix bug in compiled pl with local llm_serving #430
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
SunnyHaze
reviewed
Jan 5, 2026
Collaborator
SunnyHaze
left a comment
SunnyHaze
left a comment
There was a problem hiding this comment.
这个eval和之前的eval算子和statics/api_pipeline底下的脚本是否有“替代关系”？
- 如果有，请删掉之前的
- 如果没有，请尽量从命名和之前的遗留pipeline加以区分；此外，目前的type1 type2 type3感觉有点怪，可以在后面加后缀来进一步说明不同type的功能，不然看起来不直观？
Contributor
Author
|
已修复为讨论后的暂定方案，后续会补充doc中详细的每个type的使用方法。 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Refine model evaluation ops and pipelines , supporting all types of single modal benchmark eval.
& Fix bug in compiled pl with llmserving