Fix performance decreasing because using import padde in engineV2#623
Merged
DanielSun11 merged 4 commits intoPFCCLab:mainfrom Apr 3, 2026
Merged
Fix performance decreasing because using import padde in engineV2#623DanielSun11 merged 4 commits intoPFCCLab:mainfrom
DanielSun11 merged 4 commits intoPFCCLab:mainfrom
Conversation
|
Thanks for your contribution! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
修复engineV2直接import padde 引起的性能下降问题。
原因：spawn 模式下 worker 导入主模块触发 import paddle
Python spawn + ProcessPool 的工作机制
spawn 模式创建 worker 进程时，子进程是一个全新的 Python 解释器。为了能执行 init_worker_gpu 和 run_test_case 这两个定义在 engineV2.py（即 main）中的函数，pebble 的 worker 进程在启动时必须重新 import engineV2 模块，才能反序列化（unpickle）这些函数引用。
这意味着 engineV2.py 的所有模块级代码都会在每个 worker 进程里执行一遍，包括这次新增的顶层 import paddle。