Skip phantom entry targets in subgraph when enablePhantomTargetOptimization is enabled#1102
Skip phantom entry targets in subgraph when enablePhantomTargetOptimization is enabled#1102christiango wants to merge 7 commits intomainfrom
enablePhantomTargetOptimization is enabled#1102Conversation
|
I need to test something, going to move to draft |
| // don't define the task script. Without this, phantom entry targets pull their | ||
| // same-package dependencies into the subgraph unnecessarily. | ||
| if (this.options.enablePhantomTargetOptimization) { | ||
| const pkg = this.options.packageInfos[packageName]; |
There was a problem hiding this comment.
This is too aggressive. We need to make sure it's an npm script when we do this
packages/target-graph/src/__tests__/WorkspaceTargetGraphBuilder.test.ts
Outdated
Show resolved
Hide resolved
| // Only npmScript targets can be phantom — other types (worker, noop, etc.) | ||
| // are skipped only when shouldRun is explicitly false. | ||
| const targetId = getTargetId(packageName, task); | ||
| if (this.options.enablePhantomTargetOptimization) { |
There was a problem hiding this comment.
I'm going to simplify this
| } | ||
| } else { | ||
| // For non-npmScript targets (e.g. worker), skip if shouldRun resolves to false | ||
| const config = this.targetConfigMap.get(targetId); |
There was a problem hiding this comment.
For worker runners, this has the side effect of resolving and loading the script module, which could be expensive (though I guess it would be cached after the first load). So I wonder if it's better to just ignore non-npmScript targets for the phantom target check here, similar to the other check in expandDepSpecs?
There was a problem hiding this comment.
So for our case we actually need it to filter out workers as well. I couldn't find any evidence of this triggering the worker script module, the intent is only to look at the worker config's should run (confusing that both places have a should run). I believe the worker still isn't loaded until the runner phase
There was a problem hiding this comment.
My tests show this as well, if I comment out the target config should run here, the test fails and the task is in the build graph
builder.addTargetConfig("test:special", {
type: "worker",
dependsOn: ["bundle"],
//shouldRun: async (target) => Promise.resolve(target.packageName === "tool-app"),
});
There was a problem hiding this comment.
As a further sanity check I did the following, I made the worker script in the repo throw on the first line and I added a console log statement before the this.hasRoottarget check on line 255. Here was the output, showing the worker is not loaded with this change:
This is when we finished processing the targets
Error: This is when the worker was loaded
There was a problem hiding this comment.
This is where it loads the script in shouldRun:
lage/packages/runners/src/WorkerRunner.ts
Lines 68 to 72 in d386157
There was a problem hiding this comment.
This is not the one I am checking , confusingly there’s also one in the config as well that gets read before the script. The test that skips when should run is false is implementing it at the config level, not the runner level
There was a problem hiding this comment.
In the test it’s defined in parallel to depends on in the config
Summary
Extends the existing phantom target optimization to also filter out phantom entry targets during subgraph construction. Previously,
enablePhantomTargetOptimizationonly skipped phantom targets during^/^^dependency expansion inexpandDepSpecs. This left a gap: when running a task that only some packages define or that explicitly opts out viashouldRun, every package still got added as a subgraph entry point, pulling in their same-package dependencies unnecessarily.Problem
When running a task like
test:commandScriptthat only a single package defines, lage creates entry targets for all packages in scope. Each phantom entry target's same-package dependency onbundlepulls that expensive bundle task into the subgraph. In a ~200 package monorepo, this inflated the target graph from ~91 targets to ~366 — including multiple 6GB+ bundle tasks that had no work to do.The same problem applies to non-npmScript targets (e.g.
workertype) that define ashouldRuncallback returningfalse— they still got added as entry points, dragging in their dependency trees.Fix
In
WorkspaceTargetGraphBuilder.build(), whenenablePhantomTargetOptimizationis true, skip adding a package tosubGraphEntriesif:shouldRunresolved tofalse(covers worker, noop, and any target type with ashouldRuncallback), ornpmScripttype and the package doesn't define the task in itspackage.jsonscripts (existing phantom target check, now also applied at entry selection)To support this,
shouldRunresolution is moved earlier inbuild()— before entry filtering instead of after subgraph construction. To minimize overhead:scope × tasks) are resolved, not all targetsshouldRuncallback are set synchronously totruewith no promise overheadpLimit(8)All downstream consumers already handle
shouldRun: undefinedcorrectly via?? trueor=== falsechecks, so non-entry targets that don't get resolved are unaffected.Risk
enablePhantomTargetOptimization(defaults to false), so no behavior change for existing consumers.expandDepSpecsandisPhantomTargetare completely untouched — dependency expansion logic is unchanged.WorkspaceTargetGraphBuilder.build().Testing
shouldRunreturnsfalse, one verifying worker targets withoutshouldRunare still included.