Skip to content

test_runner: make skip/todo/expectFailure use JS truthiness#62346

Open
VaishnavIUpadyaya wants to merge 4 commits intonodejs:mainfrom
VaishnavIUpadyaya:test-runner-truthy-fix
Open

test_runner: make skip/todo/expectFailure use JS truthiness#62346
VaishnavIUpadyaya wants to merge 4 commits intonodejs:mainfrom
VaishnavIUpadyaya:test-runner-truthy-fix

Conversation

@VaishnavIUpadyaya
Copy link

Fixes #61815

Updates skip, todo, and expectFailure handling in the test runner
to use JavaScript truthiness instead of explicit undefined/false checks.

This ensures:

  • Empty string ('') is treated as false
  • Behavior matches the documented "truthy" semantics

Adds a test to verify that skip: '' does not skip the test.

@nodejs-github-bot
Copy link
Collaborator

Review requested:

  • @nodejs/test_runner

@nodejs-github-bot nodejs-github-bot added needs-ci PRs that need a full CI run. test_runner Issues and PRs related to the test runner subsystem. labels Mar 20, 2026
Copy link
Member

@JakobJingleheimer JakobJingleheimer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like the doc already state truthiness too.

@JakobJingleheimer JakobJingleheimer added request-ci Add this label to start a Jenkins CI on a PR. commit-queue-squash Add this label to instruct the Commit Queue to squash all the PR commits into the first one. labels Mar 22, 2026
@codecov
Copy link

codecov bot commented Mar 22, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 89.68%. Comparing base (1989f4d) to head (d10c419).
⚠️ Report is 66 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main   #62346      +/-   ##
==========================================
+ Coverage   89.65%   89.68%   +0.02%     
==========================================
  Files         676      676              
  Lines      206555   206694     +139     
  Branches    39547    39577      +30     
==========================================
+ Hits       185195   185377     +182     
+ Misses      13495    13446      -49     
- Partials     7865     7871       +6     
Files with missing lines Coverage Δ
lib/internal/test_runner/test.js 96.80% <100.00%> (+<0.01%) ⬆️

... and 47 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

const common = require('../common');
const { test } = require('node:test');

test('skip option empty string should not skip', { skip: '' }, common.mustCall());
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why not? this seems wrong to me

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would fix the docs, not the implementation - it seems for correct

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can see why both behaviors would make sense.
Do we know what other test runners do?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the feedback!

I based this change on the docs describing these options as "truthy", so I assumed an empty string ('') should be treated as false.

That said, I see your point — changing the implementation could introduce unintended breaking behavior. Updating the docs to reflect the current behavior sounds reasonable.

I'm happy to update this PR to focus on the docs instead. Let me know what you prefer!

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think there's a decent chance an empty string is an accident, but the user could be relying on something that outputs a potentially empty string.

I think in general, truthy/falsy makes sense; since that includes an empty string, yeah.

I'm not sure what others do. I'll take a look this evening.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks like jest and mocha do not have this—they expose them only as methods like test.only('…') and test.skip('…').

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tape has this, and uses truthiness.

@MoLow MoLow removed the commit-queue-squash Add this label to instruct the Commit Queue to squash all the PR commits into the first one. label Mar 25, 2026
@github-actions github-actions bot removed the request-ci Add this label to start a Jenkins CI on a PR. label Mar 26, 2026
@nodejs-github-bot
Copy link
Collaborator

@@ -679,7 +680,7 @@ class Test extends AsyncResource {
this.expectedAssertions = plan;
this.cancelled = false;
this.expectFailure = parseExpectFailure(expectFailure) || this.parent?.expectFailure;
this.skipped = skip !== undefined && skip !== false;
this.skipped = Boolean(skip);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
this.skipped = Boolean(skip);
this.skipped = !!skip;

nit

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

needs-ci PRs that need a full CI run. test_runner Issues and PRs related to the test runner subsystem.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

test_runner: todo/skip/expectFailure are truthy per docs, but implementation is otherwise

7 participants