Skip to content

Fix flaky AutocompletePrompt tests under React 19 on CI#6929

Open
tewson wants to merge 1 commit intomainfrom
fix/autocomplete-prompt-flaky-ci
Open

Fix flaky AutocompletePrompt tests under React 19 on CI#6929
tewson wants to merge 1 commit intomainfrom
fix/autocomplete-prompt-flaky-ci

Conversation

@tewson
Copy link
Member

@tewson tewson commented Mar 3, 2026

Summary

  • Fix the Stdin mock to re-emit 'readable' when a listener is added and there's pending data. This mirrors real Node stream behavior and prevents dropped input when data is written before Ink's useInput effect registers its listener — the root cause of the "allows searching with pagination" timeout on slow CI runners.
  • Replace sendInputAndWait fixed delays in the "doesn't submit if there are no choices" and "has a loading state" tests with sendInputAndWaitForContent, which deterministically waits for React to confirm the frame still shows expected content after the keypress
  • Use a never-resolving search promise in the "has a loading state" test so the loading state persists for the entire test, eliminating races where React 19's batched rendering could resolve the search and transition back to Idle before the ENTER keypress is processed

Test plan

  • pnpm --filter @shopify/cli-kit vitest run src/private/node/ui/components/AutocompletePrompt.test.tsx passes locally
  • pnpm --filter @shopify/app vitest run src/cli/services/dev/ui/components/Dev.test.tsx src/cli/services/dev/ui/components/DevSessionUI.test.tsx passes locally
  • CI passes without flaky failures

🤖 Generated with Claude Code

@tewson tewson force-pushed the fix/autocomplete-prompt-flaky-ci branch 4 times, most recently from 6ff3811 to 1359d8d Compare March 3, 2026 13:43
@github-actions
Copy link
Contributor

github-actions bot commented Mar 3, 2026

Coverage report

St.
Category Percentage Covered / Total
🟡 Statements 78.81% 14520/18425
🟡 Branches 73.12% 7220/9874
🟡 Functions 79.04% 3695/4675
🟡 Lines 79.14% 13719/17336

Test suite run success

3785 tests passing in 1448 suites.

Report generated by 🧪jest coverage report action from 599317f

Base automatically changed from feature/react-19-ink-6 to main March 3, 2026 13:53
@tewson tewson force-pushed the fix/autocomplete-prompt-flaky-ci branch from 1359d8d to 36cf9d6 Compare March 3, 2026 13:55
@tewson tewson marked this pull request as ready for review March 3, 2026 13:57
@tewson tewson requested a review from a team as a code owner March 3, 2026 13:57
@github-actions
Copy link
Contributor

github-actions bot commented Mar 3, 2026

We detected some changes at packages/*/src and there are no updates in the .changeset.
If the changes are user-facing, run pnpm changeset add to track your changes and include them in the next release CHANGELOG.

Caution

DO NOT create changesets for features which you do not wish to be included in the public changelog of the next CLI release.

@tewson tewson marked this pull request as draft March 3, 2026 14:02
@tewson tewson force-pushed the fix/autocomplete-prompt-flaky-ci branch from 36cf9d6 to c1ca594 Compare March 3, 2026 14:05
@binks-code-reviewer
Copy link

binks-code-reviewer bot commented Mar 3, 2026

🤖 Code Review · #projects-dev-ai for questions
React with 👍/👎 or reply — all feedback helps improve the agent.

Complete - 1 findings

📋 History

❌ Failed → ✅ 1 findings

@tewson tewson force-pushed the fix/autocomplete-prompt-flaky-ci branch from c1ca594 to 58a10a9 Compare March 3, 2026 14:23
@github-actions
Copy link
Contributor

github-actions bot commented Mar 3, 2026

Differences in type declarations

We detected differences in the type declarations generated by Typescript for this branch compared to the baseline ('main' branch). Please, review them to ensure they are backward-compatible. Here are some important things to keep in mind:

  • Some seemingly private modules might be re-exported through public modules.
  • If the branch is behind main you might see odd diffs, rebase main into this branch.

New type declarations

We found no new type declarations in this PR

Existing type declarations

packages/cli-kit/dist/private/node/testing/ui.d.ts
@@ -39,8 +39,12 @@ interface RenderOptions {
 export declare const render: (tree: ReactElement, options?: RenderOptions) => Instance;
 /**
  * Wait for the component to be ready to accept input.
+ *
+ * Listens for Ink's useInput effect to register its 'readable' listener
+ * on stdin via EventEmitter's built-in 'newListener' event — fully
+ * event-driven with no polling or sleeping.
  */
-export declare function waitForInputsToBeReady(): Promise<unknown>;
+export declare function waitForInputsToBeReady(renderInstance?: ReturnType<typeof render>): Promise<void>;
 /**
  * Wait for the last frame to change to anything.
  */

@tewson tewson force-pushed the fix/autocomplete-prompt-flaky-ci branch from 58a10a9 to 33db832 Compare March 3, 2026 14:36
Fix the Stdin mock to re-emit 'readable' when a listener is added and
there's pending data. This mirrors real Node stream behavior and
prevents dropped input when data is written before Ink's useInput
effect registers its listener — the root cause of the "allows
searching with pagination" timeout on slow CI runners.

Replace sendInputAndWait fixed delays with sendInputAndWaitForContent
in the "no submit" tests, which deterministically waits for React to
confirm the frame still shows expected content after the keypress.

Use a never-resolving search promise in the loading state test so the
loading state persists for the entire test, eliminating races with
React 19's batched rendering.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@tewson tewson force-pushed the fix/autocomplete-prompt-flaky-ci branch from 33db832 to 599317f Compare March 3, 2026 14:57
@tewson tewson marked this pull request as ready for review March 3, 2026 17:08
this.on('newListener', (event: string) => {
if (event === 'readable' && this.data !== null) {
setImmediate(() => this.emit('readable'))
}

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unbounded setImmediate re-emit on every readable listener addition (can cause hangs/flakes)

The newListener handler schedules setImmediate(() => this.emit('readable')) whenever a 'readable' listener is added and data !== null.

Evidence (current code):

this.on('newListener', (event: string) => {
  if (event === 'readable' && this.data !== null) {
    setImmediate(() => this.emit('readable'))
  }
})

If a consumer adds multiple 'readable' listeners (or re-attaches on rerenders / effect re-runs), you’ll queue a new setImmediate each time while data remains non-null. If the consumer’s readable handler doesn’t drain via .read() (or drains later), this can lead to a large backlog of immediates continually emitting 'readable', potentially starving the event loop and causing test timeouts/flakiness.

Impact:

  • User impact: Indirect but real—CI can become flaky/hang in UI tests (Ink/React) that manipulate listeners during renders/effects.
  • Infra impact: Increased CPU + longer test runtime; worst case, stuck test workers in CI.
  • Scale: Affects any test using this Stdin mock (shared helper) where listeners are added while buffered data exists.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant