Skip to content

tests: serialize conflictive runtime tests (perf improvement)#11546

Open
edsiper wants to merge 1 commit intomasterfrom
unit-tests-parallel
Open

tests: serialize conflictive runtime tests (perf improvement)#11546
edsiper wants to merge 1 commit intomasterfrom
unit-tests-parallel

Conversation

@edsiper
Copy link
Member

@edsiper edsiper commented Mar 12, 2026

This change improves unit-test wall-clock time in CI by allowing the runtime suite to run safely in parallel under ctest -j, while explicitly serializing only the known conflicting cases.

What changed:

  • Added CTest RESOURCE_LOCK handling in tests/runtime/CMakeLists.txt:300
  • Serialized only the runtime executables that bind the same fixed ports:
    • flb-rt-core_internal_logger and flb-rt-filter_wasm on 2020
    • flb-rt-in_opentelemetry and flb-rt-in_opentelemetry_routing on 4318
    • flb-rt-in_tcp and flb-rt-out_tcp on 5170
  • No CI workflow changes were required because GitHub Actions already invokes ctest -j $nparallel

Measured impact:

  • Sequential internal+runtime suite: about 2321.23s (38m 41s)
  • Parallel internal+runtime suite with locks: about 299.53s (4m 59s)
  • Reduction: about 87.1% wall-clock time

Fluent Bit is licensed under Apache 2.0, by submitting this pull request I understand that this code will be released under the terms of that license.

Summary by CodeRabbit

Release Notes

  • Tests
    • Improved reliability of runtime tests by preventing concurrent execution conflicts on shared resources.

@edsiper edsiper requested a review from cosmo0920 as a code owner March 12, 2026 13:33
@edsiper edsiper added this to the Fluent Bit v5.0 milestone Mar 12, 2026
@coderabbitai
Copy link

coderabbitai bot commented Mar 12, 2026

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: f930636b-bf36-47fc-8d19-0318ab4d81e4

📥 Commits

Reviewing files that changed from the base of the PR and between 405fed0 and d950b04.

📒 Files selected for processing (1)
  • tests/runtime/CMakeLists.txt

📝 Walkthrough

Walkthrough

A new CMake function flb_runtime_lock_tests() is introduced to add RESOURCE_LOCK properties to runtime tests, enabling test-level serialization to prevent concurrent execution of port-dependent tests that would contend on the same resource.

Changes

Cohort / File(s) Summary
Test Serialization Configuration
tests/runtime/CMakeLists.txt
Added flb_runtime_lock_tests(resource_name) function that iterates over test names and applies RESOURCE_LOCK properties to serialize execution of tests sharing the same resource.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~6 minutes

Poem

🐰 A rabbit hops through CMake lines,
With locks and tests aligned,
No more shall ports collide or race,
Each test finds its rightful place! 🔒

🚥 Pre-merge checks | ✅ 3
✅ Passed checks (3 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly and concisely summarizes the main change: introducing test serialization for conflicting runtime tests to improve CI performance.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment
  • Commit unit tests in branch unit-tests-parallel
📝 Coding Plan for PR comments
  • Generate coding plan

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Signed-off-by: Eduardo Silva <eduardo@chronosphere.io>
@edsiper edsiper force-pushed the unit-tests-parallel branch from 2d04719 to d950b04 Compare March 12, 2026 13:35
Copy link
Contributor

@cosmo0920 cosmo0920 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A big 👍!
I was struggled with handling this resource competed test cases.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants