Skip to content

perf: coalesce empty checkpoints in batch#337

Closed
yaythomas wants to merge 1 commit intomainfrom
feat/coalesce-empty-checkpoints-325
Closed

perf: coalesce empty checkpoints in batch#337
yaythomas wants to merge 1 commit intomainfrom
feat/coalesce-empty-checkpoints-325

Conversation

@yaythomas
Copy link
Copy Markdown
Contributor

Issue #, if available:
closes #325

Description of changes:
Taking over this excellent PR from #336, just to fix some minor formatting issues. Didn't have push perms on contributor branch.

Empty checkpoints (used by map/parallel branch resubmitters when resuming from timed waits) no longer count toward the 250-operation batch limit beyond the first. This prevents 300+ concurrent branch resumes from splitting across multiple API batches.

from original PR:
Implements the empty-checkpoint coalescing optimization described in #325.

When many concurrent map/parallel branches resume from a wait operation simultaneously, each branch's resubmitter enqueues an empty checkpoint (no OperationUpdate). Previously, these counted toward the 250-operation batch limit, causing 300 concurrent branches to produce 2 API calls (250 + 50) instead of 1.

  • Tracks effective_operation_count separately from len(batch)
  • The first empty checkpoint in a batch counts as 1 effective operation
  • All subsequent empty checkpoints in the same batch are coalesced in without incrementing the effective count
  • Applied consistently across all three collection phases: overflow queue drain, first-op blocking wait, and time-window loop
  • Updated debug logging to clearly show real ops sent vs empties excluded

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Empty checkpoints (used by map/parallel branch resubmitters when resuming from timed waits) no longer count toward the 250-operation batch limit beyond the first. This prevents 300+ concurrent branch resumes from splitting across multiple API batches.

closes #325

Co-authored-by: Aditya Ganti <aditya.ganti95@gmail.com>
@yaythomas yaythomas force-pushed the feat/coalesce-empty-checkpoints-325 branch from 5bbf198 to 677980f Compare April 6, 2026 17:51
@yaythomas
Copy link
Copy Markdown
Contributor Author

not necessary. got push perms on o.g pr

@yaythomas yaythomas closed this Apr 6, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature]: concurrency performance improvement on nested waits/retries

2 participants