[qs] Populate BatchKind for BatchV2 #18452

ibalajiarun · 2026-01-15T00:53:25Z

Description

This PR adds support for categorizing transactions into different batch kinds, specifically separating encrypted transactions from normal ones in the QuorumStore batch generation process. The implementation enables the batch generator to create separate batches for encrypted and normal transactions, ensuring they are processed appropriately.

Key changes:

Added a batch_kind() method to the TBatchInfo trait to identify batch types
Implemented transaction categorization logic in BatchGenerator to separate encrypted and normal transactions
Refactored batch generation to process transaction groups by batch kind
Added validation to ensure batch kinds match their transaction types

ibalajiarun · 2026-01-15T00:53:51Z

[test] enctx forge e2e #18478
dkg impl #1 #18441
BlockMetadataExtV2 with Decryption Key #18444
[qs] support pulling OptQuorumStorePayload::V2 #18453
[qs] Populate BatchKind for BatchV2 #18452 👈 (View in Graphite)
[api] local config flag to gate enc txn submission and batch validation #18086 : 1 other dependent PR (#18443 )
[batch-encryption] remove happy path in impls and traits #18445
main

This stack of pull requests is managed by Graphite. Learn more about stacking.

JoshLind · 2026-01-20T02:20:57Z

consensus/src/quorum_store/batch_generator.rs

-        if !pulled_txns.is_empty() && max_batches_remaining > 0 {
-            self.push_bucket_to_batches(
+            // Process each batch kind group. Processing order: Normal first, then others.
+            // This ensures backwards compatibility and prioritizes regular transactions.


Interesting... I wonder if we ever want to do the opposite, i.e., prioritize encrypted transactions? But, a question for another day 😄

github-actions · 2026-01-20T17:41:48Z

✅ Forge suite `compat` success on `c06e1b8824e5d9db7535c9eed058e24aa7c19bf1` ==> `4a7cef394b2ca8867e257c6f0c0d26a465caac61`

Compatibility test results for c06e1b8824e5d9db7535c9eed058e24aa7c19bf1 ==> 4a7cef394b2ca8867e257c6f0c0d26a465caac61 (PR)
1. Check liveness of validators at old version: c06e1b8824e5d9db7535c9eed058e24aa7c19bf1
compatibility::simple-validator-upgrade::liveness-check : committed: 13341.90 txn/s, latency: 2607.30 ms, (p50: 2700 ms, p70: 3000, p90: 3300 ms, p99: 3900 ms), latency samples: 437160
2. Upgrading first Validator to new version: 4a7cef394b2ca8867e257c6f0c0d26a465caac61
compatibility::simple-validator-upgrade::single-validator-upgrade : committed: 5830.50 txn/s, latency: 5801.34 ms, (p50: 6400 ms, p70: 6500, p90: 6600 ms, p99: 6800 ms), latency samples: 201140
3. Upgrading rest of first batch to new version: 4a7cef394b2ca8867e257c6f0c0d26a465caac61
compatibility::simple-validator-upgrade::half-validator-upgrade : committed: 5898.11 txn/s, latency: 5749.33 ms, (p50: 6400 ms, p70: 6500, p90: 6600 ms, p99: 6700 ms), latency samples: 202640
4. upgrading second batch to new version: 4a7cef394b2ca8867e257c6f0c0d26a465caac61
compatibility::simple-validator-upgrade::rest-validator-upgrade : committed: 9721.68 txn/s, latency: 3380.70 ms, (p50: 3300 ms, p70: 3900, p90: 4600 ms, p99: 5100 ms), latency samples: 322880
5. check swarm health
Compatibility test for c06e1b8824e5d9db7535c9eed058e24aa7c19bf1 ==> 4a7cef394b2ca8867e257c6f0c0d26a465caac61 passed
Test Ok

github-actions · 2026-01-20T17:46:05Z

✅ Forge suite `realistic_env_max_load` success on `4a7cef394b2ca8867e257c6f0c0d26a465caac61`

two traffics test: inner traffic : committed: 13495.35 txn/s, latency: 2794.17 ms, (p50: 2700 ms, p70: 2900, p90: 3000 ms, p99: 3600 ms), latency samples: 5026420
two traffics test : committed: 100.00 txn/s, latency: 734.17 ms, (p50: 700 ms, p70: 800, p90: 900 ms, p99: 1100 ms), latency samples: 1960
Latency breakdown for phase 0: ["MempoolToBlockCreation: max: 2.297, avg: 2.184", "ConsensusProposalToOrdered: max: 0.169, avg: 0.166", "ConsensusOrderedToCommit: max: 0.050, avg: 0.045", "ConsensusProposalToCommit: max: 0.216, avg: 0.211"]
Max non-epoch-change gap was: 1 rounds at version 37570 (avg 0.00) [limit 4], 1.16s no progress at version 37570 (avg 0.07s) [limit 15].
Max epoch-change gap was: 0 rounds at version 0 (avg 0.00) [limit 4], 0.56s no progress at version 5772456 (avg 0.42s) [limit 16].
Test Ok

github-actions · 2026-01-20T17:46:21Z

✅ Forge suite `framework_upgrade` success on `c06e1b8824e5d9db7535c9eed058e24aa7c19bf1` ==> `4a7cef394b2ca8867e257c6f0c0d26a465caac61`

Compatibility test results for c06e1b8824e5d9db7535c9eed058e24aa7c19bf1 ==> 4a7cef394b2ca8867e257c6f0c0d26a465caac61 (PR)
Upgrade the nodes to version: 4a7cef394b2ca8867e257c6f0c0d26a465caac61
framework_upgrade::framework-upgrade::full-framework-upgrade : committed: 2173.85 txn/s, submitted: 2181.34 txn/s, failed submission: 7.49 txn/s, expired: 7.49 txn/s, latency: 1330.06 ms, (p50: 1200 ms, p70: 1500, p90: 1800 ms, p99: 2400 ms), latency samples: 197240
framework_upgrade::framework-upgrade::full-framework-upgrade : committed: 2255.77 txn/s, submitted: 2264.74 txn/s, failed submission: 8.97 txn/s, expired: 8.97 txn/s, latency: 1281.87 ms, (p50: 1200 ms, p70: 1500, p90: 1800 ms, p99: 2400 ms), latency samples: 206120
5. check swarm health
Compatibility test for c06e1b8824e5d9db7535c9eed058e24aa7c19bf1 ==> 4a7cef394b2ca8867e257c6f0c0d26a465caac61 passed
Upgrade the remaining nodes to version: 4a7cef394b2ca8867e257c6f0c0d26a465caac61
framework_upgrade::framework-upgrade::full-framework-upgrade : committed: 2287.35 txn/s, submitted: 2295.52 txn/s, failed submission: 8.16 txn/s, expired: 8.16 txn/s, latency: 1274.11 ms, (p50: 1200 ms, p70: 1400, p90: 1700 ms, p99: 2100 ms), latency samples: 207362
Test Ok

This was referenced Jan 15, 2026

[api] local config flag to gate enc txn submission and batch validation #18086

Merged

dkg impl #1 #18441

Draft

[qs] Populate BatchKind for BatchV2 #18443

Closed

BlockMetadataExtV2 with Decryption Key #18444

Draft

This was referenced Jan 15, 2026

[batch-encryption] remove happy path in impls and traits #18445

Merged

[qs] support pulling OptQuorumStorePayload::V2 #18453

Merged

ibalajiarun marked this pull request as ready for review January 15, 2026 00:55

ibalajiarun requested review from bchocho, sasha8 and zekun000 as code owners January 15, 2026 00:55

ibalajiarun requested review from JoshLind and danielxiangzl January 15, 2026 00:57

ibalajiarun force-pushed the balaji/batch-kind branch from 8066269 to f9f72c8 Compare January 15, 2026 19:41

ibalajiarun force-pushed the balaji/enc-pool-5 branch from aea0d98 to b94c8a8 Compare January 16, 2026 00:11

ibalajiarun requested review from 0xmaayan, banool and gregnazario as code owners January 16, 2026 00:11

danielxiangzl approved these changes Jan 16, 2026

View reviewed changes

ibalajiarun force-pushed the balaji/enc-pool-5 branch from b94c8a8 to c8fe203 Compare January 16, 2026 00:19

ibalajiarun force-pushed the balaji/batch-kind branch from f9f72c8 to 8f474b1 Compare January 16, 2026 00:19

ibalajiarun force-pushed the balaji/enc-pool-5 branch 2 times, most recently from 20044f2 to 93e5285 Compare January 16, 2026 01:22

Base automatically changed from balaji/enc-pool-5 to main January 16, 2026 02:54

[qs] Populate BatchKind for BatchV2

4a7cef3

ibalajiarun force-pushed the balaji/batch-kind branch from 8f474b1 to 4a7cef3 Compare January 16, 2026 03:02

JoshLind approved these changes Jan 20, 2026

View reviewed changes

ibalajiarun enabled auto-merge (squash) January 20, 2026 17:07

This comment has been minimized.

Sign in to view

ibalajiarun merged commit 41b19a6 into main Jan 20, 2026
90 checks passed

ibalajiarun deleted the balaji/batch-kind branch January 20, 2026 17:52

ibalajiarun mentioned this pull request Jan 20, 2026

[test] enctx forge e2e #18478

Draft

22 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[qs] Populate BatchKind for BatchV2 #18452

[qs] Populate BatchKind for BatchV2 #18452

ibalajiarun commented Jan 15, 2026 •

edited

Loading

Uh oh!

ibalajiarun commented Jan 15, 2026 •

edited

Loading

Uh oh!

JoshLind Jan 20, 2026

Uh oh!

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

github-actions bot commented Jan 20, 2026

Uh oh!

github-actions bot commented Jan 20, 2026

Uh oh!

github-actions bot commented Jan 20, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

[qs] Populate BatchKind for BatchV2 #18452

[qs] Populate BatchKind for BatchV2 #18452

Conversation

ibalajiarun commented Jan 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Uh oh!

ibalajiarun commented Jan 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

JoshLind Jan 20, 2026

Choose a reason for hiding this comment

Uh oh!

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

github-actions bot commented Jan 20, 2026

✅ Forge suite compat success on c06e1b8824e5d9db7535c9eed058e24aa7c19bf1 ==> 4a7cef394b2ca8867e257c6f0c0d26a465caac61

Uh oh!

github-actions bot commented Jan 20, 2026

✅ Forge suite realistic_env_max_load success on 4a7cef394b2ca8867e257c6f0c0d26a465caac61

Uh oh!

github-actions bot commented Jan 20, 2026

✅ Forge suite framework_upgrade success on c06e1b8824e5d9db7535c9eed058e24aa7c19bf1 ==> 4a7cef394b2ca8867e257c6f0c0d26a465caac61

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

ibalajiarun commented Jan 15, 2026 •

edited

Loading

ibalajiarun commented Jan 15, 2026 •

edited

Loading

✅ Forge suite `compat` success on `c06e1b8824e5d9db7535c9eed058e24aa7c19bf1` ==> `4a7cef394b2ca8867e257c6f0c0d26a465caac61`

✅ Forge suite `realistic_env_max_load` success on `4a7cef394b2ca8867e257c6f0c0d26a465caac61`

✅ Forge suite `framework_upgrade` success on `c06e1b8824e5d9db7535c9eed058e24aa7c19bf1` ==> `4a7cef394b2ca8867e257c6f0c0d26a465caac61`