Skip to content

Conversation

@fresh-borzoni
Copy link
Contributor

Purpose

Linked issue: close #162

Implement the ReadContext pattern and typed row decoding for the KV record read path, mirroring the Java reference implementation. This enables reading KV records with typed CompactedRow access while maintaining performance through decoder caching.

Core Infrastructure:

  • Added ReadContext trait for providing schema-specific row decoders
  • Implemented KvRecordReadContext with double-check locking for decoder caching
  • Created RowDecoder trait abstraction with CompactedRowDecoder implementation
  • Added SchemaGetter trait for fallible schema retrieval

API Changes:

  • Modified KvRecordBatch::records() to accept &dyn ReadContext parameter
  • Updated KvRecord to support typed decoding via row(&dyn RowDecoder) method
  • Added KvRecordIterator::row() helper method for ergonomic typed access
  • Iterator yields raw KvRecord (no Arc clone per record)

Design Decisions:

  • Batch remains stateless (ReadContext passed as parameter, not stored)
  • Decoder cached by schema ID in ReadContext (performance optimization)
  • On-demand decoding: stores raw bytes, decodes via helper method
  • Mutex poison recovery for cache robustness

Test Improvements:

  • Consolidated tests from 19 → 12
  • All tests use typed API (not raw bytes at batch level)
  • Full coverage maintained for read path, caching, and error handling

API and Format

API Changes:

  • New public API: ReadContext, SchemaGetter, RowDecoder traits
  • KvRecordBatch::records() now requires &dyn ReadContext parameter
  • KvRecord::row() returns Option<CompactedRow<'_>> with decoder parameter
  • KvRecordIterator::row() helper for ergonomic typed access

Storage Format:

  • No changes to on-disk format
  • Binary compatibility maintained

Documentation

New Features:

  • ReadContext pattern for decoder management and caching
  • Typed row access via RowDecoder abstraction

@fresh-borzoni
Copy link
Contributor Author

@luoyuxia @leekeiabstraction @Kelvinyu1117
PTAL 🙏

Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR implements the ReadContext pattern and typed row decoding for the KV record read path, mirroring the Java reference implementation. It enables reading KV records with typed CompactedRow access while maintaining performance through decoder caching.

Changes:

  • Added ReadContext trait and KvRecordReadContext implementation with efficient double-check locking for decoder caching
  • Created RowDecoder trait abstraction with CompactedRowDecoder implementation for typed row decoding
  • Modified KV record API to support on-demand typed decoding via KvRecord::row() method with decoder parameter
  • Consolidated tests from 19 to 12 while maintaining full coverage, all using the new typed API

Reviewed changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
crates/fluss/src/row/row_decoder.rs New row decoder trait and CompactedRowDecoder implementation with factory
crates/fluss/src/row/mod.rs Exports new RowDecoder types
crates/fluss/src/record/kv/read_context.rs ReadContext trait for providing schema-specific row decoders
crates/fluss/src/record/kv/kv_record_read_context.rs Production ReadContext implementation with decoder caching
crates/fluss/src/record/kv/mod.rs Module exports and TestReadContext utility for simplified testing
crates/fluss/src/record/kv/kv_record.rs API changes: value_bytes field rename, new row() and is_deletion() methods
crates/fluss/src/record/kv/kv_record_batch.rs API changes: records() now accepts ReadContext, returns KvRecords wrapper
crates/fluss/src/record/kv/kv_record_batch_builder.rs Test consolidation and updates to use typed API

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copy link
Contributor

@luoyuxia luoyuxia left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@fresh-borzoni Thanks for the pr. Left minor comments. PTAL

@fresh-borzoni
Copy link
Contributor Author

fresh-borzoni commented Jan 17, 2026

@luoyuxia Thanks for the review
Addressed, PTAL 🙏

Copy link
Contributor

@luoyuxia luoyuxia left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@fresh-borzoni Thanks. LGTM

@luoyuxia luoyuxia merged commit f4808d8 into apache:main Jan 17, 2026
13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add KvReadContext to KvRecordBatchBuilder and return CompactedRows on read_from

2 participants