Skip to content

Conversation

@abey79
Copy link
Member

@abey79 abey79 commented Nov 6, 2025

Related

What

I want to introduce a caching mechanism for dataset schema, since this operation is currently very expansive and often used in dataframe queries. A simple way to invalidate that cache would be to use the last_updated_at field that we maintain anyways. As a first step, this PR makes it more robust to maintain that last_update_at flag by introducing a Tracked helper which does that automagically.

Also introduces a couple of tests.

@abey79 abey79 added exclude from changelog PRs with this won't show up in CHANGELOG.md OSS-server labels Nov 6, 2025
@github-actions
Copy link

github-actions bot commented Nov 6, 2025

Web viewer built successfully.

Result Commit Link Manifest
545bbe9 https://rerun.io/viewer/pr/11807 +nightly +main

View image diff on kitdiff.

Note: This comment is updated whenever you push a commit.

@abey79 abey79 mentioned this pull request Nov 6, 2025
@abey79 abey79 marked this pull request as ready for review November 6, 2025 16:04
@abey79 abey79 requested a review from Copilot November 6, 2025 16:20
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR introduces a Tracked<T> wrapper type to automatically maintain last_updated_at timestamps for dataset state mutations. The wrapper provides immutable access via Deref and mutable access via a guard pattern that updates timestamps on drop. The implementation refactors Dataset and Partition to use Tracked for their mutable state, ensuring timestamp updates occur automatically whenever state is modified.

Key changes:

  • New Tracked<T> type with guard-based mutation tracking
  • Refactored Dataset and Partition to wrap mutable state in Tracked
  • Added tests verifying timestamp updates on rename and partition registration operations

Reviewed Changes

Copilot reviewed 8 out of 9 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
crates/store/re_server/src/store/tracked.rs Introduces the new Tracked<T> wrapper and TrackedGuard types for automatic timestamp tracking
crates/store/re_server/src/store/partition.rs Refactors Partition to use Tracked<PartitionInner> for automatic timestamp updates
crates/store/re_server/src/store/mod.rs Exports the new Tracked type from the store module
crates/store/re_server/src/store/dataset.rs Refactors Dataset to use Tracked<DatasetInner> and adds updated_at() accessor
crates/store/re_redap_tests/src/tests/update_entry.rs Adds test verifying timestamp updates when renaming datasets
crates/store/re_redap_tests/src/tests/register_partition.rs Adds test verifying timestamp updates when registering partitions and layers
crates/store/re_redap_tests/src/tests/mod.rs Registers the new timestamp-related tests
crates/store/re_redap_tests/Cargo.toml Adds tokio dependency for async test delays

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copy link
Member

@oxkitsune oxkitsune left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

very elegant and satisfying solution!

Comment on lines -105 to +120
self.details = details;
self.updated_at = jiff::Timestamp::now();
self.inner.modify().details = details;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Similarly to the set_name shouldn't it be trivial to check whether the new details are actually new? In the case where they are, this will update the last_updated_timestamp even though nothing changed.

DatasetDetails can implement PartialEq and Eq, all its components already do.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

exclude from changelog PRs with this won't show up in CHANGELOG.md OSS-server

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants