Introduce Temporal Graph Storage support to NativeStore #349

Nayanatharapmc · 2025-12-31T05:35:11Z

No description provided.

…DELETE operations, and improve timestamp resolution for consistency

…lQueryFilter

…essing

… and telemetry features

sonarqubecloud · 2025-12-31T05:40:02Z

Quality Gate failed

Failed conditions
17.3% Duplication on New Code (required ≤ 3%)

See analysis details on SonarQube Cloud

Copilot

Pull request overview

This PR introduces comprehensive temporal graph storage support to JasminGraph's NativeStore, enabling point-in-time and time-window queries over graph evolution. The implementation adds temporal metadata tracking (creation, update, deletion timestamps and status) to both nodes and edges, integrates temporal filtering into query execution, and provides an event logging system for audit trails.

Key Changes:

Temporal metadata tracking for nodes and edges with creation/update/deletion timestamps and status fields
Query-time temporal filtering with AS_OF (point-in-time) and RANGE (time-window) modes
Event logging system with JSONL-based audit trail, indexing, rotation, and compaction capabilities

Reviewed changes

Copilot reviewed 16 out of 16 changed files in this pull request and generated 11 comments.

Show a summary per file

File	Description
src/util/kafka/StreamHandler.h	Adds temporal metadata resolution method declarations
src/util/kafka/StreamHandler.cpp	Implements operation type and timestamp extraction from Kafka messages
src/query/processor/cypher/runtime/OperatorExecutor.cpp	CRITICAL: Contains unresolved merge conflicts - Integrates temporal filtering into all scan operators
src/nativestore/temporal/TemporalQueryFilter.h	Defines temporal constraint structures and filtering API
src/nativestore/temporal/TemporalQueryFilter.cpp	Implements temporal visibility evaluation with epoch millisecond normalization
src/nativestore/temporal/TemporalEventLogger.h	Defines event logging API with indexing, rotation, and compaction
src/nativestore/temporal/TemporalEventLogger.cpp	Implements JSONL-based event logging system
src/nativestore/temporal/TemporalConstants.h	Defines temporal metadata keys and timestamp utilities
src/nativestore/RelationBlock.h	Adds getAllMetaProperties method declaration
src/nativestore/RelationBlock.cpp	CRITICAL: Memory leak in getAllMetaProperties - Implements meta properties retrieval
src/nativestore/NodeBlock.h	Adds getAllMetaProperties method declaration
src/nativestore/NodeBlock.cpp	CRITICAL: Memory leak in getAllMetaProperties - Implements meta properties retrieval
src/localstore/incremental/JasmineGraphIncrementalLocalStore.h	Adds temporal operation handlers and event logger integration
src/localstore/incremental/JasmineGraphIncrementalLocalStore.cpp	CRITICAL: Unresolved merge conflict - Implements ADD/UPDATE/DELETE operations with temporal metadata attachment
TEMPORAL_DATA_FLOW.md	Comprehensive documentation of temporal data flow from ingestion to querying
CMakeLists.txt	Adds temporal source files to build (with duplicate entries)

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2025-12-31T05:42:43Z

src/nativestore/RelationBlock.cpp

+        delete current;
+        current = temp;
+    }
+    delete current;


Memory leak: double delete of current pointer. At line 877, 'current' is deleted after the loop, but 'current' is already either null (normal loop exit) or points to already-deleted memory (from line 874). This will cause undefined behavior. Remove line 877.

Copilot · 2025-12-31T05:42:43Z

src/nativestore/temporal/TemporalEventLogger.h

@@ -0,0 +1,58 @@
+/**
+Copyright 2024 JasmineGraph Team


Copyright year is 2024, but other files in this PR (TemporalQueryFilter.h, TemporalConstants.h) use 2025. For consistency, this should also be 2025.

Suggested change

Copyright 2024 JasmineGraph Team

Copyright 2025 JasmineGraph Team

Copilot · 2025-12-31T05:42:43Z

CMakeLists.txt

        src/nativestore/MetaPropertyLink.cpp
        src/nativestore/MetaPropertyEdgeLink.cpp


Duplicate source file entries in CMakeLists.txt. Lines 207-208 list MetaPropertyLink.cpp and MetaPropertyEdgeLink.cpp, and they are duplicated again at lines 211-212. This duplication should be removed to clean up the build configuration.

Suggested change

src/nativestore/MetaPropertyLink.cpp

src/nativestore/MetaPropertyEdgeLink.cpp

Copilot · 2025-12-31T05:42:44Z

src/util/kafka/StreamHandler.cpp

+    if (edgeJson.contains("operationTimestamp") && edgeJson["operationTimestamp"].is_string()) {
+        return edgeJson["operationTimestamp"].get<std::string>();
+    }
+    if (edgeJson.contains("timestamp") && edgeJson["timestamp"].is_string()) {
+        return edgeJson["timestamp"].get<std::string>();


The resolveOperationTimestamp function only accepts string timestamps, but it should also accept numeric timestamps (as mentioned in the documentation and as implemented in JasmineGraphIncrementalLocalStore.cpp). This creates inconsistency in timestamp handling. The function should check for both is_string() and is_number() types and convert numeric values to strings.

Suggested change

if (edgeJson.contains("operationTimestamp") && edgeJson["operationTimestamp"].is_string()) {

return edgeJson["operationTimestamp"].get<std::string>();

}

if (edgeJson.contains("timestamp") && edgeJson["timestamp"].is_string()) {

return edgeJson["timestamp"].get<std::string>();

if (edgeJson.contains("operationTimestamp")) {

const auto &opTs = edgeJson["operationTimestamp"];

if (opTs.is_string()) {

return opTs.get<std::string>();

}

if (opTs.is_number_integer() || opTs.is_number_unsigned()) {

return std::to_string(opTs.get<long long>());

}

if (opTs.is_number_float()) {

return std::to_string(opTs.get<double>());

}

}

if (edgeJson.contains("timestamp")) {

const auto &ts = edgeJson["timestamp"];

if (ts.is_string()) {

return ts.get<std::string>();

}

if (ts.is_number_integer() || ts.is_number_unsigned()) {

return std::to_string(ts.get<long long>());

}

if (ts.is_number_float()) {

return std::to_string(ts.get<double>());

}

Copilot · 2025-12-31T05:42:44Z

src/query/processor/cypher/runtime/OperatorExecutor.cpp

+<<<<<<< HEAD
+    TemporalConstraints temporalConstraints = TemporalQueryFilter::fromJson(query);
+=======
    OpenTelemetryUtil::addSpanAttribute("relationship.type", query["relType"].get<std::string>());

+>>>>>>> master


Merge conflict markers present in UndirectedRelationshipTypeScan function. The conflict shows both HEAD (temporal constraints) and master (telemetry attribute) code. Both pieces of code should likely be kept, but the merge conflict markers need to be resolved.

Suggested change

<<<<<<< HEAD

TemporalConstraints temporalConstraints = TemporalQueryFilter::fromJson(query);

=======

OpenTelemetryUtil::addSpanAttribute("relationship.type", query["relType"].get<std::string>());

>>>>>>> master

TemporalConstraints temporalConstraints = TemporalQueryFilter::fromJson(query);

OpenTelemetryUtil::addSpanAttribute("relationship.type", query["relType"].get<std::string>());

Copilot · 2025-12-31T05:42:44Z

src/nativestore/NodeBlock.cpp

+        delete current;
+        current = temp;
+    }
+    delete current;


Memory leak: double delete of current pointer. At line 440, 'current' is deleted after the loop, but 'current' is already either null (normal loop exit) or points to already-deleted memory (from line 437). This will cause undefined behavior. Remove line 440.

Copilot · 2025-12-31T05:42:45Z

src/nativestore/temporal/TemporalEventLogger.cpp

@@ -0,0 +1,244 @@
+/**
+Copyright 2024 JasmineGraph Team


Copyright year is 2024, but other files in this PR (TemporalQueryFilter.cpp, TemporalConstants.h) use 2025. For consistency, this should also be 2025.

Suggested change

Copyright 2024 JasmineGraph Team

Copyright 2025 JasmineGraph Team

Copilot · 2025-12-31T05:42:45Z

src/nativestore/temporal/TemporalQueryFilter.cpp

+    // Try direct conversion first (for epoch milliseconds)
+    try {
+        long long value = std::stoll(timestamp);
+        // If value is reasonable epoch millis (between 2000 and 2100), use it
+        if (value > 946684800000LL && value < 4102444800000LL) {
+            return value;
+        }
+    } catch (...) {
+        // Fall through to cleaned version
+    }
+


Potential bug in timestamp normalization: if the direct conversion at line 177 succeeds but the value is outside the reasonable range (not between 2000-2100), the function falls through to clean and parse again. However, if the original timestamp was a valid numeric string outside this range (e.g., "100" for a relative timestamp), it will be parsed twice with potentially different results. Consider either returning kNegativeInfinity for out-of-range values or logging a warning about unexpected timestamp ranges.

Suggested change

// Try direct conversion first (for epoch milliseconds)

try {

long long value = std::stoll(timestamp);

// If value is reasonable epoch millis (between 2000 and 2100), use it

if (value > 946684800000LL && value < 4102444800000LL) {

return value;

}

} catch (...) {

// Fall through to cleaned version

}

// Try direct conversion first (for epoch milliseconds or numeric relative timestamps)

try {

long long value = std::stoll(timestamp);

// If value is reasonable epoch millis (between 2000 and 2100), use it directly

if (value > 946684800000LL && value < 4102444800000LL) {

return value;

}

// For other numeric values (e.g., relative timestamps), return the parsed value as-is

return value;

} catch (...) {

// Fall through to cleaned version for legacy/non-numeric formats

}

Copilot · 2025-12-31T05:42:45Z

src/localstore/incremental/JasmineGraphIncrementalLocalStore.h

    static std::pair<std::string, unsigned int> getIDs(std::string edgeString);
    JasmineGraphIncrementalLocalStore(unsigned int graphID = 0, unsigned int partitionID = 0,
                                      std::string openMode = "trunk", bool embedNode = false);
+    ~JasmineGraphIncrementalLocalStore() { delete nm; }


Incomplete destructor: the destructor only deletes 'nm' but doesn't clean up other dynamically allocated resources. The 'embedding_requests' vector pointer is allocated in the constructor but never freed. If 'embedNode' is true, 'faissStore' and 'textEmbedder' are also allocated but not freed. This causes memory leaks. The destructor should delete all dynamically allocated resources.

Nayanatharapmc added 8 commits December 14, 2025 07:03

Add operation type and timestamp handling in StreamHandler

5f11a73

Add temporal graph storage support for time-evolving graphs

200fb1e

Enhance node and edge operation handling with support for UPDATE and …

3628e74

…DELETE operations, and improve timestamp resolution for consistency

Add index management and log compaction features to TemporalEventLogger

cd5dbae

Add core temporal metadata keys and timestamp utility functions

b4f1f92

Add timestamp parsing utility and enhance range evaluation in Tempora…

5098194

…lQueryFilter

Add comprehensive documentation for temporal graph data flow and proc…

b7ec711

…essing

Merge master into temporal-native-store: combine temporal, embedding,…

2a757e1

… and telemetry features

Copilot AI review requested due to automatic review settings December 31, 2025 05:35

Nayanatharapmc closed this Dec 31, 2025

Copilot started reviewing on behalf of Nayanatharapmc December 31, 2025 05:35 View session

Copilot AI reviewed Dec 31, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Introduce Temporal Graph Storage support to NativeStore #349

Introduce Temporal Graph Storage support to NativeStore #349

Uh oh!

Nayanatharapmc commented Dec 31, 2025

Uh oh!

sonarqubecloud bot commented Dec 31, 2025

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Dec 31, 2025

Uh oh!

Copilot AI Dec 31, 2025

Uh oh!

Copilot AI Dec 31, 2025

Uh oh!

Copilot AI Dec 31, 2025

Uh oh!

Copilot AI Dec 31, 2025

Uh oh!

Copilot AI Dec 31, 2025

Uh oh!

Copilot AI Dec 31, 2025

Uh oh!

Copilot AI Dec 31, 2025

Uh oh!

Copilot AI Dec 31, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

	Copyright 2024 JasmineGraph Team
	Copyright 2025 JasmineGraph Team

		src/nativestore/MetaPropertyLink.cpp
		src/nativestore/MetaPropertyEdgeLink.cpp

-    if (edgeJson.contains("operationTimestamp") && edgeJson["operationTimestamp"].is_string()) {
-        return edgeJson["operationTimestamp"].get<std::string>();
-    }
-    if (edgeJson.contains("timestamp") && edgeJson["timestamp"].is_string()) {
-        return edgeJson["timestamp"].get<std::string>();
+    if (edgeJson.contains("operationTimestamp")) {
+        const auto &opTs = edgeJson["operationTimestamp"];
+        if (opTs.is_string()) {
+            return opTs.get<std::string>();
+        }
+        if (opTs.is_number_integer() || opTs.is_number_unsigned()) {
+            return std::to_string(opTs.get<long long>());
+        }
+        if (opTs.is_number_float()) {
+            return std::to_string(opTs.get<double>());
+        }
+    }
+    if (edgeJson.contains("timestamp")) {
+        const auto &ts = edgeJson["timestamp"];
+        if (ts.is_string()) {
+            return ts.get<std::string>();
+        }
+        if (ts.is_number_integer() || ts.is_number_unsigned()) {
+            return std::to_string(ts.get<long long>());
+        }
+        if (ts.is_number_float()) {
+            return std::to_string(ts.get<double>());
+        }

Introduce Temporal Graph Storage support to NativeStore #349

Introduce Temporal Graph Storage support to NativeStore #349

Uh oh!

Conversation

Nayanatharapmc commented Dec 31, 2025

Uh oh!

sonarqubecloud bot commented Dec 31, 2025

Quality Gate failed

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI Dec 31, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Dec 31, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Dec 31, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Dec 31, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Dec 31, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Dec 31, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Dec 31, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Dec 31, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Dec 31, 2025

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant