Add cache initialization for existing files in DatafusionEngine

[cocosz]

Description

This PR adds eager cache initialization for existing files in the DatafusionEngine constructor. Previously, files were only added to the cache during refresh operations, which meant the first query after engine initialization would experience cache misses. This change improves query performance by pre-populating the cache with existing files at initialization time.

Benefits

• Improved First Query Performance: Eliminates cache misses on the first query after engine initialization
• 100% Cache Hit Rate: Achieves optimal cache hit rate from the first query onwards
• Better Resource Utilization: Ensures cache is populated and ready before queries are executed

Testing

• Added testExistingFilesAddedToCacheOnInitialization() test that verifies:

  • Cache memory increases after engine initialization
  • All existing files are present in the cache
  • Test passes successfully with proper file setup

• Tested with Clickbench dataset showing:

  • Existing files loaded at initialization
  • Queries achieves 100% hit rate

[coderabbitai]

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

✨ Finishing touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Post copyable unit tests in a comment

Tip

Issue Planner is now in beta. Read the docs and try it out! Share your feedback on Discord.

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[github-actions]

❌ Gradle check result for c9018ca: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?