Rag

[wosledon]

No description provided.

[Copilot]

Pull request overview

This pull request introduces RAG (Retrieval Augmented Generation) capabilities to the Berry framework. It adds a complete vector storage and semantic search infrastructure with a MiniLM-L6-v2 embedding model implementation, enabling knowledge base retrieval and question-answering functionality.

Key Changes:

• New vector storage module with in-memory implementation and embedding abstractions
• RAG module providing document ingestion, semantic search, and conversation memory
• MiniLM-L6-v2 embedding provider with ONNX runtime and tokenizer bridge
• Complete RAG demo application showcasing the new capabilities

Reviewed changes

Copilot reviewed 57 out of 77 changed files in this pull request and generated no comments.

Show a summary per file

File	Description
src/Berry.Modules.VectorStore/*	Core vector storage module with in-memory implementation and interfaces
src/Berry.Modules.Rag/*	RAG service providing query, ingestion, and hybrid retrieval with lexical boosting
src/Berry.Abstractions.Embeddings/*	Embedding abstractions for provider, tokenizer, and model resolution
src/Berry.Embeddings.MiniLmL6v2/*	MiniLM-L6-v2 implementation with ONNX runtime and tokenizer support
samples/Berry.RagDemo/*	Demo application with REST API endpoints for document upload and semantic search
demos/RAGDemo/*	Standalone CLI demo for local RAG experimentation
docs/develop/rag.md	Documentation for RAG module usage and extension
docs/develop/embeddings.md	Documentation for embeddings architecture and customization
Directory.Build.props	Version bump to 1.0.3
Berry.slnx	Solution file updated with new projects

Comments suppressed due to low confidence (2)

src/Berry.Embeddings.MiniLmL6v2/TokenizerBridge.cs:1

• The comment references 'tokenizer.json' priority, but the PackageReference 'Tokenizers.DotNet' in the csproj may not be available or stable. Verify that this package exists and is compatible with net8.0, or provide fallback documentation if it's optional.
  samples/Berry.RagDemo/Controllers/RagController.cs:1
• [nitpick] Complex logic compressed into a single line reduces readability. Consider breaking this into multiple lines with proper formatting for better maintainability.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.