Fix: Ensure lil-rag documents are deleted when notes are removed

[Copilot]

Problem

When deleting notes using the ml-notes delete command, the system was only removing notes from the local database but not cleaning up the corresponding indexed documents in the lil-rag vector search service. This resulted in:

• Orphaned documents in the lil-rag index
• Inconsistent search results showing references to deleted notes
• Wasted storage space in the vector index
• Potential confusion when search results include non-existent notes

Solution

This PR implements comprehensive document deletion for lil-rag integration:

New API Methods

lilrag.Client:

• Added DeleteDocument(id string) and DeleteDocumentWithNamespace(id, namespace string) methods
• Added DeleteRequest and DeleteResponse structs for proper API communication
• Follows the same pattern as existing index and search methods

search.LilRagSearch:

• Added DeleteNote(noteID int) and DeleteNoteWithNamespace(noteID int, namespace, projectID string) methods
• Uses identical document ID format as indexing: notes-{projectID}-{noteID}
• Maintains namespace consistency with ml-notes prefix

Updated Delete Commands

Both single note deletion and bulk deletion (--all) now:

1. Delete the note from the database (existing behavior)
2. Remove the corresponding document from lil-rag index (new)
3. Handle errors gracefully - database deletion succeeds even if lil-rag fails

Example Usage

# Delete single note - removes from both database and lil-rag
ml-notes delete 123

# Delete multiple notes - cleans up all corresponding lil-rag documents  
ml-notes delete 45 67 89

# Delete all notes - removes entire project's documents from lil-rag
ml-notes delete --all

Key Features

• Consistency: Ensures database and vector index stay synchronized
• Resilience: System continues working even when lil-rag service is unavailable
• Project isolation: Respects namespace boundaries for multi-project setups
• Debug visibility: Comprehensive logging for troubleshooting
• Backward compatibility: No breaking changes to existing functionality

Testing

• Added unit tests to verify document ID consistency between indexing and deletion
• Manual testing confirms proper cleanup in both single and bulk deletion scenarios
• Verified graceful degradation when lil-rag service is unavailable
• All existing tests continue to pass

This change ensures the ml-notes system maintains data integrity across all storage layers, preventing the accumulation of orphaned search index entries.

---

✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.

[Copilot]

Pull Request Overview

This PR ensures data consistency by implementing comprehensive document deletion for the lil-rag vector search service when notes are removed. Previously, deleting notes only removed them from the local database, leaving orphaned documents in the vector index.

• Added new deletion methods to both the lil-rag client and search interface to remove indexed documents
• Enhanced delete commands to clean up both database and vector index entries
• Added unit tests to verify document ID consistency between indexing and deletion operations

Reviewed Changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.

File	Description
internal/lilrag/client.go	Added DeleteDocument methods and corresponding request/response structs for API communication
internal/search/lilrag_search.go	Implemented DeleteNote methods that generate consistent document IDs and handle namespace management
internal/search/lilrag_search_test.go	Added unit tests to verify document ID format consistency between indexing and deletion
cmd/delete.go	Enhanced both single and bulk deletion commands to remove documents from lil-rag vector index

---

_{Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.}

[Copilot]

This helper function duplicates the document ID generation logic from the production code. Consider extracting the actual document ID generation into a shared utility function to avoid code duplication and ensure consistency.