CSHARP-5763: Auto-embedding for vector search #1842

ajcvickers · 2026-01-06T14:04:38Z

Note that I have not been able to test this against a server that fully supports auto-embedding. The version I have access to throws for a variety of cases, has no available embedding models, and index creation never completes.

Copilot

Pull request overview

This PR adds support for auto-embedding in vector search indexes, allowing MongoDB Atlas to automatically create vector embeddings from text fields using specified embedding models (e.g., "voyage-4"). The implementation includes new API constructors, query vector handling for text input, and comprehensive test coverage.

Key changes:

New VectorEmbeddingModality enum to specify the type of data being embedded
QueryVector now supports text strings for auto-embedding indexes
CreateVectorSearchIndexModel has new constructors for auto-embedding indexes with support for compression profiles and explicit compression settings

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 10 comments.

Show a summary per file

File	Description
VectorEmbeddingModality.cs	New enum defining modality types for auto-embedding (currently only Text)
QueryVector.cs	Added string constructor and implicit operator for text-based queries
PipelineStageDefinitionBuilder.cs	Modified to use "query" field for text vs "queryVector" for arrays
CreateVectorSearchIndexModel.cs	Added auto-embedding constructors, compression profile support, and updated rendering logic
VectorSearchTests.cs	Added test for auto-embedding vector search with new Movie class
AtlasSearchIndexManagmentTests.cs	Added comprehensive tests for auto-embedding index creation with various options and renamed existing tests for consistency

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

tests/MongoDB.Driver.Tests/Search/AtlasSearchIndexManagmentTests.cs

src/MongoDB.Driver/CreateVectorSearchIndexModel.cs

tests/MongoDB.Driver.Tests/Search/AtlasSearchIndexManagmentTests.cs

src/MongoDB.Driver/CreateVectorSearchIndexModel.cs

tests/MongoDB.Driver.Tests/Search/VectorSearchTests.cs

src/MongoDB.Driver/QueryVector.cs

src/MongoDB.Driver/CreateVectorSearchIndexModel.cs

src/MongoDB.Driver/QueryVector.cs

src/MongoDB.Driver/PipelineStageDefinitionBuilder.cs

src/MongoDB.Driver/CreateVectorSearchIndexModel.cs

src/MongoDB.Driver/PipelineStageDefinitionBuilder.cs

src/MongoDB.Driver/QueryVector.cs

…elease. - Changing QueryVector to take string instead of BsonString - Added model to vector query options - Fixed lookup for index status on community server - Added more query tests - Changed query tests to run on small set of documents that can be queried in a reasonable amount of time.

BorisDog

Few minor comments

tests/MongoDB.Driver.Tests/Search/AtlasSearchIndexManagmentTests.cs

src/MongoDB.Driver/CreateVectorSearchIndexModelBase.cs

src/MongoDB.Driver/CreateVectorSearchIndexModel.cs

BorisDog

There are some failing unittests.

BorisDog · 2026-01-09T21:19:51Z

src/MongoDB.Driver/CreateAutoEmbeddingVectorSearchIndexModel.cs

+    }
+
+    /// <inheritdoc/>
+    internal override BsonDocument Render(RenderArgs<TDocument> renderArgs)


Suggestion for CreateSearchIndexModel:
Should we introduce
BsonDocument Render(RenderArgs<TDocument> renderArgs) => Definition
for more uniform handling in CreateCreateIndexesOperation?

Also now that Render is internal, the exception in CreateSearchIndexModel.Definition is not accurate. Is it possible to override that property and throw there? I think making a property is acceptable as breaking change?

Can you refer back to the comments that Robert had on the previous PR, and see if you are okay with the breaking changes that were rejected there?
#1769
#1795

. Is it possible to override that property and throw there?

From: #1795 (review)

If making this property virtual is a breaking change, we could just let it return null in the base class.

Likewise, anything that breaks what the property currently does--like throwing and expecting people (almost certainly just us) to call Render instead for current cases that work with the old type.

My preference historically would be to take small binary breaks to things like this that are primarily called by the driver, in order to keep internal quality high, and to create a coherent design and experience. But I was under the impression that we didn't want to take binary breaks because of the impact of third-party libraries that have not been rebuilt against latest.

property virtual is a breaking change
I think this is something we can tolerate, I don't see how this can be breaking in practice.

Also my other suggestion was having Render method in CreateSearchIndexModel
BsonDocument Render(RenderArgs<TDocument> renderArgs) => Definition, for consistency.

Both suggestions for your consideration.

BorisDog · 2026-01-09T21:25:00Z

tests/MongoDB.Driver.Tests/Search/AutoEmbedVectorSearchTests.cs

+        ]);
+
+        _collection.SearchIndexes.CreateOne(new CreateAutoEmbeddingVectorSearchIndexModel<Movie>(
+            e => e.Plot, _autoEmbedIndexName, "voyage-4", filterFields: [e => e.Runtime, e => e.Year]));


Should we introduce a temporary expectation for a failure here?

Do you mean use an ifdef to disable the tests when auto-embedding is not available? Or something else?

Assuming the auto-embedding is not available yet and the tests fail:
I thought to implement your strategy of checking for the test failure instead of disabling the test.

That strategy doesn't work well when the tests are actually passing against a compatible server. Because it means the tests checked in no longer test that the working server works, they just test that the failing server fails. I think if we're going to implement and release without having a C.I. server to test against, then the C.I. should just fail, and keep failing until such a server is available. This reflects the reality of the testing situation in which we are releasing, and means if we don't want the C.I. to keep failing, then we must do something to properly fix it, rather than excluding more tests.

I'd love to see it fixed properly and have the CI green. But if the time doesn't allow we can't introduce constantly failing tests in this PR, so I'd prefer the disabling option.
In any case this discrepancy in server behavior is temporary.

BorisDog

Code LGTM.
Need to finalize the tests strategy.

BorisDog · 2026-01-12T20:24:16Z

tests/MongoDB.Driver.Tests/Search/AutoEmbedVectorSearchTests.cs

+        ]);
+
+        _collection.SearchIndexes.CreateOne(new CreateAutoEmbeddingVectorSearchIndexModel<Movie>(
+            e => e.Plot, _autoEmbedIndexName, "voyage-4", filterFields: [e => e.Runtime, e => e.Year]));


Assuming the auto-embedding is not available yet and the tests fail:
I thought to implement your strategy of checking for the test failure instead of disabling the test.

BorisDog

Please notice that there are also failing unittests.

BorisDog · 2026-01-14T02:31:28Z

src/MongoDB.Driver/CreateAutoEmbeddingVectorSearchIndexModel.cs

+    }
+
+    /// <inheritdoc/>
+    internal override BsonDocument Render(RenderArgs<TDocument> renderArgs)


property virtual is a breaking change
I think this is something we can tolerate, I don't see how this can be breaking in practice.

Also my other suggestion was having Render method in CreateSearchIndexModel
BsonDocument Render(RenderArgs<TDocument> renderArgs) => Definition, for consistency.

Both suggestions for your consideration.

BorisDog · 2026-01-14T02:35:44Z

tests/MongoDB.Driver.Tests/Search/AutoEmbedVectorSearchTests.cs

+        ]);
+
+        _collection.SearchIndexes.CreateOne(new CreateAutoEmbeddingVectorSearchIndexModel<Movie>(
+            e => e.Plot, _autoEmbedIndexName, "voyage-4", filterFields: [e => e.Runtime, e => e.Year]));


I'd love to see it fixed properly and have the CI green. But if the time doesn't allow we can't introduce constantly failing tests in this PR, so I'd prefer the disabling option.
In any case this discrepancy in server behavior is temporary.

CSHARP-5763: Auto-embedding for vector search

07b4a96

Note that I have not been able to test this against a server that fully supports auto-embedding. The version I have access to throws for a variety of cases, has no available embedding models, and index creation never completes.

ajcvickers requested review from BorisDog and Copilot January 6, 2026 14:04

ajcvickers requested a review from a team as a code owner January 6, 2026 14:04

Copilot started reviewing on behalf of ajcvickers January 6, 2026 14:05 View session

ajcvickers added the feature Adds new user-facing functionality. label Jan 6, 2026

Copilot AI reviewed Jan 6, 2026

View reviewed changes

Fixes for Copilot feedback.

653a923

BorisDog requested a review from adelinowona January 6, 2026 16:58

BorisDog requested changes Jan 7, 2026

View reviewed changes

Updates based on community server testing.

92a1f02

BorisDog requested changes Jan 7, 2026

View reviewed changes

ajcvickers added 3 commits January 8, 2026 15:14

- Cleanup and testing of negative cases

e2e42d8

Create subclasses for different kinds of vector index.

baf4731

ajcvickers requested a review from BorisDog January 8, 2026 16:21

BorisDog requested changes Jan 8, 2026

View reviewed changes

tests/MongoDB.Driver.Tests/Search/AtlasSearchIndexManagmentTests.cs Outdated Show resolved Hide resolved

src/MongoDB.Driver/CreateVectorSearchIndexModelBase.cs Outdated Show resolved Hide resolved

src/MongoDB.Driver/CreateVectorSearchIndexModel.cs Outdated Show resolved Hide resolved

Review updates.

343199a

ajcvickers requested a review from BorisDog January 9, 2026 09:47

Updated ordering in test baseline.

df5e499

BorisDog requested changes Jan 9, 2026

View reviewed changes

BorisDog reviewed Jan 12, 2026

View reviewed changes

BorisDog requested changes Jan 14, 2026

View reviewed changes

CSHARP-5763: Auto-embedding for vector search #1842

Are you sure you want to change the base?

CSHARP-5763: Auto-embedding for vector search #1842

Uh oh!

Conversation

ajcvickers commented Jan 6, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

BorisDog left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

BorisDog left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ajcvickers Jan 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ajcvickers Jan 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

BorisDog left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

BorisDog left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

ajcvickers Jan 13, 2026 •

edited

Loading

ajcvickers Jan 13, 2026 •

edited

Loading