-
Notifications
You must be signed in to change notification settings - Fork 5.5k
feat: Allow view querying with Mongo connector #26995
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
Conversation
Reviewer's guide (collapsed on small PRs)Reviewer's GuideAdds view-awareness to the Mongo connector by short-circuiting index lookup for views and introducing a helper to detect MongoDB views via listCollections, ensuring disallowed operations are not used on views. Sequence diagram for MongoSession.getIndexes handling viewssequenceDiagram
actor PrestoPlanner
participant MongoSession
participant MongoClient
participant MongoDatabase
participant MongoCollection
PrestoPlanner->>MongoSession: getIndexes(SchemaTableName)
MongoSession->>MongoSession: isView(schema, table)
MongoSession->>MongoClient: getDatabase(schema)
MongoClient-->>MongoSession: MongoDatabase
MongoSession->>MongoDatabase: runCommand(listCollections filter name, type=view)
MongoDatabase-->>MongoSession: result Document
MongoSession->>MongoSession: parse cursor.firstBatch
alt is view
MongoSession-->>PrestoPlanner: return empty MongoIndex list
else is not view
MongoSession->>MongoSession: getCollection(SchemaTableName)
MongoSession->>MongoCollection: listIndexes()
MongoCollection-->>MongoSession: indexDocuments
MongoSession->>MongoSession: MongoIndex.parse(indexDocuments)
MongoSession-->>PrestoPlanner: List<MongoIndex>
end
Updated class diagram for MongoSession view-aware index lookupclassDiagram
class MongoSession {
+List~MongoIndex~ getIndexes(SchemaTableName tableName)
+boolean isView(String schema, String table)
-MongoClient client
-LoadingCache~SchemaTableName, MongoTableHandle~ tableCache
+MongoCollection~Document~ getCollection(SchemaTableName tableName)
+MongoCollection~Document~ getCollection(String schema, String table)
+void dropColumn(MongoTableHandle table, String columnName)
}
class MongoClient {
+MongoDatabase getDatabase(String schema)
}
class MongoDatabase {
+Document runCommand(Document command)
}
class MongoCollection {
+ListIndexesIterable~Document~ listIndexes()
}
class SchemaTableName {
+String getSchemaName()
+String getTableName()
}
class MongoIndex {
+List~MongoIndex~ parse(ListIndexesIterable~Document~ indexes)
}
class Document {
}
class LoadingCache~SchemaTableName, MongoTableHandle~ {
+void invalidate(SchemaTableName key)
}
class MongoTableHandle {
+SchemaTableName getSchemaTableName()
}
MongoSession --> MongoClient : uses
MongoSession --> MongoDatabase : via client.getDatabase
MongoSession --> MongoCollection : uses
MongoSession --> SchemaTableName : parameter
MongoSession --> MongoIndex : returns
MongoSession --> Document : constructs commands
MongoSession --> LoadingCache~SchemaTableName, MongoTableHandle~ : uses
MongoSession --> MongoTableHandle : parameter
MongoClient --> MongoDatabase : creates
MongoDatabase --> Document : returns
MongoCollection --> Document : returns
MongoIndex --> Document : parses from
File-Level Changes
Tips and commandsInteracting with Sourcery
Customizing Your ExperienceAccess your dashboard to:
Getting Help
|
8bf28f1 to
f213c5e
Compare
f213c5e to
4eb188a
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hey - I've left some high level feedback:
- The new isView check adds a round-trip listCollections command on every getIndexes call; consider caching view metadata or reusing existing table metadata to avoid repeated database commands on hot paths.
- isView currently assumes the listCollections command succeeds and that the result always contains cursor/firstBatch; it would be safer to handle missing fields or command failures explicitly (e.g., permissions issues) to avoid unexpected runtime errors.
- isView is declared public but appears to be an internal helper; if it’s not intended as part of the MongoSession API, consider making it private or package-private and/or taking SchemaTableName directly instead of separate schema/table strings for consistency with the rest of the class.
Prompt for AI Agents
Please address the comments from this code review:
## Overall Comments
- The new isView check adds a round-trip listCollections command on every getIndexes call; consider caching view metadata or reusing existing table metadata to avoid repeated database commands on hot paths.
- isView currently assumes the listCollections command succeeds and that the result always contains cursor/firstBatch; it would be safer to handle missing fields or command failures explicitly (e.g., permissions issues) to avoid unexpected runtime errors.
- isView is declared public but appears to be an internal helper; if it’s not intended as part of the MongoSession API, consider making it private or package-private and/or taking SchemaTableName directly instead of separate schema/table strings for consistency with the rest of the class.Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.
agrawalreetika
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for the changes. Letf few comments.
Also, can we add test?
| tableCache.invalidate(table.getSchemaTableName()); | ||
| } | ||
|
|
||
| public boolean isView(String schema, String table) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Lets make it private
| .get("cursor", Document.class) | ||
| .getList("firstBatch", Document.class); | ||
|
|
||
| return !collections.isEmpty(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Lets use avaiable Driver APIs instead of running raw commands - https://mongodb.github.io/mongo-java-driver/4.11/apidocs/mongodb-driver-sync/com/mongodb/client/MongoDatabase.html#listCollections()
Description
Since Mongo treats views as regular documents, the find function that's used in the connector works fine. The change here is to prevent mongo from using functions that are not allowed on views when querying a view. In this instance, get indexes
Motivation and Context
Impact
Test Plan
Contributor checklist
Release Notes
Please follow release notes guidelines and fill in the release notes below.
Summary by Sourcery
Allow MongoDB connector to handle views without invoking unsupported index operations.
New Features:
Enhancements: