chore(deps): update getmeili/meilisearch docker tag to v1.38.2#76
Open
renovate[bot] wants to merge 1 commit intomasterfrom
Open
chore(deps): update getmeili/meilisearch docker tag to v1.38.2#76renovate[bot] wants to merge 1 commit intomasterfrom
renovate[bot] wants to merge 1 commit intomasterfrom
Conversation
10e3884 to
4f133e2
Compare
c645ae2 to
4fef743
Compare
4fef743 to
bdfd6d8
Compare
bdfd6d8 to
b7eeaa3
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR contains the following updates:
v1.11.1→v1.38.2Release Notes
meilisearch/meilisearch (getmeili/meilisearch)
v1.38.2: 🐍Compare Source
Meilisearch v1.38.2 fixes a regression introduced in Meilisearch v1.38.0 where Meilisearch could sometimes stop processing tasks after an automatic task queue cleanup performed while more tasks where being added.
We recommend that all users in v1.38.0 and v1.38.1 migrate to v1.38.2.
Note that the dumpless upgrade process can take up to one minute before the HTTP server becomes available when migrating from a previous v1.38 version.
Full Changelog: meilisearch/meilisearch@v1.38.1...v1.38.2
v1.38.1: 🐍Compare Source
Meilisearch v1.38.1 fixes a regression we have seen in task processing, which caused the engine to stop processing tasks.
🪲 Bug fixes
Full Changelog: meilisearch/meilisearch@v1.38.0...v1.38.1
v1.38.0: 🐍Compare Source
Meilisearch v1.38 drastically improves indexing performance for embeddings, cleans up the task queue of orphan tasks and batches, ensures they are correctly deleted now, fixes issues users may encounter with OpenAI or VoyageAI when generating embeddings, and improves the tooling for generating routes in the documentation.
✨ Enhancement
We drastically reduced embedding indexing time by eliminating full database scans. It is more efficient to add a couple of embeddings to a large database now.
🪲 Bug fixes
connection reset by peererrors when embedding documents or search queries using a remote embedder. By @dureuill in #6191🔩 Miscellaneous
The reference API is now generated from the source code, but it used to miss some actix-declared routes. It is now mandatory to attach a route when using the newly-provided
routes::routesandroutes::pathmacros, which ensures the route is added to the reference. Please refer to the documentation of the individual macros and the many examples throughout the codebase for more details.New Contributors
v1.37.0Compare Source
Meilisearch v1.37 introduces replicated sharding, removes the
vectorStoreSettingexperimental feature, stabilizes our new vector store for best performance, adds a security fix and miscellaneous improvements.✨ Improvements
§ Replicated sharding
§ Breaking changes
networkobjects sent to thePATCH /networkroute must now contain at least oneshardobject containing at least one remote whenleaderis notnull.Existing databases will be migrated automatically when upgraded with
--experimental-dumpless-upgradewhenleaderis notnull, such that for each remote:remoteslist: the remote with the same name as the shard.This change will not cause any document to be resharded.
To be able to upgrade without resharding, the migration uses the same name for remotes and for shards. However, in new configurations, we recommend using different names for shards and remotes.
Example of migration
For instance, the following network object:
{ "leader": "ms-00", "self": "ms-01", "remotes": { "ms-00": { /* .. */ }, "ms-01": { /* .. */ } } }is converted to:
{ "leader": "ms-00", "self": "ms-01", "remotes": { "ms-00": { /* .. */ }, "ms-01": { /* .. */ } }, "shards": { // ✨ NEW "ms-00": { // shard named like the remote "remotes": ["ms-00"] // is owned by the remote }, "ms-01": { "remotes": ["ms-01"] } } }Addition of
network.shardsThe
networkobject for routesPATCH /networkandGET /networknow contains the new fieldshards, which is an object whose values areshardobjects, and keys the name of each shard.Each
shardobject contains a single fieldremotes, which is an array of strings, each string representing the name of an existing remote.Convenience fields
The
shardobjects inPATCH /networkcontain the additional fieldsaddRemotesandremoveRemotesmeant for convenience:shard.addRemotesto add these remotes to the list of remotes of a shard.shard.removeRemotesto remove these remotes from the list of remotes of a shard.null,shard.remoteswill completely override the existing list of remotes for a shard.null, then the order of application isshard.remotes, thenshard.addRemotes, thenshard.removeShards.Adding a new shard with some remotes
Remotes
ms-0andms-1own the new shards-a.Fully overriding the list of remotes owning a shard
ms-2is now the sole owner ofs-a, replacingms-0andms-1.Adding a remote without overriding the list of remotes owning a shard
ms-0andms-2are now the owners ofs-a.Removing a remote without overriding the list of remotes owning a shard
ms-0is now the sole owner ofs-a.Entirely removing a shard from the list of shards
Set the shard to
null:Or set its
remoteslist to the empty list:network.shardsvalidityWhen
network.leaderis notnull, eachshardobject innetwork.shardsmust:remotesthat exist in the list ofremotes.Additionally,
network.shardsmust contain at least one shard.Failure to meet any of these conditions will cause the
PATCH /networkroute to respond with400 invalid_network_shards.Change in sharding logic
Documents are now sharded according to the list of shards declared in the network rather than the list of remotes. All remotes owning a shard will process the documents that belong to this shard, allowing for replication.
Example of replication
The following configuration defines 3 remotes
0,1and2, and 3 shardsA,B,C, such that each remote owns two shards, achieving replication (losing one remote does not lose any document).{ "leader": "0", "self": "0", "remotes": { "0": { /* .. */ }, "1": { /* .. */ }, "2": { /* .. */ } }, "shards": { "A": { "remotes": ["0", "1"] }, "B": { "remotes": ["1", "2"] }, "C": { "remotes": ["2", "0"] } } }useNetworktakesnetwork.shardsinto accountWhen
useNetwork: trueis passed to a search query, it is expanded to multiple queries such that each shard declared innetwork.shardsappears exactly once, associated with a remote that owns that shard.This ensures that there is no missing or duplicate documents in the results.
_shardfiltersWhen the
networkexperimental feature is enabled, then it becomes possible to filter documents depending on the shard they belong to.Given
s-aands-bthe names of two shards declared innetwork.shards, then:_shard = "s-a"in afilterparameter to the search or documents fetch will return the documents that belong tos-a._shard != "s-a"will return the documents that do not belong tos-a_shard IN ["s-a", "s-b"]will return the documents that belong tos-aor tos-b.You can use these new filters in manual remote federated search to create a partitioning over all shards in the network.
Update instructions
When updating your Meilisearch network using dumpless upgrade, please observe the following guidelines:
PATCH /networkroute until all remotes of the network are finished updatinguseNetwork: true, call them on un-updated remotes. Calling it on already updated remotes will cause un-updated remotes to fail the search as they don't know about the_shardfilters.By @dureuill in #6128
§ Remove
vectorStoreSettingexperimental featureThe new HNSW vector store (hannoy) has been stabilized and is now the only supported vector store in Meilisearch.
As a result, updating to v1.37.0 will migrate all remaining legacy vector store indexes (using arroy) to hannoy, and the
vectorStoreSettingexperimental feature is no longer available.By @Kerollmops in #6176
Improve indexing performance for embeddings
We removed a computationally expensive step from vector indexing.
On a DB with 20M documents, this removes 300s per indexing batch of 1100s.
By @Kerollmops in #6175
§ 🔒 Security
localStorageBy @Strift and @curquiza in #6186 and #6172
§ 🔩 Miscellaneous
Full Changelog: meilisearch/meilisearch@v1.36.0...v1.37.0
v1.36.0: 🐙Compare Source
Version v1.36.0 introduces an exciting update to the ranking rules to improve the engine's relevance. It's actually the first time we've made such a change since v1.0, and we're really happy about this improvement!
✨ Enhancement
Introduce the
attributeRankandwordPositioncriteria by @Kerollmops in #6154, #6155, and #6164We released two new ranking rules that Meilisearch had already been using internally for the
attributeone, which is basically both ranking rules applied one after the other:attributeRank: A document is considered better if the query words match in a higher searchable attribute. It ignores the position of the query words in this attribute.wordPosition: A document is considered better if the query words match closer to the beginning of an attribute. The attribute rank is ignored by this rule.We continue our policy of migrating everyone to use a homemade HNSW by introducing a new dumpless upgrade step that migrates index uses the old annoy vector store to the new Hannoy one. Changing the vector store backend affects the ranking score. This step can take a couple of minutes when the number of embeddings is high, and we recommend changing the vector store backend beforehand to gain more control if needed. To do so, you must enable the
vectorStoreSettingexperimental feature and set thevectorStoreroot setting toexperimental.🪲 Bug fixes
attributesToSearchOnon an empty index by @zen-zap in #5944🔒 Security
🔩 Miscellaneous
❤️ Thanks to @zen-zap for contributing to this release!
v1.35.1Compare Source
🦋 Bug fixes
Meilisearch v1.35.1 fixes a possible task database corruption issue that would occur when using the S3 streaming snapshot method.
Details
While snapshotting the task database, sending a task to Meilisearch could corrupt it. Index DBs were not affected, so manually rebuilding the task database fixes the corruption.
By @dureuill in #6160
Full Changelog: meilisearch/meilisearch@v1.35.0...v1.35.1
v1.35.0Compare Source
Meilisearch v1.35.0 contains improvements to the observability of the search performance, a breaking change to the recently introduced
POST /indexes/<index_uid>/fieldsroute, and other bugfixes and improvements.🌈 Improvements
Observe the search performance
The search routes accept a new field named
showPerformanceDetails. When set totrue, the search response contains a performance trace, allowing the user to betterunderstand what takes time during a search query.
impacted routes:
POST /indexes/<index_uid>/searchGET /indexes/<index_uid>/searchPOST /multi-searchPOST /indexes/<index_uid>/similarGET /indexes/<index_uid>/similarSearch
routes:
POST /indexes/<index_uid>/searchandGET /indexes/<index_uid>/searchRequest
new request parameters:
showPerformanceDetails:true/false(boolean)example
{ "q": "glass", "showPerformanceDetails": true }Response
new response field:
performanceDetails:{"<span>": "<human_duration>", .. }(map)example
{ "hits": [ // hits .. ], "query": "glass", "processingTimeMs": 5, "limit": 20, "offset": 0, "estimatedTotalHits": 1, "requestUid": "<uuid>", "performanceDetails": { "wait for permit": "295.29µs", "search > tokenize": "436.67µs", "search > resolve universe": "649.00µs", "search > keyword search": "515.71µs", "search > format": "288.54µs", "search": "3.56ms" } }Multi-search
route:
POST /multi-searchRequest
new request parameters:
queries.showPerformanceDetails:true/false(boolean)example
Response
new response field:
results.performanceDetails:{"<span>": "<human_duration>", .. }(map)example
{ "results": [ { "indexUid": "<index_uid>", "hits": [ // hits .. ], "query": "glass", "processingTimeMs": 5, "limit": 20, "offset": 0, "estimatedTotalHits": 1, "requestUid": "<uuid>", "performanceDetails": { "wait for permit": "295.29µs", "search > tokenize": "436.67µs", "search > resolve universe": "649.00µs", "search > keyword search": "515.71µs", "search > format": "288.54µs", "search": "3.56ms" } } ] }Federated Search
route:
POST /multi-searchRequest
new request parameters:
federation.showPerformanceDetails:true/false(boolean)example
{ "federation": { "showPerformanceDetails": true }, "queries": [ { "indexUid": "<index_uid>", "q": "glass" } ] }Response
new response field:
performanceDetails:{"<span>": "<human_duration>", .. }(map)example
{ "hits": [ // hits .. ], "query": "glass", "processingTimeMs": 5, "limit": 20, "offset": 0, "estimatedTotalHits": 1, "requestUid": "<uuid>", "performanceDetails": { "wait for permit": "213.83µs", "search > tokenize": "171.67µs", "search > resolve universe": "257.63µs", "search > keyword search": "577.71µs", "search > format": "114.96µs", "search > federation > wait for remote results": "62.71µs", "search > federation > merge results": "120.04µs", "search > federation > merge facets": "53.42µs", "search > federation": "237.04µs", "search": "289.08ms" } }Similar
routes:
POST /indexes/<index_uid>/similarandGET /indexes/<index_uid>/similarRequest
new request parameters:
showPerformanceDetails:true/false(boolean)example
{ "id": 143, "embedder": "manual", "showPerformanceDetails": true }Response
new response field:
performanceDetails:{"<span>": "<human_duration>", .. }(map)example
{ "hits": [ // hits .. ], "id": "143", "processingTimeMs": "[duration]", "limit": 20, "offset": 0, "estimatedTotalHits": 4, "performanceDetails": { "search > format": "244.92µs", "search": "1.25ms" } }By @ManyTheFish in #6132
Stabilize multithreaded post-processing of facets and prefixes
Multithreaded post-processing of facets and prefixes speeds-up indexing on multi-core machines.
Before v1.35.0, it could be disabled with an experimental feature.
Meilisearch v1.35.0 removes that experimental feature, resulting in the multithreaded post-processing being always enabled.
Cloud users upgrading to v1.35.0 will benefit from the speed-up.
By @ManyTheFish in #6130
🦋 Fixes
Breaking change: return pagination fields in
POST /indexes/<index_uid>/fieldsrouteExample: before/after
Before:
[ // fields .. ]After:
{ "results": [ // [fields .. ] ], "offset": 0, "limit": 20, "total": 0 }By @ManyTheFish in #6126
POST /indexes/<index_uid>/fields: Fix pattern filtering when a parent field was matching a child pattern:titlewas matching the patterntitle.to*.Example: before/after
For the following request:
{ "filter": { "attributePatterns": [ "title.en" ] } }Before Response,
titlefield is returned:{ "results": [ { "name": "title", //metadata... }, ], "offset": 0, "limit": 20, "total": 1 }After Response, nothing returned:
{ "results": [], "offset": 0, "limit": 20, "total": 0 }By @ManyTheFish, also in #6126
🐛 Other fixes
🔩 Miscellaneous
New Contributors
Full Changelog: meilisearch/meilisearch@v1.34.3...v1.35.0
v1.34.3Compare Source
🐛 Bug Fixes
Full Changelog: meilisearch/meilisearch@v1.34.2...v1.34.3
v1.34.2Compare Source
This patch fixes an accidental breaking change in v1.34.1 where Meilisearch would not start with a configuration file if
experimental_allowed_ip_networkswas not defined.Full Changelog: meilisearch/meilisearch@v1.34.1...v1.34.2
v1.34.1Compare Source
In Meilisearch v1.34.1, we released a security fix that affect web queries initiated by Meilisearch.
🔐 Security
In Meilisearch versions v1.8 to v1.34.0, a user with an API key with write permissions to the configuration of the Meilisearch instance could set up Meilisearch to send POST or GET requests to domains and IPs local to the private network of the Meilisearch instance, effectively bypassing the firewall. The vulnerability has not yet been assigned a CVE number; it has CVE Request
1975471for CVE ID.Meilisearch v1.34.1 fixes the vulnerability by forbidding Meilisearch from making any requests to a host resolving to a non-global IP, in the sense of the IANA IPv4 Special-Purpose Address Registry or the IANA IPv6 Special-Purpose Address Registry. If you need this functionality in your Meilisearch instance, you can set private IP networks as allowed with the
--experimental-allow-ip-networksparameter after reviewing the security implications.This is a breaking change, which is allowed for security reasons.
Allowing non-global IP networks
The
--experimental-allow-ip-networksCLI flag and theMEILI_EXPERIMENTAL_ALLOW_IP_NETWORKSenvironment variable control the behavior of Meilisearch with regards to non-global IP networks, with the CLI flag taking precedence over the environment variable when both are specified.192.168.0.0/16,10.0.0.0), then web requests to the networks from the list will be permitted. Web requests to the non-global IPs not part of the list will still be rejected.any, all web requests will be permitted regardless of the target IP, similar to the behavior of Meilisearch v1.34.0 and lower. Use this option when you control both the machine and the configuration of the deployed Meilisearch instance.Acknowledgments
Thanks to Gabriel Rodrigues (aka Texugo), for reporting this vulnerability and for helping us improve the security of Meilisearch.
v1.34.0Compare Source
🌈 Improvements
Easy search over your network of machines
useNetworkfield inPOST /indexes/{:indexUid}/searchThe search query object passed in the body of
POST /indexes/{:indexUid}/searchnow accepts an optional booleanuseNetwork. When present and set totrue, the search is executed "as-if" it was a remote federated search over all remotes in the network.That is, the following:
Search request
Is executed by Meilisearch as if it was the following, assuming a network of 3 Meilisearch instances with names
"0", "1" and "2":Equivalent multi-search request
Resulting in:
Search Response
{ "hits": [ { "id": 123025, "title": "Batman: The Dark Knight Returns, Part 1", "overview": "Batman has not been seen for ten years. A new breed of criminal ravages Gotham City, forcing 55-year-old Bruce Wayne back into the cape and cowl. But, does he still have what it takes to fight crime in a new era?", "genres": [ "Action", "Animation", "Mystery" ], "poster": "https://image.tmdb.org/t/p/w500/kkjTbwV1Xnj8wBL52PjOcXzTbnb.jpg", "release_date": 1345507200, "_federation": { "indexUid": "mieli", "queriesPosition": 2, "weightedRankingScore": 0.9894586894586894, "remote": "2" } }, { "id": 142061, "title": "Batman: The Dark Knight Returns, Part 2", "overview": "Batman has stopped the reign of terror that The Mutants had cast upon his city. Now an old foe wants a reunion and the government wants The Man of Steel to put a stop to Batman.", "genres": [ "Action", "Animation", "Mystery" ], "poster": "https://image.tmdb.org/t/p/w500/arEZYd6uMOFTILne9Ux0A8qctMe.jpg", "release_date": 1357171200, "_federation": { "indexUid": "mieli", "queriesPosition": 1, "weightedRankingScore": 0.9894558963186414, "remote": "1" } }, { "id": 16234, "title": "Batman Beyond: Return of the Joker", "overview": "The Joker is back with a vengeance, and Gotham's newest Dark Knight needs answers as he stands alone to face Gotham's most infamous Clown Prince of Crime.", "genres": [ "Animation", "Family", "Action", "Science Fiction" ], "poster": "https://image.tmdb.org/t/p/w500/7RlBs0An83fqAuKfwH5gKMcqgMc.jpg", "release_date": 976579200, "_federation": { "indexUid": "mieli", "queriesPosition": 1, "weightedRankingScore": 0.9427964918160996, "remote": "1" } }, { "id": 155, "title": "The Dark Knight", "overview": "Batman raises the stakes in his war on crime. With the help of Lt. Jim Gordon and District Attorney Harvey Dent, Batman sets out to dismantle the remaining criminal organizations that plague the streets. The partnership proves to be effective, but they soon find themselves prey to a reign of chaos unleashed by a rising criminal mastermind known to the terrified citizens of Gotham as the Joker.", "genres": [ "Drama", "Action", "Crime", "Thriller" ], "poster": "https://image.tmdb.org/t/p/w500/qJ2tW6WMUDux911r6m7haRef0WH.jpg", "release_date": 1216166400, "_federation": { "indexUid": "mieli", "queriesPosition": 1, "weightedRankingScore": 0.5784178187403994, "remote": "1" } }, { "id": 49026, "title": "The Dark Knight Rises", "overview": "Following the death of District Attorney Harvey Dent, Batman assumes responsibility for Dent's crimes to protect the late attorney's reputation and is subsequently hunted by the Gotham City Police Department. Eight years later, Batman encounters the mysterious Selina Kyle and the villainous Bane, a new terrorist leader who overwhelms Gotham's finest. The Dark Knight resurfaces to protect a city that has branded him an enemy.", "genres": [ "Action", "Crime", "Drama", "Thriller" ], "poster": "https://image.tmdb.org/t/p/w500/vzvKcPQ4o7TjWeGIn0aGC9FeVNu.jpg", "release_date": 1342396800, "_federation": { "indexUid": "mieli", "queriesPosition": 2, "weightedRankingScore": 0.5772657450076805, "remote": "2" } } ], "query": "Batman dark knight returns 1", "processingTimeMs": 173, "limit": 5, "offset": 0, "estimatedTotalHits": 47, "facetDistribution": { "genres": { "Action": 46, "Adventure": 15, "Animation": 34, "Comedy": 3, "Crime": 14, "Drama": 6, "Family": 15, "Fantasy": 8, "Horror": 1, "Mystery": 4, "Romance": 1, "Science Fiction": 14, "TV Movie": 4, "Thriller": 4, "Western": 1 } }, "facetStats": {}, "requestUid": "019bbcf4-a609-7701-8d82-d370611adfb3", "remoteErrors": {} }useNetworkrequires thenetworkexperimental feature to be enabled.useNetworkquery parameter inGET /indexes/{:indexUid}/searchPassing
useNetwork=trueas a query parameter toGET /indexes/{:indexUid}/searchhas the same effect as passinguseNetwork: trueas a field parameter toPOST /indexes/{:indexUid}/search.queries[].useNetworkfield inPOST /multi-searchuseNetworkcan also be passed as a field of the individual queries inside of amulti-searchrequest.POST /indexes/{:indexUid}/searchfor that queryFederated search example:
Multi-search request
{ "federation": { "limit": 5 }, "queries": [ { "q": "Batman returns", "indexUid": "mieli", "useNetwork": true }, { "q": "Superman returns", "indexUid": "mieli", "useNetwork": true } ] }Multi-search response
{ "hits": [ { "id": 364, "title": "Batman Returns", "overview": "While Batman deals with a deformed man calling himself the Penguin, an employee of a corrupt businessman transforms into the Catwoman.", "genres": [ "Action", "Fantasy" ], "poster": "https://image.tmdb.org/t/p/w500/jKBjeXM7iBBV9UkUcOXx3m7FSHY.jpg", "release_date": 708912000, "_federation": { "indexUid": "mieli", "queriesPosition": 0, "weightedRankingScore": 1.0, "remote": "1" } }, { "id": 1452, "title": "Superman Returns", "overview": "Superman returns to discover his 5-year absence has allowed Lex Luthor to walk free, and that those he was closest to felt abandoned and have moved on. Luthor plots his ultimate revenge that could see millions killed and change the face of the planet forever, as well as ridding himself of the Man of Steel.", "genres": [ "Science Fiction", "Action", "Adventure" ], "poster": "https://image.tmdb.org/t/p/w500/qIegbn6DSUYmggfwxOBNOVS35q.jpg", "release_date": 1151452800, "_federation": { "indexUid": "mieli", "queriesPosition": 1, "weightedRankingScore": 1.0, "remote": "0" } }, { "id": 324249, "title": "Requiem for Krypton: Making 'Superman Returns'", "overview": "A detailed behind-the-scenes documentary on the making of Superman Returns.", "genres": [ "Documentary" ], "poster": "https://image.tmdb.org/t/p/w500/eC1XKswKSoyDyJXXZszLTuwUHli.jpg", "release_date": 1164672000, "_federation": { "indexUid": "mieli", "queriesPosition": 1, "weightedRankingScore": 0.9907407407407408, "remote": "1" } }, { "id": 294294, "title": "Saltwater", "overview": "This American Indie drama follows several endearing characters as they wade through life seeking happiness, peace and ultimately, love. Will (Ronnie Kerr, Vampire Boys 2, Shut Up and Kiss Me) leaves the Navy after many years, soon reunites old friends and begins to start his new civilian life. His friend Rich (Bruce L Hart) tries to set him up with ruggedly handsome Josh (Ian Roberts-a former Australian professional rugby player, actor and model-Cedar Boys, Superman Returns, Little Fish). While there is immense chemistry between the two, timing and certain ideals never seem to align. When a shocking tragedy happens the two are paired up to pick up the pieces and sort through the after effects. Saltwater is a story about men of all ages, finding love, losing friends, navigating their way through life and knowing it's the journey rather then the destination that's important.", "genres": [ "Romance", "Drama" ], "poster": "https://image.tmdb.org/t/p/w500/bDnLdYqpH9abHo4ASMPKiInx8dm.jpg", "release_date": 1342310400, "_federation": { "indexUid": "mieli", "queriesPosition": 1, "weightedRankingScore": 0.966931216931217, "remote": "1" } }, { "id": 142061, "title": "Batman: The Dark Knight Returns, Part 2", "overview": "Batman has stopped the reign of terror that The Mutants had cast upon his city. Now an old foe wants a reunion and the government wants The Man of Steel to put a stop to Batman.", "genres": [ "Action", "Animation", "Mystery" ], "poster": "https://image.tmdb.org/t/p/w500/arEZYd6uMOFTILne9Ux0A8qctMe.jpg", "release_date": 1357171200, "_federation": { "indexUid": "mieli", "queriesPosition": 0, "weightedRankingScore": 0.8697089947089947, "remote": "1" } } ], "processingTimeMs": 247, "limit": 5, "offset": 0, "estimatedTotalHits": 97, "requestUid": "019bbd3a-5106-70e0-94fc-f58b2f0c28c8", "remoteErrors": {} }Limitations
useNetworkare not supporteduseNetworkat the moment: doing so is not trivial implementation-wise, because chat route expects to be able to open the index (to fetch chat configs), but federated search only opens the indexes once during a short critical section.By @dureuill in #6101
Federated search supports
pageandhitsPerPagePass
federation.pageandfederation.hitsPerPagewith the same meaning as in a regular search request to use exhaustive pagination in the federated searchBy @dureuill in #6101
Speed up settings changes when removing searchable
The settings indexer is more efficient when users are removing searchable attributes from the searchable fields.
By @VedantMadane in #6109
🔒 Security
Solves a low-severity timing attack vulnerability on key comparison by using constant-time comparison
By @curquiza in #6077
🔩 Maintenance
Remove some unwanted dependencies
New Contributors
❤️ Thanks again @VedantMadane for the contribution to this release!
Full Changelog: meilisearch/meilisearch@v1.33.1...v1.34.0
v1.33.1: 🐞Compare Source
In Meilisearch v1.33.1, we released a security fix that affected the dump import, improvements to vector store relevance in massive databases, and a fix related to ranking scores.
🔐 Security Fix
All versions of Meilisearch before v1.33.0 are vulnerable to a path traversal vulnerability involving the dump import functionality.
Importing a specially crafted dump could result in giving access to the Meilisearch instance to arbitrary, specifically formatted files, present on the file system of the Meilisearch instance.
✨ Enhancements
🐛 Bug Fixes
Full Changelog: meilisearch/meilisearch@v1.33.0...v1.33.1
v1.33.0: 🐞Compare Source
✨ Enhancement
Add
/fieldsroute to get all the fields of an index by @YoEight in #6082Adds a new POST
/indexes/{indexUid}/fieldsendpoint that returns detailed metadata about all fields in an index. This endpoint provides comprehensive information about each field's configuration, including display, search, filtering, and localization settings.Implement parallel cleanup of old field IDs by @Kerollmops in #6100
We reduce the time required to perform the dumpless upgrade for instances before v1.32.0 by multi-threading database fetches. By doing that, we noticed improvements from 2 hours and 50 minutes to a bit less than 7 minutes.
Bump hannoy to 0.1.4-nested-rtxns by @Kerollmops in #6103
We updated our internal vector store to speed up the dumpless upgrade by improving the graph rebuilding and improving the speed and relevance of the search by using the explore factor as a limit to stop document searches rather than the query limit.
🪲 Bug fixes
🔒 Security
🔩 Miscellaneous
Thank you @Vipul-045 for your first contribution ❤️
v1.32.2: 🐟Compare Source
🐛 Bug fixes
Full Changelog: meilisearch/meilisearch@v1.32.1...v1.32.2
v1.32.1: 🐟Compare Source
🌈 Improvements
Skip cleaning up the field-ID-based databases
Introduce a
MEILI_EXPERIMENTAL_DISABLE_FID_BASED_DATABASES_CLEANUPenv var to opt out of the field ID-based database cleanup when upgrading a Meilisearch from versions inferior to 1.32.0.Full Changelog: meilisearch/meilisearch@v1.32.0...v1.32.1
v1.32.0: 🐟Compare Source
🌈 Improvements
Log Search performance trace
Introduces comprehensive progress tracking and logging for search operations in Meilisearch. It adds detailed timing information for each step of the search process, enabling better observability and performance analysis.
Extract document operations from payloads in parallel
We accelerate document indexing by processing a large number of tasks in batches or a large number of records in parallel. We expedited the preparation of the payloads by extracting the various changes and assigning internal IDs in parallel. We achieved a 7x speedup on a four-million-document insertion using four CPUs, and the performance scales with the number of CPUs.
The
indexedDocumentsfield in tasks using skipCreation no longer precisely reflects the number of document operations performed, specifically forPOSTandPUToperations. This count may be higher than the actual number of operations, but it doesn't affect the computation; only the reported count is impacted. We prioritize speed over perfect accuracy here, and the documents are still correctly indexed as before.🐛 Bug fixes
Vector sort: Bucket documents with same similarity
Fixed vector sort bucketing so documents with identical similarity scores are grouped together, ensuring subsequent ranking rules are applied correctly.
Properly Delete Documents from FID-Based Databases
Fixes a bug where changing searchableAttributes from ["*"] to a subset of fields left orphaned data in fid-based databases, causing corruption and warnings during search.
Rebuild the graph links when dumpless-upgrading
Bumps hannoy to v0.1.3-nested-rtxns, which fixes graph-related recall issues and adds a method to rebuild graph links to recover previously malformed graphs. Also fixes a minor issue in the dumpless upgrade flow where the upgrade description was not displayed correctly and related operations were not properly associated with the upgrade.
🛠️ Maintenance and Misc.
Update JS SDKs tests to use
pnpminstead ofyarnUpdated the JavaScript SDK tests to use pnpm instead of yarn in CI workflows, switching the package manager across test configurations to ensure the SDK test suite runs correctly and consistently with the current tooling.
Adapt JS tests in SDK tests CI
Updated the SDK tests CI workflow for the JavaScript SDKs
Bump lru from 0.16.2 to 0.16.3
Fix Stacked Borrows violation in
IterMut.Full Changelog: meilisearch/meilisearch@v1.31.0...v1.32.0
v1.31.0: 🦃Compare Source
🌈 Improvements
🗄️ [Enterprise Edition] Make the S3-streaming snapshots an Enterprise Edition feature
The recently introduced S3-streaming snapshots featur
Configuration
📅 Schedule: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).
🚦 Automerge: Disabled by config. Please merge this manually once you are satisfied.
♻ Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.
🔕 Ignore: Close this PR and you won't be reminded about this update again.
This PR was generated by Mend Renovate. View the repository job log.