Skip to content

Commit 3f7dc03

Browse files
Merge branch 'EXP-248-multimodal-search' of github.com:meilisearch/documentation into EXP-248-multimodal-search
2 parents aac30a4 + f9af6cc commit 3f7dc03

6 files changed

+40
-10
lines changed

learn/ai_powered_search/image_search_with_multimodal_embeddings.mdx

Lines changed: 11 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -32,11 +32,11 @@ Much like other embedders, multimodal embedders must set their `source` to `rest
3232

3333
All multimodal embedders must contain an `indexingFragments` field and a `searchFragments` field. Fragments are sets of embeddings built out of specific parts of document data.
3434

35-
Fragments must follow the structure defined by your chosen model and provider.
35+
Fragments must follow the structure defined by the REST API of your chosen provider.
3636

3737
### `indexingFragments`
3838

39-
Use `indexingFragments` to tell Meilisearch what data to use when generating document embeddings.
39+
Use `indexingFragments` to tell Meilisearch how to send document data to the provider's API when generating document embeddings.
4040

4141
For example, when using VoyageAI's multimodal model, an indexing fragment might look like this:
4242

@@ -67,11 +67,11 @@ For example, when using VoyageAI's multimodal model, an indexing fragment might
6767

6868
The example above requests Meilisearch to create two sets of embeddings during indexing: one for the textual description of an image, and another for the actual image.
6969

70-
Each fragment must have one field with a Liquid template where you interpolate document data present in `doc`. In `IMAGE_FRAGMENT_NAME`, that's `image_url` which outputs the plain URL string in the document field `poster_url`. In `TEXT_FRAGMENT_NAME`, `text` contains a longer string contextualizing two document fields, `title` and `description`.
70+
Any JSON string value appearing in a fragment is handled as a Liquid template, where you interpolate document data present in `doc`. In `IMAGE_FRAGMENT_NAME`, that's `image_url` which outputs the plain URL string in the document field `poster_url`. In `TEXT_FRAGMENT_NAME`, `text` contains a longer string contextualizing two document fields, `title` and `description`.
7171

7272
### `searchFragments`
7373

74-
Use `searchFragments` to tell Meilisearch what data to use when converting user queries into embeddings:
74+
Use `searchFragments` to tell Meilisearch how to send search query data to the chosen provider's REST API when converting them into embeddings:
7575

7676
```json
7777
"searchFragments": {
@@ -98,10 +98,15 @@ Use `searchFragments` to tell Meilisearch what data to use when converting user
9898
}
9999
```
100100

101-
This configuration tells Meilisearch how to submit query data for vectorization.
101+
In this example, two modes of search are configured:
102+
103+
1. A textual search based on the `q` parameter, which will be embedded as text
104+
2. An image search based on [data url](https://developer.mozilla.org/en-US/docs/Web/URI/Reference/Schemes/data) rebuilt from the `image.mime` and `image.data` field in the `media` field of the query
102105

103106
Search fragments have access to data present in the query parameters `media` and `q`.
104107

108+
Each semantic search query for this embedder should match exactly one search fragment of this embedder, so the fragments should each have at least one disambiguating field
109+
105110
### Complete embedder configuration
106111

107112
Your embedder should look similar to this example with all fragments and embedding provider data:
@@ -218,7 +223,7 @@ In most cases you will need a GUI interface that allows users to submit their im
218223

219224
With multimodal embedders you can:
220225

221-
1. Configure Meilisearch to vectorize both images and queries
226+
1. Configure Meilisearch to embed both images and queries
222227
2. Add image documents — Meilisearch automatically generates embeddings
223228
3. Accept text or image input from users
224229
4. Run hybrid searches using a mix of textual and input from other types of media, or run pure semantic semantic searches using only non-textual input

snippets/samples/code_samples_facet_search_3.mdx

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -49,7 +49,8 @@ client.Index("books").FacetSearch(&meilisearch.FacetSearchRequest{
4949
```csharp C#
5050
var query = new SearchFacetsQuery()
5151
{
52-
FacetQuery = "c"
52+
FacetQuery = "c",
53+
ExhaustiveFacetCount: true
5354
};
5455
await client.Index("books").FacetSearchAsync("genres", query);
5556
```
Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
<CodeGroup>
2+
3+
```bash cURL
4+
curl \
5+
-X POST 'MEILISEARCH_URL/indexes/restaurants/search' \
6+
-H 'Content-type:application/json' \
7+
--data-binary '{ "filter": "_geoPolygon([45.494181, 9.214024], [45.449484, 9.179175], [45.449486, 9.179177])" }'
8+
```
9+
</CodeGroup>

snippets/samples/code_samples_getting_started_add_documents.mdx

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -78,14 +78,14 @@ $client->index('movies')->addDocuments($movies);
7878
// <dependency>
7979
// <groupId>com.meilisearch.sdk</groupId>
8080
// <artifactId>meilisearch-java</artifactId>
81-
// <version>0.15.0</version>
81+
// <version>0.16.1</version>
8282
// <type>pom</type>
8383
// </dependency>
8484

8585
// For Gradle
8686
// Add the following line to the `dependencies` section of your `build.gradle`:
8787
//
88-
// implementation 'com.meilisearch.sdk:meilisearch-java:0.15.0'
88+
// implementation 'com.meilisearch.sdk:meilisearch-java:0.16.1'
8989

9090
// In your .java file:
9191
import com.meilisearch.sdk;
@@ -192,7 +192,7 @@ namespace Meilisearch_demo
192192
```text Rust
193193
// In your .toml file:
194194
[dependencies]
195-
meilisearch-sdk = "0.29.1"
195+
meilisearch-sdk = "0.30.0"
196196
# futures: because we want to block on futures
197197
futures = "0.3"
198198
# serde: required if you are going to use documents
Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
<CodeGroup>
2+
3+
```dart Dart
4+
await client
5+
.index('INDEX_NAME')
6+
.search('badman', SearchQuery(rankingScoreThreshold: 0.2));
7+
```
8+
</CodeGroup>

snippets/samples/code_samples_rename_an_index_1.mdx

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -6,4 +6,11 @@ curl \
66
-H 'Content-Type: application/json' \
77
--data-binary '{ "uid": "INDEX_B" }'
88
```
9+
10+
```rust Rust
11+
curl \
12+
-X PATCH 'MEILISEARCH_URL/indexes/INDEX_A' \
13+
-H 'Content-Type: application/json' \
14+
--data-binary '{ "uid": "INDEX_B" }'
15+
```
916
</CodeGroup>

0 commit comments

Comments
 (0)