Skip to content

Commit 38ac253

Browse files
committed
Update guide to inform that automatic sharding is a EE feature + document the automatic sharding
1 parent e4d259b commit 38ac253

File tree

1 file changed

+39
-12
lines changed

1 file changed

+39
-12
lines changed

learn/multi_search/implement_sharding.mdx

Lines changed: 39 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -11,6 +11,10 @@ Sharding is the process of splitting an index containing many documents into mul
1111

1212
This guide walks you through activating the `/network` route, configuring the network object, and performing remote federated searches.
1313

14+
<NoticeTag label="Enterprise Edition">
15+
Sharding is an Enterprise Edition feature. You are free to use for evaluation purposes. Please [reach out to us](mailto:[email protected]) before using it in production.
16+
</NoticeTag>
17+
1418
<Tip>
1519
## Configuring multiple instances
1620

@@ -19,7 +23,7 @@ To minimize issues and limit unexpected behavior, instance, network, and index c
1923

2024
## Prerequisites
2125

22-
- Multiple Meilisearch projects (instances) running Meilisearch >=v1.13
26+
- Multiple Meilisearch projects (instances) running Meilisearch >=v1.19
2327

2428
## Activate the `/network` endpoint
2529

@@ -48,6 +52,7 @@ Next, you must configure the network object. It consists of the following fields
4852

4953
- `remotes`: defines a list with the required information to access each remote instance
5054
- `self`: specifies which of the configured `remotes` corresponds to the current instance
55+
- `sharding`: whether to use sharding.
5156

5257
### Setting up the list of remotes
5358

@@ -93,32 +98,54 @@ curl \
9398

9499
Meilisearch processes searches on the remote that corresponds to `self` locally instead of making a remote request.
95100

101+
### Enabling sharding
102+
103+
Finally enable the automatic sharding of documents by Meilisearch on all instances:
104+
105+
```sh
106+
curl \
107+
-X PATCH 'MEILISEARCH_URL/network' \
108+
-H 'Content-Type: application/json' \
109+
--data-binary '{
110+
"sharding": true
111+
}'
112+
```
113+
96114
### Adding or removing an instance
97115

98116
Changing the topology of the network involves moving some documents from an instance to another, depending on your hashing scheme.
99117

100118
As Meilisearch does not provide atomicity across multiple instances, you will need to either:
101119

102-
1. accept search downtime while migrating documents
103-
2. accept some documents will not appear in search results during the migration
120+
1. accept search downtime while migrating documents
121+
2. accept some documents will not appear in search results during the migration
104122
3. accept some duplicate documents may appear in search results during the migration
105123

106124
#### Reducing downtime
107125

108126
If your disk space allows, you can reduce the downtime by applying the following algorithm:
109127

110-
1. Create a new temporary index in each remote instance
111-
2. Compute the new instance for each document
112-
3. Send the documents to the temporary index of their new instance
113-
4. Once Meilisearch has copied all documents to their instance of destination, swap the new index with the previously used index
114-
5. Delete the temporary index after the swap
115-
6. Update network configuration and search queries across all instances
128+
1. Create a new temporary index in each remote instance
129+
2. Compute the new instance for each document
130+
3. Send the documents to the temporary index of their new instance
131+
4. Once Meilisearch has copied all documents to their instance of destination, swap the new index with the previously used index
132+
5. Delete the temporary index after the swap
133+
6. Update network configuration and search queries across all instances
134+
135+
## Create indexes
136+
137+
Create the same empty indexes with the same settings on all instances.
138+
Keeping the settings and indexes in sync is important to avoid errors and unexpected behavior, though not strictly required.
139+
140+
## Add documents
141+
142+
Send documents to any instance to have them replicated to the other instances.
116143

117-
## Create indexes and add documents
144+
Each instance will index the documents they are responsible of and ignore the others.
118145

119-
Create the same empty indexes with the same settings on all instances. Keeping the settings and indexes in sync is important to avoid errors and unexpected behavior, though not strictly required.
146+
You *may* send send the same document to multiple instances, the task will be replicated to all instances, and only the instance responsible for the document will index it.
120147

121-
Distribute your documents across all instances. Do not send the same document to multiple instances as this may lead to duplicate search results. Similarly, you should ensure all future versions of a document are sent to the same instance. Meilisearch recommends you hash their primary key using [rendezvous hashing](https://en.wikipedia.org/wiki/Rendezvous_hashing).
148+
Similarly, you may send any future versions of any document to any instance and only the correct instance will process that document.
122149

123150
### Updating index settings
124151

0 commit comments

Comments
 (0)