Skip to content

Commit 56a87d0

Browse files
committed
[E&A] Refines NLP model conceptual docs.
1 parent 368fd9c commit 56a87d0

File tree

7 files changed

+99
-225
lines changed

7 files changed

+99
-225
lines changed

explore-analyze/machine-learning/nlp.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -13,6 +13,7 @@ You can use {{stack-ml-features}} to analyze natural language data and make pred
1313
* [Add NLP {{infer}} to ingest pipelines](nlp/ml-nlp-inference.md)
1414
* [API quick reference](nlp/ml-nlp-apis.md)
1515
* [ELSER](nlp/ml-nlp-elser.md)
16+
* [Elastic Rerank](nlp/ml-nlp-rerank.md)
1617
* [E5](nlp/ml-nlp-e5.md)
1718
* [Language identification](nlp/ml-nlp-lang-ident.md)
1819
* [Examples](nlp/ml-nlp-examples.md)

explore-analyze/machine-learning/nlp/ml-nlp-built-in-models.md

Lines changed: 0 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -10,8 +10,3 @@ There are {{nlp}} models that are available for use in every cluster out-of-the-
1010
* [ELSER](ml-nlp-elser.md) trained by Elastic
1111
* [E5](ml-nlp-e5.md)
1212
* [{{lang-ident-cap}}](ml-nlp-lang-ident.md)
13-
14-
15-
16-
17-

explore-analyze/machine-learning/nlp/ml-nlp-e5.md

Lines changed: 15 additions & 31 deletions
Original file line numberDiff line numberDiff line change
@@ -4,11 +4,8 @@ mapped_pages:
44
- https://www.elastic.co/guide/en/machine-learning/current/ml-nlp-e5.html
55
---
66

7-
8-
97
# E5 [ml-nlp-e5]
108

11-
129
EmbEddings from bidirEctional Encoder rEpresentations - or E5 - is a {{nlp}} model that enables you to perform multi-lingual semantic search by using dense vector representations. This model is recommended for non-English language documents and queries. If you want to perform semantic search on English language documents, use the [ELSER](ml-nlp-elser.md) model.
1310

1411
[Semantic search](../../../solutions/search/semantic-search.md) provides you search results based on contextual meaning and user intent, rather than exact keyword matches.
@@ -17,23 +14,21 @@ E5 has two versions: one cross-platform version which runs on any hardware and o
1714

1815
Refer to the model cards of the [multilingual-e5-small](https://huggingface.co/elastic/multilingual-e5-small) and the [multilingual-e5-small-optimized](https://huggingface.co/elastic/multilingual-e5-small-optimized) models on HuggingFace for further information including licensing.
1916

20-
2117
## Requirements [e5-req]
2218

2319
To use E5, you must have the [appropriate subscription](https://www.elastic.co/subscriptions) level for semantic search or the trial period activated.
2420

2521
Enabling trained model autoscaling for your E5 deployment is recommended. Refer to [*Trained model autoscaling*](ml-nlp-auto-scale.md) to learn more.
2622

27-
2823
## Download and deploy E5 [download-deploy-e5]
2924

3025
The easiest and recommended way to download and deploy E5 is to use the [{{infer}} API](https://www.elastic.co/guide/en/elasticsearch/reference/current/inference-apis.html).
3126

3227
1. In {{kib}}, navigate to the **Dev Console**.
3328
2. Create an {{infer}} endpoint with the `elasticsearch` service by running the following API request:
3429

35-
```console
36-
PUT _inference/text_embedding/my-e5-model
30+
```console
31+
PUT _inference/text_embedding/my-e5-model
3732
{
3833
"service": "elasticsearch",
3934
"service_settings": {
@@ -42,16 +37,14 @@ The easiest and recommended way to download and deploy E5 is to use the [{{infer
4237
"model_id": ".multilingual-e5-small"
4338
}
4439
}
45-
```
40+
```
4641

4742
The API request automatically initiates the model download and then deploy the model.
4843

49-
5044
Refer to the [`elasticsearch` {{infer}} service documentation](../../../solutions/search/inference-api/elasticsearch-inference-integration.md) to learn more about the available settings.
5145

5246
After you created the E5 {{infer}} endpoint, it’s ready to be used for semantic search. The easiest way to perform semantic search in the {{stack}} is to [follow the `semantic_text` workflow](../../../solutions/search/semantic-search/semantic-search-semantic-text.md).
5347

54-
5548
### Alternative methods to download and deploy E5 [alternative-download-deploy-e5]
5649

5750
You can also download and deploy the E5 model either from **{{ml-app}}** > **Trained Models**, from **Search** > **Indices**, or by using the trained models API in Dev Console.
@@ -60,7 +53,6 @@ You can also download and deploy the E5 model either from **{{ml-app}}** > **Tra
6053
For most cases, the preferred version is the **Intel and Linux optimized** model, it is recommended to download and deploy that version.
6154
::::
6255

63-
6456
::::{dropdown} Using the Trained Models page
6557

6658
#### Using the Trained Models page [trained-model-e5]
@@ -87,7 +79,6 @@ For most cases, the preferred version is the **Intel and Linux optimized** model
8779

8880
::::
8981

90-
9182
::::{dropdown} Using the search indices UI
9283

9384
#### Using the search indices UI [elasticsearch-e5]
@@ -111,65 +102,61 @@ Alternatively, you can download and deploy the E5 model to an {{infer}} pipeline
111102
:class: screenshot
112103
:::
113104

114-
115105
When your E5 model is deployed and started, it is ready to be used in a pipeline.
116106

117107
::::
118108

119-
120109
::::{dropdown} Using the trained models API in Dev Console
121110

122111
#### Using the trained models API in Dev Console [dev-console-e5]
123112

124113
1. In {{kib}}, navigate to the **Dev Console**.
125114
2. Create the E5 model configuration by running the following API call:
126115

127-
```console
128-
PUT _ml/trained_models/.multilingual-e5-small
116+
```console
117+
PUT _ml/trained_models/.multilingual-e5-small
129118
{
130119
"input": {
131120
"field_names": ["text_field"]
132121
}
133122
}
134-
```
123+
```
135124

136125
The API call automatically initiates the model download if the model is not downloaded yet.
137126

138127
3. Deploy the model by using the [start trained model deployment API](https://www.elastic.co/guide/en/elasticsearch/reference/current/start-trained-model-deployment.html) with a delpoyment ID:
139128

140-
```console
141-
POST _ml/trained_models/.multilingual-e5-small/deployment/_start?deployment_id=for_search
142-
```
143-
129+
```console
130+
POST _ml/trained_models/.multilingual-e5-small/deployment/_start?deployment_id=for_search
131+
```
144132

145133
::::
146134

147-
148-
149135
## Deploy the E5 model in an air-gapped environment [air-gapped-install-e5]
150136

151-
If you want to install E5 in an air-gapped environment, you have the following options: * put the model artifacts into a directory inside the config directory on all master-eligible nodes (for `multilingual-e5-small` and `multilingual-e5-small-linux-x86-64`) * install the model by using HuggingFace (for `multilingual-e5-small` model only).
137+
If you want to install E5 in an air-gapped environment, you have the following options:
152138

139+
* put the model artifacts into a directory inside the config directory on all master-eligible nodes (for `multilingual-e5-small` and `multilingual-e5-small-linux-x86-64`)
140+
* install the model by using HuggingFace (for `multilingual-e5-small` model only).
153141

154142
### Model artifact files [e5-model-artifacts]
155143

156144
For the `multilingual-e5-small` model, you need the following files in your system:
157145

158-
```
146+
```url
159147
https://ml-models.elastic.co/multilingual-e5-small.metadata.json
160148
https://ml-models.elastic.co/multilingual-e5-small.pt
161149
https://ml-models.elastic.co/multilingual-e5-small.vocab.json
162150
```
163151

164152
For the optimized version, you need the following files in your system:
165153

166-
```
154+
```url
167155
https://ml-models.elastic.co/multilingual-e5-small_linux-x86_64.metadata.json
168156
https://ml-models.elastic.co/multilingual-e5-small_linux-x86_64.pt
169157
https://ml-models.elastic.co/multilingual-e5-small_linux-x86_64.vocab.json
170158
```
171159

172-
173160
### Using file-based access [_using_file_based_access_3]
174161

175162
For a file-based access, follow these steps:
@@ -178,7 +165,7 @@ For a file-based access, follow these steps:
178165
2. Put the files into a `models` subdirectory inside the `config` directory of your {{es}} deployment.
179166
3. Point your {{es}} deployment to the model directory by adding the following line to the `config/elasticsearch.yml` file:
180167

181-
```
168+
```yml
182169
xpack.ml.model_repository: file://${path.home}/config/models/`
183170
```
184171
@@ -190,7 +177,6 @@ For a file-based access, follow these steps:
190177
9. Provide a deployment ID, select the priority, and set the number of allocations and threads per allocation values.
191178
10. Click **Start**.
192179
193-
194180
### Using the HuggingFace repository [_using_the_huggingface_repository]
195181
196182
You can install the `multilingual-e5-small` model in a restricted or closed network by pointing the `eland_import_hub_model` script to the model’s local files.
@@ -230,8 +216,6 @@ For an offline install, the model first needs to be cloned locally, Git and [Git
230216

231217
Once it’s uploaded to {{es}}, the model will have the ID specified by `--es-model-id`. If it is not set, the model ID is derived from `--hub-model-id`; spaces and path delimiters are converted to double underscores `__`.
232218

233-
234-
235219
## Disclaimer [terms-of-use-e5]
236220

237221
Customers may add third party trained models for management in Elastic. These models are not owned by Elastic. While Elastic will support the integration with these models in the performance according to the documentation, you understand and agree that Elastic has no control over, or liability for, the third party models or the underlying training data they may utilize.

0 commit comments

Comments
 (0)