You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: solutions/search/semantic-search/semantic-search-semantic-text.md
+65-7Lines changed: 65 additions & 7 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -27,6 +27,16 @@ This tutorial uses the `elasticsearch` service for demonstration, which is creat
27
27
28
28
The mapping of the destination index - the index that contains the embeddings that the inference endpoint will generate based on your input text - must be created. The destination index must have a field with the [`semantic_text`](elasticsearch://reference/elasticsearch/mapping-reference/semantic-text.md) field type to index the output of the used inference endpoint.
29
29
30
+
You can run {{infer}} either using the [Elastic {{infer-cap}} Service](/explore-analyze/elastic-inference/eis.md) or on your own ML-nodes. The following examples show you both scenarios.
31
+
32
+
:::::::{tab-set}
33
+
34
+
::::::{tab-item} Using EIS on Serverless
35
+
36
+
```{applies_to}
37
+
serverless: ga
38
+
```
39
+
30
40
```console
31
41
PUT semantic-embeddings
32
42
{
@@ -41,7 +51,61 @@ PUT semantic-embeddings
41
51
```
42
52
43
53
1. The name of the field to contain the generated embeddings.
44
-
2. The field to contain the embeddings is a `semantic_text` field. Since no `inference_id` is provided, the default endpoint `.elser-2-elasticsearch` for the `elasticsearch` service is used. To use a different {{infer}} service, you must create an {{infer}} endpoint first using the [Create {{infer}} API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-inference-put) and then specify it in the `semantic_text` field mapping using the `inference_id` parameter.
54
+
2. The field to contain the embeddings is a `semantic_text` field. Since no `inference_id` is provided, the default endpoint `.elser-v2-elastic` for the `elasticsearch` service is used. This {{infer}} endpoint uses the [Elastic {{infer-cap}} Service (EIS)](/explore-analyze/elastic-inference/eis.md).
55
+
56
+
::::::
57
+
58
+
::::::{tab-item} Using EIS in Cloud
59
+
60
+
```{applies_to}
61
+
stack: ga
62
+
deployment:
63
+
self: unavailable
64
+
```
65
+
66
+
```console
67
+
PUT semantic-embeddings
68
+
{
69
+
"mappings": {
70
+
"properties": {
71
+
"content": { <1>
72
+
"type": "semantic_text", <2>
73
+
"inference_id": ".elser-v2-elastic" <3>
74
+
}
75
+
}
76
+
}
77
+
}
78
+
```
79
+
80
+
1. The name of the field to contain the generated embeddings.
81
+
2. The field to contain the embeddings is a `semantic_text` field.
82
+
3. The `.elser-v2-elastic` preconfigured {{infer}} endpoint for the `elasticsearch` service is used. This {{infer}} endpoint uses the [Elastic {{infer-cap}} Service (EIS)](/explore-analyze/elastic-inference/eis.md).
83
+
84
+
::::::
85
+
86
+
::::::{tab-item} Using ML-nodes
87
+
88
+
```console
89
+
PUT semantic-embeddings
90
+
{
91
+
"mappings": {
92
+
"properties": {
93
+
"content": { <1>
94
+
"type": "semantic_text", <2>
95
+
"inference_id": ".elser-2-elasticsearch" <3>
96
+
}
97
+
}
98
+
}
99
+
}
100
+
```
101
+
102
+
1. The name of the field to contain the generated embeddings.
103
+
2. The field to contain the embeddings is a `semantic_text` field.
104
+
3. The `.elser-2-elasticsearch` preconfigured {{infer}} endpoint for the `elasticsearch` service is used. To use a different {{infer}} service, you must create an {{infer}} endpoint first using the [Create {{infer}} API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-inference-put) and then specify it in the `semantic_text` field mapping using the `inference_id` parameter.
105
+
106
+
::::::
107
+
108
+
:::::::
45
109
46
110
To try the ELSER model on the Elastic Inference Service, explicitly set the `inference_id` to `.elser-2-elastic`. For instructions, refer to [Using `semantic_text` with ELSER on EIS](https://www.elastic.co/docs/reference/elasticsearch/mapping-reference/semantic-text#using-elser-on-eis).
47
111
@@ -50,8 +114,6 @@ If you’re using web crawlers or connectors to generate indices, you have to [u
50
114
51
115
::::
52
116
53
-
54
-
55
117
## Load data [semantic-text-load-data]
56
118
57
119
In this step, you load the data that you later use to create embeddings from it.
@@ -60,7 +122,6 @@ Use the `msmarco-passagetest2019-top1000` data set, which is a subset of the MS
60
122
61
123
Download the file and upload it to your cluster using the [Data Visualizer](../../../manage-data/ingest/upload-data-files.md) in the {{ml-app}} UI. After your data is analyzed, click **Override settings**. Under **Edit field names**, assign `id` to the first column and `content` to the second. Click **Apply**, then **Import**. Name the index `test-data`, and click **Import**. After the upload is complete, you will see an index named `test-data` with 182,469 documents.
62
124
63
-
64
125
## Reindex the data [semantic-text-reindex-data]
65
126
66
127
Create the embeddings from the text by reindexing the data from the `test-data` index to the `semantic-embeddings` index. The data in the `content` field will be reindexed into the `content` semantic text field of the destination index. The reindexed data will be processed by the {{infer}} endpoint associated with the `content` semantic text field.
@@ -70,7 +131,6 @@ This step uses the reindex API to simulate data ingestion. If you are working wi
70
131
71
132
::::
72
133
73
-
74
134
```console
75
135
POST _reindex?wait_for_completion=false
76
136
{
@@ -86,7 +146,6 @@ POST _reindex?wait_for_completion=false
86
146
87
147
1. The default batch size for reindexing is 1000. Reducing size to a smaller number makes the update of the reindexing process quicker which enables you to follow the progress closely and detect errors early.
88
148
89
-
90
149
The call returns a task ID to monitor the progress:
91
150
92
151
```console
@@ -99,7 +158,6 @@ Reindexing large datasets can take a long time. You can test this workflow using
After the data has been indexed with the embeddings, you can query the data using semantic search. Choose between [Query DSL](/explore-analyze/query-filter/languages/querydsl.md) or [{{esql}}](elasticsearch://reference/query-languages/esql.md) syntax to execute the query.
0 commit comments