Skip to content

[DOCS] Simplifies semantic_text tutorial by removing copy_to field #112864

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 6 commits into from
Sep 13, 2024
Merged

[DOCS] Simplifies semantic_text tutorial by removing copy_to field #112864

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
8f7e348
[DOCS] Highlights that copy_to is not required for using semantic_text.
szabosteve Sep 13, 2024
dffb38a
[DOCS] Adds note about possible timeout to the ELSER tab of the infer…
szabosteve Sep 13, 2024
2fc17a7
[DOCS] Simplifies wording.
szabosteve Sep 13, 2024
e243890
[DOCS] Further edits.
szabosteve Sep 13, 2024
6d39fe9
[DOCS] Simplifies tutorial.
szabosteve Sep 13, 2024
a58c9fa
[DOCS] Adds link to hybrid search example.
szabosteve Sep 13, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions docs/reference/search/search-your-data/semantic-search-inference.asciidoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,7 @@ Create an {infer} endpoint by using the <<put-inference-api>>:

include::{es-ref-dir}/tab-widgets/inference-api/infer-api-task-widget.asciidoc[]


[discrete]
[[infer-service-mappings]]
==== Create the index mapping
Expand Down
51 changes: 21 additions & 30 deletions docs/reference/search/search-your-data/semantic-search-semantic-text.asciidoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -59,24 +59,18 @@ If using the Python client, you can set the `timeout` parameter to a higher valu
[[semantic-text-index-mapping]]
==== Create the index mapping

The mapping of the destination index - the index that contains the embeddings
that the inference endpoint will generate based on your input text - must be created. The
destination index must have a field with the <<semantic-text,`semantic_text`>>
field type to index the output of the used inference endpoint.
The mapping of the destination index - the index that contains the embeddings that the inference endpoint will generate based on your input text - must be created.
The destination index must have a field with the <<semantic-text,`semantic_text`>> field type to index the output of the used inference endpoint.

[source,console]
------------------------------------------------------------
PUT semantic-embeddings
{
"mappings": {
"properties": {
"semantic_text": { <1>
"content": { <1>
"type": "semantic_text", <2>
"inference_id": "my-elser-endpoint" <3>
},
"content": { <4>
"type": "text",
"copy_to": "semantic_text" <5>
}
}
}
Expand All @@ -88,9 +82,6 @@ PUT semantic-embeddings
<3> The `inference_id` is the inference endpoint you created in the previous step.
It will be used to generate the embeddings based on the input text.
Every time you ingest data into the related `semantic_text` field, this endpoint will be used for creating the vector representation of the text.
<4> The field to store the text reindexed from a source index in the <<semantic-text-reindex-data,Reindex the data>> step.
<5> The textual data stored in the `content` field will be copied to `semantic_text` and processed by the {infer} endpoint.
The `semantic_text` field will store the embeddings generated based on the input data.


[discrete]
Expand All @@ -116,13 +107,9 @@ you can see an index named `test-data` with 182469 documents.
[[semantic-text-reindex-data]]
==== Reindex the data

Create the embeddings from the text by reindexing the data from the `test-data`
index to the `semantic-embeddings` index. The data in the `content` field will
be reindexed into the `content` field of the destination index.
The `content` field data will be copied to the `semantic_text` field as a result of the `copy_to`
parameter set in the index mapping creation step. The copied data will be
processed by the {infer} endpoint associated with the `semantic_text` semantic text
field.
Create the embeddings from the text by reindexing the data from the `test-data` index to the `semantic-embeddings` index.
The data in the `content` field will be reindexed into the `content` semantic text field of the destination index.
The reindexed data will be processed by the {infer} endpoint associated with the `content` semantic text field.

[source,console]
------------------------------------------------------------
Expand Down Expand Up @@ -164,18 +151,17 @@ POST _tasks/<task_id>/_cancel
[[semantic-text-semantic-search]]
==== Semantic search

After the data set has been enriched with the embeddings, you can query the data
using semantic search. Provide the `semantic_text` field name and the query text
in a `semantic` query type. The {infer} endpoint used to generate the embeddings
for the `semantic_text` field will be used to process the query text.
After the data set has been enriched with the embeddings, you can query the data using semantic search.
Provide the `semantic_text` field name and the query text in a `semantic` query type.
The {infer} endpoint used to generate the embeddings for the `semantic_text` field will be used to process the query text.

[source,console]
------------------------------------------------------------
GET semantic-embeddings/_search
{
"query": {
"semantic": {
"field": "semantic_text", <1>
"field": "content", <1>
"query": "How to avoid muscle soreness while running?" <2>
}
}
Expand All @@ -196,7 +182,7 @@ query from the `semantic-embedding` index:
"_id": "6DdEuo8B0vYIvzmhoEtt",
"_score": 24.972616,
"_source": {
"semantic_text": {
"content": {
"inference": {
"inference_id": "my-elser-endpoint",
"model_settings": {
Expand All @@ -213,15 +199,15 @@ query from the `semantic-embedding` index:
}
},
"id": 1713868,
"content": "There are a few foods and food groups that will help to fight inflammation and delayed onset muscle soreness (both things that are inevitable after a long, hard workout) when you incorporate them into your postworkout eats, whether immediately after your run or at a meal later in the day. Advertisement. Advertisement."
"text": "There are a few foods and food groups that will help to fight inflammation and delayed onset muscle soreness (both things that are inevitable after a long, hard workout) when you incorporate them into your postworkout eats, whether immediately after your run or at a meal later in the day. Advertisement. Advertisement."
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This text field should be contained within the content object, like so:

"content": {
  "text": <text value>,
  "inference": <inference value>,
}

}
},
{
"_index": "semantic-embeddings",
"_id": "-zdEuo8B0vYIvzmhplLX",
"_score": 22.143118,
"_source": {
"semantic_text": {
"content": {
"inference": {
"inference_id": "my-elser-endpoint",
"model_settings": {
Expand All @@ -238,15 +224,15 @@ query from the `semantic-embedding` index:
}
},
"id": 3389244,
"content": "During Your Workout. There are a few things you can do during your workout to help prevent muscle injury and soreness. According to personal trainer and writer for Iron Magazine, Marc David, doing warm-ups and cool-downs between sets can help keep muscle soreness to a minimum."
"text": "During Your Workout. There are a few things you can do during your workout to help prevent muscle injury and soreness. According to personal trainer and writer for Iron Magazine, Marc David, doing warm-ups and cool-downs between sets can help keep muscle soreness to a minimum."
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same here

}
},
{
"_index": "semantic-embeddings",
"_id": "77JEuo8BdmhTuQdXtQWt",
"_score": 21.506052,
"_source": {
"semantic_text": {
"content": {
"inference": {
"inference_id": "my-elser-endpoint",
"model_settings": {
Expand All @@ -263,11 +249,16 @@ query from the `semantic-embedding` index:
}
},
"id": 363742,
"content": "This is especially important if the soreness is due to a weightlifting routine. For this time period, do not exert more than around 50% of the level of effort (weight, distance and speed) that caused the muscle groups to be sore."
"text": "This is especially important if the soreness is due to a weightlifting routine. For this time period, do not exert more than around 50% of the level of effort (weight, distance and speed) that caused the muscle groups to be sore."
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same here

}
},
(...)
]
------------------------------------------------------------
// NOTCONSOLE

[discrete]
[[semantic-text-further-examples]]
==== Further examples

If you want to use `semantic_text` in hybrid search, refer to https://colab.research.google.com/github/elastic/elasticsearch-labs/blob/main/notebooks/search/09-semantic-text.ipynb[this notebook] for a step-by-step guide.
8 changes: 8 additions & 0 deletions docs/reference/tab-widgets/inference-api/infer-api-task.asciidoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,14 @@ is the unique identifier of the {infer} endpoint is `elser_embeddings`.
You don't need to download and deploy the ELSER model upfront, the API request
above will download the model if it's not downloaded yet and then deploy it.

[NOTE]
====
You might see a 502 bad gateway error in the response when using the {kib} Console.
This error usually just reflects a timeout, while the model downloads in the background.
You can check the download progress in the {ml-app} UI.
If using the Python client, you can set the `timeout` parameter to a higher value.
====

// end::elser[]

// tag::hugging-face[]
Expand Down