Updates the semantic text tutorial to use EIS (#490)

Author: szabosteve

bin/find-notebooks-to-test.sh

@@ -5,6 +5,7 @@ EXEMPT_NOTEBOOKS=(
     "notebooks/esql/esql-getting-started.ipynb"
     "notebooks/search/07-inference.ipynb"
     "notebooks/search/08-learning-to-rank.ipynb"
+    "notebooks/search/09-semantic-text.ipynb"
     "notebooks/search/10-semantic-reranking-retriever-cohere.ipynb"
     "notebooks/search/11-semantic-reranking-hugging-face.ipynb"
     "notebooks/search/12-semantic-reranking-elastic-rerank.ipynb"
@@ -57,7 +58,6 @@ EXEMPT_NOTEBOOKS__8_12=(
 
 EXEMPT_NOTEBOOKS__8_14=(
     # Add any notebooks that must be skipped on versions 8.14 or older here
-    "notebooks/search/09-semantic-text.ipynb",
     # This notebook has the text_expansion deprecation notice for 8.15. 
     # Only running on 8.15 so includes the deprecation notice and newer so the local output is the same as CI
     "notebooks/langchain/langchain-vector-store-using-elser.ipynb",

notebooks/search/09-semantic-text.ipynb

@@ -34,7 +34,7 @@
     "- An Elastic deployment:\n",
     "  - We'll be using [Elastic Cloud](https://www.elastic.co/guide/en/cloud/current/ec-getting-started.html) for this example (available with a [free trial](https://cloud.elastic.co/registration?onboarding_token=vectorsearch&utm_source=github&utm_content=elasticsearch-labs-notebook))\n",
     "\n",
-    "- Elasticsearch 8.15 or above, or [Elasticsearch serverless](https://www.elastic.co/elasticsearch/serverless)"
+    "- Elasticsearch 9.1 or above, or [Elasticsearch serverless](https://www.elastic.co/elasticsearch/serverless)"
    ]
   },
   {
@@ -84,7 +84,7 @@
    },
    "outputs": [],
    "source": [
-    "!pip install \"elasticsearch<9\""
+    "!pip install elasticsearch"
    ]
   },
   {
@@ -241,109 +241,6 @@
     "Read [this page](https://www.elastic.co/guide/en/elasticsearch/client/python-api/current/connecting.html#connect-self-managed-new) to learn how to connect using API keys."
    ]
   },
-  {
-   "cell_type": "markdown",
-   "id": "22fa643780acd44a",
-   "metadata": {
-    "collapsed": false,
-    "jupyter": {
-     "outputs_hidden": false
-    }
-   },
-   "source": [
-    "## Create the Inference Endpoint\n",
-    "\n",
-    "Let's create the inference endpoint by using the [Create inference API](https://www.elastic.co/guide/en/elasticsearch/reference/current/put-inference-api.html).\n",
-    "\n",
-    "For this example we'll use the [ELSER service](https://www.elastic.co/guide/en/machine-learning/current/ml-nlp-elser.html), but the inference API also supports [many other inference services](https://www.elastic.co/guide/en/elasticsearch/reference/current/put-inference-api.html#put-inference-api-desc)."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "id": "8ee2188ea71324f5",
-   "metadata": {
-    "collapsed": false,
-    "jupyter": {
-     "outputs_hidden": false
-    }
-   },
-   "outputs": [],
-   "source": [
-    "try:\n",
-    "    client.inference.delete(inference_id=\"my-elser-endpoint\")\n",
-    "except exceptions.NotFoundError:\n",
-    "    # Inference endpoint does not exist\n",
-    "    pass\n",
-    "\n",
-    "try:\n",
-    "    client.options(\n",
-    "        request_timeout=60, max_retries=3, retry_on_timeout=True\n",
-    "    ).inference.put(\n",
-    "        task_type=\"sparse_embedding\",\n",
-    "        inference_id=\"my-elser-endpoint\",\n",
-    "        body={\n",
-    "            \"service\": \"elser\",\n",
-    "            \"service_settings\": {\"num_allocations\": 1, \"num_threads\": 1},\n",
-    "        },\n",
-    "    )\n",
-    "    print(\"Inference endpoint created successfully\")\n",
-    "except exceptions.BadRequestError as e:\n",
-    "    if e.error == \"resource_already_exists_exception\":\n",
-    "        print(\"Inference endpoint created successfully\")\n",
-    "    else:\n",
-    "        raise e"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "e94fd66761fd8087",
-   "metadata": {
-    "collapsed": false,
-    "jupyter": {
-     "outputs_hidden": false
-    }
-   },
-   "source": [
-    "Once the endpoint is created, we must wait until the backing ELSER service is deployed.\n",
-    "This can take a few minutes to complete."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "id": "adb33329ce20b2f1",
-   "metadata": {
-    "collapsed": false,
-    "jupyter": {
-     "outputs_hidden": false
-    }
-   },
-   "outputs": [],
-   "source": [
-    "inference_endpoint_info = client.inference.get(inference_id=\"my-elser-endpoint\")\n",
-    "model_id = inference_endpoint_info[\"endpoints\"][0][\"service_settings\"][\"model_id\"]\n",
-    "\n",
-    "while True:\n",
-    "    status = client.ml.get_trained_models_stats(\n",
-    "        model_id=model_id,\n",
-    "    )\n",
-    "\n",
-    "    deployment_stats = status[\"trained_model_stats\"][0].get(\"deployment_stats\")\n",
-    "    if deployment_stats is None:\n",
-    "        print(\"ELSER Model is currently being deployed.\")\n",
-    "        time.sleep(5)\n",
-    "        continue\n",
-    "\n",
-    "    nodes = deployment_stats.get(\"nodes\")\n",
-    "    if nodes is not None and len(nodes) > 0:\n",
-    "        print(\"ELSER Model has been successfully deployed.\")\n",
-    "        break\n",
-    "    else:\n",
-    "        print(\"ELSER Model is currently being deployed.\")\n",
-    "    time.sleep(5)"
-   ]
-  },
   {
    "cell_type": "markdown",
    "id": "818f7a72a83b5776",
@@ -356,7 +253,8 @@
    "source": [
     "## Create the Index\n",
     "\n",
-    "Now we need to create an index with a `semantic_text` field. Let's create one that enables us to perform semantic search on movie plots."
+    "Now we need to create an index with a `semantic_text` field. Let's create one that enables us to perform semantic search on movie plots.\n",
+    "We use the preconfigured `.elser-2-elastic` inference endpoint which uses the ELSER model via the [Elastic Inference Service](https://www.elastic.co/docs/explore-analyze/elastic-inference/eis)."
    ]
   },
   {
@@ -381,7 +279,7 @@
     "            \"plot\": {\"type\": \"text\", \"copy_to\": \"plot_semantic\"},\n",
     "            \"plot_semantic\": {\n",
     "                \"type\": \"semantic_text\",\n",
-    "                \"inference_id\": \"my-elser-endpoint\",\n",
+    "                \"inference_id\": \".elser-v2-elastic\",\n",
     "            },\n",
     "        }\n",
     "    },\n",