Update llm-app templates (#9526)

GitOrigin-RevId: 4718886d0a2a760580966169774e438e510ea6fb

Author: szymondudycz

templates/adaptive_rag/README.md

@@ -19,7 +19,7 @@ To learn more about building & deploying RAG applications with Pathway, includin
 ## Introduction
 This app relies on modules provided under `pathway.xpacks.llm`. 
 
-BaseRAGQuestionAnswerer is the base class to build RAG applications with Pathway vector store and Pathway xpack components.
+`BaseRAGQuestionAnswerer` is the base class to build RAG applications with Pathway vector store and Pathway xpack components.
 It is meant to get you started with your RAG application right away. 
 
 Here, we extend the `BaseRAGQuestionAnswerer` to implement the adaptive retrieval and reply to requests in the endpoint `/v2/answer`. 
@@ -54,21 +54,20 @@ Here some examples of what can be modified.
 
 ### LLM Model
 
-You can choose any of the GPT-3.5 Turbo, GPT-4, or GPT-4 Turbo models proposed by Open AI.
-You can find the whole list on their [models page](https://platform.openai.com/docs/models/gpt-4-and-gpt-4-turbo).
+You can choose any of the models offered by Open AI, like GPT-5, GPT-4.1, or GPT-4o.
+You can find the whole list on their [models page](https://platform.openai.com/docs/models).
 
-You simply need to change the `model` to the one you want to use:
+You simply need to change the `model` to the one you want to use, e.g., to use GPT-5:
 ```yaml
 $llm: !pw.xpacks.llm.llms.OpenAIChat
-  model: "gpt-3.5-turbo"
+  model: "gpt-5"
   retry_strategy: !pw.udfs.ExponentialBackoffRetryStrategy
     max_retries: 6
-  cache_strategy: !pw.udfs.DiskCache
-  temperature: 0.05
+  cache_strategy: !pw.udfs.DefaultCache
   capacity: 8
 ```
 
-The default model is `gpt-3.5-turbo`
+The default model is `gpt-4.1-mini`.
 
 You can also use different provider, by using different class from [Pathway LLM xpack](https://pathway.com/developers/user-guide/llm-xpack/overview),
 e.g. here is configuration for locally run Mistral model.
@@ -95,19 +94,19 @@ port: 8000
 
 ### Cache
 
-You can configure whether you want to enable cache, to avoid repeated API accesses, and where the cache is stored.
+You can configure whether you want to enable cache or persistence, to avoid repeated API accesses, and where the cache is stored.
 Default values:
 ```yaml
-with_cache: True
-cache_backend: !pw.persistence.Backend.filesystem
+persistence_mode: !pw.PersistenceMode.UDF_CACHING
+persistence_backend: !pw.persistence.Backend.filesystem
   path: ".Cache"
 ```
 
 ### Data sources
 
 You can configure the data sources by changing `$sources` in `app.yaml`.
 You can add as many data sources as you want. You can have several sources of the same kind, for instance, several local sources from different folders.
-The sections below describe how to configure local, Google Drive and Sharepoint source, but you can use any input [connector](https://pathway.com/developers/user-guide/connecting-to-data/connectors) from Pathway package.
+The sections below describe how to configure local, Google Drive and Sharepoint source, and you can check the examples of YAML configuration in our [user guide](https://pathway.com/developers/templates/yaml-snippets/data-sources-examples/). While these are not described in this Section, you can also use any input [connector](https://pathway.com/developers/user-guide/connecting-to-data/connectors) from Pathway package.
 
 By default, the app uses a local data source to read documents from the `data` folder.
 

templates/adaptive_rag/app.py

@@ -1,4 +1,5 @@
 import logging
+from warnings import warn
 
 import pathway as pw
 from dotenv import load_dotenv
@@ -25,17 +26,47 @@ class App(BaseModel):
     host: str = "0.0.0.0"
     port: int = 8000
 
-    with_cache: bool = True
+    with_cache: bool | None = None  # deprecated
+    persistence_backend: pw.persistence.Backend | None = None
+    persistence_mode: pw.PersistenceMode | None = pw.PersistenceMode.UDF_CACHING
     terminate_on_error: bool = False
 
     def run(self) -> None:
-        server = QASummaryRestServer(self.host, self.port, self.question_answerer)
-        server.run(
-            with_cache=self.with_cache,
+        server = QASummaryRestServer(  # noqa: F841
+            self.host, self.port, self.question_answerer
+        )
+
+        if self.persistence_mode is None:
+            if self.with_cache is True:
+                warn(
+                    "`with_cache` is deprecated. Please use `persistence_mode` instead.",
+                    DeprecationWarning,
+                )
+                persistence_mode = pw.PersistenceMode.UDF_CACHING
+            else:
+                persistence_mode = None
+        else:
+            persistence_mode = self.persistence_mode
+
+        if persistence_mode is not None:
+            if self.persistence_backend is None:
+                persistence_backend = pw.persistence.Backend.filesystem("./Cache")
+            else:
+                persistence_backend = self.persistence_backend
+            persistence_config = pw.persistence.Config(
+                persistence_backend,
+                persistence_mode=persistence_mode,
+            )
+        else:
+            persistence_config = None
+
+        pw.run(
+            persistence_config=persistence_config,
             terminate_on_error=self.terminate_on_error,
+            monitoring_level=pw.MonitoringLevel.NONE,
         )
 
-    model_config = ConfigDict(extra="forbid")
+    model_config = ConfigDict(extra="forbid", arbitrary_types_allowed=True)
 
 
 if __name__ == "__main__":

templates/adaptive_rag/app.yaml

@@ -47,7 +47,7 @@ $sources:
 # https://pathway.com/developers/templates/rag-customization/llm-chats
 
 $llm: !pw.xpacks.llm.llms.OpenAIChat
-  model: "gpt-4o-mini"
+  model: "gpt-4.1-mini"
   retry_strategy: !pw.udfs.ExponentialBackoffRetryStrategy
     max_retries: 6
   cache_strategy: !pw.udfs.DefaultCache {}
@@ -56,22 +56,25 @@ $llm: !pw.xpacks.llm.llms.OpenAIChat
 
 # Specifies the embedder model for converting text into embeddings.
 $embedder: !pw.xpacks.llm.embedders.OpenAIEmbedder
-  model: "text-embedding-ada-002"
+  model: "text-embedding-3-small"
   cache_strategy: !pw.udfs.DefaultCache {}
+  retry_strategy: !pw.udfs.ExponentialBackoffRetryStrategy {}
 
 # Defines the splitter settings for dividing text into smaller chunks.
 $splitter: !pw.xpacks.llm.splitters.TokenCountSplitter
   max_tokens: 400
 
 # Configures the parser for processing and extracting information from documents.
 $parser: !pw.xpacks.llm.parsers.DoclingParser
+  async_mode: "fully_async"
+  chunk: false
   cache_strategy: !pw.udfs.DefaultCache {}
 
 # Sets up the retriever factory for indexing and retrieving documents.
-$retriever_factory: !pw.stdlib.indexing.BruteForceKnnFactory
+$retriever_factory: !pw.indexing.UsearchKnnFactory
   reserved_space: 1000
   embedder: $embedder
-  metric: !pw.stdlib.indexing.BruteForceKnnMetricKind.COS
+  metric: !pw.indexing.USearchMetricKind.COS
 
 # Manages the storage and retrieval of documents for the RAG template.
 $document_store: !pw.xpacks.llm.document_store.DocumentStore
@@ -96,8 +99,15 @@ question_answerer: !pw.xpacks.llm.question_answering.AdaptiveRAGQuestionAnswerer
 # host: "0.0.0.0"
 # port: 8000
 
-# Activate on-disk caching for UDFs for which `cache_strategy` is set
-# with_cache: true
+# By default, caching is enabled for UDFs with cache_strategy set.
+# You can disable it by uncommenting the following line.
+# persistence_mode: null
+# You can also set persistence_mode to !pw.PersistenceMode.PERSISTING to enable persistence
+# across restarts.
+# By default, when enabled, Cache is stored in .Cache directory.
+# You can customize the location by uncommenting and modifying the following lines:
+# persistence_backend: !pw.persistence.Backend.filesystem
+#   path: ".Cache"
 
 # If `terminate_on_error` is true then the program will terminate whenever any error is encountered.
 # Defaults to false, uncomment the following line if you want to set it to true

templates/document_indexing/README.md

@@ -40,7 +40,7 @@ Finally, the embeddings are indexed with the capabilities of Pathway's machine-l
 ## Pipeline Organization
 
 This folder contains several objects:
-- `main.py`, the pipeline code using Pathway and written in Python;
+- `app.py`, the pipeline code using Pathway and written in Python;
 - `app.yaml`, the file containing configuration of the pipeline, like embedding model, sources, or the server address;
 - `requirements.txt`, the textfile denoting the pip dependencies for running this pipeline. It can be passed to `pip install -r ...` to install everything that is needed to launch the pipeline locally;
 - `Dockerfile`, the Docker configuration for running the pipeline in the container;
@@ -96,9 +96,9 @@ cache_backend: !pw.persistence.Backend.filesystem
 
 You can configure the data sources by changing `$sources` in `app.yaml`.
 You can add as many data sources as you want. You can have several sources of the same kind, for instance, several local sources from different folders.
-The sections below describe how to configure local, Google Drive and Sharepoint source, but you can use any input [connector](https://pathway.com/developers/user-guide/connecting-to-data/connectors) from Pathway package.
+The sections below describe how to configure local, Google Drive and Sharepoint source, and you can check the examples of YAML configuration in our [user guide](https://pathway.com/developers/templates/yaml-snippets/data-sources-examples/). While these are not described in this Section, you can also use any input [connector](https://pathway.com/developers/user-guide/connecting-to-data/connectors) from Pathway package.
 
-By default, the app uses a local data source to read documents from the `data` folder.
+By default, the app uses a local data source to read documents from the `files-from-indexing` folder.
 
 #### Local Data Source
 

templates/document_indexing/app.py

@@ -1,4 +1,5 @@
 import logging
+from warnings import warn
 
 import pathway as pw
 from dotenv import load_dotenv
@@ -25,17 +26,46 @@ class App(BaseModel):
     host: str = "0.0.0.0"
     port: int = 8000
 
-    with_cache: bool = True
+    with_cache: bool | None = None  # deprecated
+    persistence_backend: pw.persistence.Backend | None = None
+    persistence_mode: pw.PersistenceMode | None = pw.PersistenceMode.UDF_CACHING
     terminate_on_error: bool = False
 
     def run(self) -> None:
-        server = DocumentStoreServer(self.host, self.port, self.document_store)
-        server.run(
-            with_cache=self.with_cache,
+        server = DocumentStoreServer(  # noqa: F841
+            self.host, self.port, self.document_store
+        )
+        if self.persistence_mode is None:
+            if self.with_cache is True:
+                warn(
+                    "`with_cache` is deprecated. Please use `persistence_mode` instead.",
+                    DeprecationWarning,
+                )
+                persistence_mode = pw.PersistenceMode.UDF_CACHING
+            else:
+                persistence_mode = None
+        else:
+            persistence_mode = self.persistence_mode
+
+        if persistence_mode is not None:
+            if self.persistence_backend is None:
+                persistence_backend = pw.persistence.Backend.filesystem("./Cache")
+            else:
+                persistence_backend = self.persistence_backend
+            persistence_config = pw.persistence.Config(
+                persistence_backend,
+                persistence_mode=persistence_mode,
+            )
+        else:
+            persistence_config = None
+
+        pw.run(
+            persistence_config=persistence_config,
             terminate_on_error=self.terminate_on_error,
+            monitoring_level=pw.MonitoringLevel.NONE,
         )
 
-    model_config = ConfigDict(extra="forbid")
+    model_config = ConfigDict(extra="forbid", arbitrary_types_allowed=True)
 
 
 if __name__ == "__main__":

templates/document_indexing/app.yaml

@@ -51,14 +51,16 @@ $splitter: !pw.xpacks.llm.splitters.TokenCountSplitter
   max_tokens: 400
 
 # Configures the parser for processing and extracting information from documents.
-$parser: !pw.xpacks.llm.parsers.UnstructuredParser
+$parser: !pw.xpacks.llm.parsers.DoclingParser
+  async_mode: "fully_async"
+  chunk: false
   cache_strategy: !pw.udfs.DefaultCache {}
 
 # Sets up the retriever factory for indexing and retrieving documents.
-$retriever_factory: !pw.stdlib.indexing.BruteForceKnnFactory
+$retriever_factory: !pw.indexing.UsearchKnnFactory
   reserved_space: 1000
   embedder: $embedder
-  metric: !pw.stdlib.indexing.BruteForceKnnMetricKind.COS
+  metric: !pw.indexing.USearchMetricKind.COS
 
 # Manages the storage and retrieval of documents for the RAG template.
 document_store: !pw.xpacks.llm.document_store.DocumentStore
@@ -71,8 +73,15 @@ document_store: !pw.xpacks.llm.document_store.DocumentStore
 # host: "0.0.0.0"
 # port: 8000
 
-# Activate on-disk caching for UDFs for which `cache_strategy` is set
-# with_cache: true
+# By default, caching is enabled for UDFs with cache_strategy set.
+# You can disable it by uncommenting the following line.
+# persistence_mode: null
+# You can also set persistence_mode to !pw.PersistenceMode.PERSISTING to enable persistence
+# across restarts.
+# By default, when enabled, Cache is stored in .Cache directory.
+# You can customize the location by uncommenting and modifying the following lines:
+# persistence_backend: !pw.persistence.Backend.filesystem
+#   path: ".Cache"
 
 # If `terminate_on_error` is true then the program will terminate whenever any error is encountered.
 # Defaults to false, uncomment the following line if you want to set it to true

templates/multimodal_rag/README.md

@@ -89,24 +89,24 @@ Here some examples of what can be modified.
 
 ### LLM Model
 
-This template by default uses two llm models - GPT-3.5 Turbo for answering queries and GPT-4o for parsing tables and images.
+This template by default uses two llm models - GPT-4.1-mini for answering queries and GPT-4o for parsing tables and images.
 
-You can replace GPT-3.5 Turbo with other Open AI models, like GPT-4, or GPT-4 Turbo.
-You can find the whole list on their [models page](https://platform.openai.com/docs/models/gpt-4-and-gpt-4-turbo).
+You can replace either of them with other Open AI models, like GPT-4.1 or GPT-5, but keep in mind that the model used for parsing needs to support image input.
+You can find the whole list on their [models page](https://platform.openai.com/docs/models).
 
-You simply need to change the `model` to the one you want to use:
+To change the model of the answering llm, you simply need to change the `model` in the `$llm` variable to the one you want to use, e.g. to use `GPT-5` set:
 ```yaml
 $llm: !pw.xpacks.llm.llms.OpenAIChat
-  model: "gpt-3.5-turbo"
+  model: "gpt-5"
   retry_strategy: !pw.udfs.ExponentialBackoffRetryStrategy
     max_retries: 6
-  cache_strategy: !pw.udfs.DiskCache
-  temperature: 0.05
+  cache_strategy: !pw.udfs.DefaultCache {}
+  temperature: 0
   capacity: 8
 ```
 
 You can also use different provider, by using different class from [Pathway LLM xpack](https://pathway.com/developers/user-guide/llm-xpack/overview),
-e.g. here is configuration for locally run Mistral model.
+e.g. here is configuration for locally run Mistral model with Ollama.
 
 ```yaml
 $llm: !pw.xpacks.llm.llms.LiteLLMChat
@@ -132,19 +132,19 @@ port: 8000
 
 ### Cache
 
-You can configure whether you want to enable cache, to avoid repeated API accesses, and where the cache is stored.
+You can configure whether you want to enable cache or persistence, to avoid repeated API accesses, and where the cache is stored.
 Default values:
 ```yaml
-with_cache: True
-cache_backend: !pw.persistence.Backend.filesystem
+persistence_mode: !pw.PersistenceMode.UDF_CACHING
+persistence_backend: !pw.persistence.Backend.filesystem
   path: ".Cache"
 ```
 
 ### Data sources
 
 You can configure the data sources by changing `$sources` in `app.yaml`.
 You can add as many data sources as you want. You can have several sources of the same kind, for instance, several local sources from different folders.
-The sections below describe how to configure local, Google Drive and Sharepoint source, but you can use any input [connector](https://pathway.com/developers/user-guide/connecting-to-data/connectors) from Pathway package.
+The sections below describe how to configure local, Google Drive and Sharepoint source, and you can check the examples of YAML configuration in our [user guide](https://pathway.com/developers/templates/yaml-snippets/data-sources-examples/). While these are not described in this Section, you can also use any input [connector](https://pathway.com/developers/user-guide/connecting-to-data/connectors) from Pathway package.
 
 By default, the app uses a local data source to read documents from the `data` folder.