Move pipelines to templates in the llm-app (#9417)

Co-authored-by: Olivier Ruas <olivier@pathway.com>
GitOrigin-RevId: c419a8a2d12d91418ef55537327a51c1cccd20dc

Author: 2 people

CODE_OF_CONDUCT.md

@@ -42,7 +42,7 @@ Project maintainers have the right and responsibility to remove, edit, or reject
 
 This Code of Conduct applies to all content on pathway.com, Pathway’s GitHub organization, or any other official Pathway web presence allowing for community interactions, as well as at all official Pathway events, whether offline or online.
 
-The Code of Conduct also applies within project spaces and in public spaces whenever an individual is representing Pathwayor its community. Examples of representing a project or community include using an official project e-mail address, posting via an official social media account, or acting as an appointed or de facto representative at an online or offline event. 
+The Code of Conduct also applies within project spaces and in public spaces whenever an individual is representing Pathway or its community. Examples of representing a project or community include using an official project e-mail address, posting via an official social media account, or acting as an appointed or de facto representative at an online or offline event. 
 
 
 ## Conflict Resolution

README.md

@@ -21,13 +21,13 @@ The application templates provided in this repo scale up to **millions of pages
 
 | Application (template)                                                                           | Description                                                                                                                                                                                                                                                                                                                                                         |
 | --------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
-| [`Question-Answering RAG App`](examples/pipelines/demo-question-answering/)    | Basic end-to-end RAG app. A question-answering pipeline that uses the GPT model of choice to provide answers to queries to your documents (PDF, DOCX,...) on a live connected data source (files, Google Drive, Sharepoint,...). You can also try out a [demo REST endpoint](https://pathway.com/solutions/rag-pipelines#try-it-out).              |
-| [`Live Document Indexing (Vector Store / Retriever)`](examples/pipelines/demo-document-indexing/)     | A real-time document indexing pipeline for RAG that acts as a vector store service. It performs live indexing on your documents (PDF, DOCX,...) from a connected data source (files, Google Drive, Sharepoint,...). It can be used with any frontend, or integrated as a retriever backend for a [Langchain](https://pathway.com/blog/langchain-integration) or [Llamaindex](https://pathway.com/blog/llamaindex-pathway) application. You can also try out a [demo REST endpoint](https://pathway.com/solutions/ai-contract-management#try-it-out).         |
-| [`Multimodal RAG pipeline with GPT4o`](examples/pipelines/gpt_4o_multimodal_rag/) | Multimodal RAG using GPT-4o in the parsing stage to index PDFs and other documents from a connected data source files, Google Drive, Sharepoint,...). It is perfect for extracting information from unstructured financial documents in your folders (including charts and tables), updating results as documents change or new ones arrive.|
-| [`Unstructured-to-SQL pipeline + SQL question-answering`](examples/pipelines/unstructured_to_sql_on_the_fly/) | A RAG example which connects to unstructured financial data sources (financial report PDFs), structures the data into SQL, and loads it into a PostgreSQL table. It also answers natural language user queries to these financial documents by translating them into SQL using an LLM and executing the query on the PostgreSQL table. |
-| [`Adaptive RAG App`](examples/pipelines/adaptive-rag/) | A RAG application using Adaptive RAG, a technique developed by Pathway to reduce token cost in RAG up to 4x while maintaining accuracy. |
-| [`Private RAG App with Mistral and Ollama`](examples/pipelines/private-rag/) |  A fully private (local) version of the `demo-question-answering` RAG pipeline using Pathway, Mistral, and Ollama. |
-| [`Slides AI Search App`](examples/pipelines/slides_ai_search/)                                        | An indexing pipeline for retrieving slides. It performs multi-modal of PowerPoint and PDF and maintains live index of your slides."|
+| [`Question-Answering RAG App`](templates/question_answering_rag/)    | Basic end-to-end RAG app. A question-answering pipeline that uses the GPT model of choice to provide answers to queries to your documents (PDF, DOCX,...) on a live connected data source (files, Google Drive, Sharepoint,...). You can also try out a [demo REST endpoint](https://pathway.com/solutions/rag-pipelines#try-it-out).              |
+| [`Live Document Indexing (Vector Store / Retriever)`](templates/document_indexing/)     | A real-time document indexing pipeline for RAG that acts as a vector store service. It performs live indexing on your documents (PDF, DOCX,...) from a connected data source (files, Google Drive, Sharepoint,...). It can be used with any frontend, or integrated as a retriever backend for a [Langchain](https://pathway.com/blog/langchain-integration) or [Llamaindex](https://pathway.com/blog/llamaindex-pathway) application. You can also try out a [demo REST endpoint](https://pathway.com/solutions/ai-contract-management#try-it-out).         |
+| [`Multimodal RAG pipeline with GPT4o`](templates/multimodal_rag/) | Multimodal RAG using GPT-4o in the parsing stage to index PDFs and other documents from a connected data source files, Google Drive, Sharepoint,...). It is perfect for extracting information from unstructured financial documents in your folders (including charts and tables), updating results as documents change or new ones arrive.|
+| [`Unstructured-to-SQL pipeline + SQL question-answering`](templates/unstructured_to_sql_on_the_fly/) | A RAG example which connects to unstructured financial data sources (financial report PDFs), structures the data into SQL, and loads it into a PostgreSQL table. It also answers natural language user queries to these financial documents by translating them into SQL using an LLM and executing the query on the PostgreSQL table. |
+| [`Adaptive RAG App`](templates/adaptive_rag/) | A RAG application using Adaptive RAG, a technique developed by Pathway to reduce token cost in RAG up to 4x while maintaining accuracy. |
+| [`Private RAG App with Mistral and Ollama`](templates/private_rag/) |  A fully private (local) version of the `question_answering_rag` RAG pipeline using Pathway, Mistral, and Ollama. |
+| [`Slides AI Search App`](templates/slides_ai_search/)                                        | An indexing pipeline for retrieving slides. It performs multi-modal of PowerPoint and PDF and maintains live index of your slides."|
 
 
 ## How do these AI Pipelines work?
@@ -38,7 +38,7 @@ The apps rely on the [Pathway Live Data framework](https://github.com/pathwaycom
 
 ## Getting started
 
-Each of the [App templates](examples/pipelines/) in this repo contains a README.md with instructions on how to run it.
+Each of the [App templates](templates/) in this repo contains a README.md with instructions on how to run it.
 
 You can also find [more ready-to-run code templates](https://pathway.com/developers/templates/) on the Pathway website.
 
@@ -47,16 +47,16 @@ You can also find [more ready-to-run code templates](https://pathway.com/develop
 
 Effortlessly extract and organize table and chart data from PDFs, docs, and more with multimodal RAG - in real-time:
 
-![Effortlessly extract and organize table and chart data from PDFs, docs, and more with multimodal RAG - in real-time](https://github.com/pathwaycom/llm-app/blob/main/examples/pipelines/gpt_4o_multimodal_rag/gpt4o_with_pathway_comparison.gif)
+![Effortlessly extract and organize table and chart data from PDFs, docs, and more with multimodal RAG - in real-time](https://github.com/pathwaycom/llm-app/blob/main/templates/multimodal_rag/gpt4o_with_pathway_comparison.gif)
 
-(Check out [`Multimodal RAG pipeline with GPT4o`](examples/pipelines/gpt_4o_multimodal_rag/) to see the whole pipeline in the works. You may also check out the [`Unstructured-to-SQL pipeline`](examples/pipelines/unstructured_to_sql_on_the_fly/) for a minimal example that works with non-multimodal models as well.)
+(Check out [`Multimodal RAG pipeline with GPT4o`](templates/multimodal_rag/) to see the whole pipeline in the works. You may also check out the [`Unstructured-to-SQL pipeline`](templates/unstructured_to_sql_on_the_fly/) for a minimal example that works with non-multimodal models as well.)
 
 
 Automated real-time knowledge mining and alerting:
 
-![Automated real-time knowledge mining and alerting](examples/pipelines/drive_alert/drive_alert_demo.gif)
+![Automated real-time knowledge mining and alerting](templates/drive_alert/drive_alert_demo.gif)
 
-(Check out the [`Alerting when answers change on Google Drive`](https://github.com/pathwaycom/llm-app/tree/main/examples/pipelines/drive_alert) app example.)
+(Check out the [`Alerting when answers change on Google Drive`](https://github.com/pathwaycom/llm-app/tree/main/templates/drive_alert) app example.)
 
 
 ###  Do-it-Yourself Videos

pyproject.toml

@@ -57,7 +57,7 @@ profile = "black"
 
 [tool.mypy]
 python_version = "3.10"
-exclude = ["examples/data", "examples/pipelines/adaptive-rag", "examples/pipelines/private-rag", "examples/pipelines/demo-document-indexing"]
+exclude = ["templates/adaptive_rag", "templates/private_rag", "templates/document_indexing", "templates/multimodal_rag", "templates/question_answering_rag", "examples/pipelines/adaptive-rag", "examples/pipelines/private-rag", "examples/pipelines/demo-document-indexing", "examples/pipelines/demo-question-answering"]
 ignore_missing_imports = true
 check_untyped_defs = true
 warn_redundant_casts = true

templates/adaptive_rag/.env.example

@@ -0,0 +1 @@
+OPENAI_API_KEY="sk-***"

templates/adaptive_rag/Dockerfile

@@ -0,0 +1,16 @@
+FROM pathwaycom/pathway:latest
+
+WORKDIR /app
+
+RUN apt-get update \
+    && apt-get install -y python3-opencv tesseract-ocr-eng \
+    && rm -rf /var/lib/apt/lists/* /var/cache/apt/archives/*
+
+COPY requirements.txt .
+RUN pip install -U --no-cache-dir -r requirements.txt
+
+COPY . .
+
+EXPOSE 8000
+
+CMD ["python", "app.py"]

templates/adaptive_rag/README.md

@@ -0,0 +1,169 @@
+<p align="center" class="flex items-center gap-1 justify-center flex-wrap">
+    <img src="../../../assets/gcp-logo.svg?raw=true" alt="GCP Logo" height="20" width="20">
+    <a href="https://pathway.com/developers/user-guide/deployment/gcp-deploy">Deploy with GCP</a> |
+    <img src="../../../assets/aws-fargate-logo.svg?raw=true" alt="AWS Logo" height="20" width="20">
+    <a href="https://pathway.com/developers/user-guide/deployment/aws-fargate-deploy">Deploy with AWS</a> |
+    <img src="../../../assets/azure-logo.svg?raw=true" alt="Azure Logo" height="20" width="20">
+    <a href="https://pathway.com/developers/user-guide/deployment/azure-aci-deploy">Deploy with Azure</a> |
+    <img src="../../../assets/render.png?raw=true" alt="Render Logo" height="20" width="20">
+    <a href="https://pathway.com/developers/user-guide/deployment/render-deploy"> Deploy with Render </a>
+</p>
+
+
+# End to end Adaptive RAG with Pathway
+
+This is the accompanying code for deploying the `adaptive RAG` technique with Pathway. To understand the technique and learn how it can save tokens without sacrificing accuracy, read [our showcase](https://pathway.com/developers/templates/rag/adaptive-rag).
+
+To learn more about building & deploying RAG applications with Pathway, including containerization, refer to [demo question answering](../question_answering_rag/README.md).
+
+## Introduction
+This app relies on modules provided under `pathway.xpacks.llm`. 
+
+BaseRAGQuestionAnswerer is the base class to build RAG applications with Pathway vector store and Pathway xpack components.
+It is meant to get you started with your RAG application right away. 
+
+Here, we extend the `BaseRAGQuestionAnswerer` to implement the adaptive retrieval and reply to requests in the endpoint `/v2/answer`. 
+Since we are interested in changing the behavior and logic of the RAG, we only modify `answer` function that handles all this logic, and then replies to the post request.
+
+`answer` function takes the `pw_ai_queries` table as the input, this table contains the prompt, and other arguments coming from the post request, see the `BaseRAGQuestionAnswerer` class and defined schemas to learn more about getting inputs with post requests.
+We use the data in this table to call our adaptive retrieval logic.
+
+To do that, we use `answer_with_geometric_rag_strategy_from_index` implementation provided under the `pathway.xpacks.llm.question_answering`. 
+This function takes an index, LLM, prompt and adaptive parameters such as the starting number of documents. Then, iteratively asks the question to the LLM with an increasing number of context documents retrieved from the index.
+
+We encourage you to check the implementation of `answer_with_geometric_rag_strategy_from_index`.
+
+## Customizing the pipeline
+
+The code can be modified by changing the `app.yaml` configuration file. To read more about YAML files used in Pathway templates, read [our guide](https://pathway.com/developers/templates/configure-yaml).
+
+In the `app.yaml` file we define:
+- input connectors
+- LLM
+- embedder
+- index
+and any of these can be replaced or, if no longer needed, removed. For components that can be used check 
+Pathway [LLM xpack](https://pathway.com/developers/user-guide/llm-xpack/overview), or you can implement your own.
+ 
+You can also check our other templates - [Question Answering RAG](https://github.com/pathwaycom/llm-app/tree/main/templates/question_answering_rag), 
+[Multimodal RAG](https://github.com/pathwaycom/llm-app/tree/main/templates/multimodal_rag) or 
+[Private RAG](https://github.com/pathwaycom/llm-app/tree/main/templates/private_rag). As all of these only differ 
+in the YAML configuration file, you can also use them as an inspiration for your custom pipeline.
+
+Here some examples of what can be modified.
+
+### LLM Model
+
+You can choose any of the GPT-3.5 Turbo, GPT-4, or GPT-4 Turbo models proposed by Open AI.
+You can find the whole list on their [models page](https://platform.openai.com/docs/models/gpt-4-and-gpt-4-turbo).
+
+You simply need to change the `model` to the one you want to use:
+```yaml
+$llm: !pw.xpacks.llm.llms.OpenAIChat
+  model: "gpt-3.5-turbo"
+  retry_strategy: !pw.udfs.ExponentialBackoffRetryStrategy
+    max_retries: 6
+  cache_strategy: !pw.udfs.DiskCache
+  temperature: 0.05
+  capacity: 8
+```
+
+The default model is `gpt-3.5-turbo`
+
+You can also use different provider, by using different class from [Pathway LLM xpack](https://pathway.com/developers/user-guide/llm-xpack/overview),
+e.g. here is configuration for locally run Mistral model.
+
+```yaml
+$llm: !pw.xpacks.llm.llms.LiteLLMChat
+  model: "ollama/mistral"
+  retry_strategy: !pw.udfs.ExponentialBackoffRetryStrategy
+    max_retries: 6
+  cache_strategy: !pw.udfs.DiskCache
+  temperature: 0
+  top_p: 1
+  api_base: "http://localhost:11434"
+```
+
+### Webserver
+
+You can configure the host and the port of the webserver.
+Here is the default configuration:
+```yaml
+host: "0.0.0.0"
+port: 8000
+```
+
+### Cache
+
+You can configure whether you want to enable cache, to avoid repeated API accesses, and where the cache is stored.
+Default values:
+```yaml
+with_cache: True
+cache_backend: !pw.persistence.Backend.filesystem
+  path: ".Cache"
+```
+
+### Data sources
+
+You can configure the data sources by changing `$sources` in `app.yaml`.
+You can add as many data sources as you want. You can have several sources of the same kind, for instance, several local sources from different folders.
+The sections below describe how to configure local, Google Drive and Sharepoint source, but you can use any input [connector](https://pathway.com/developers/user-guide/connecting-to-data/connectors) from Pathway package.
+
+By default, the app uses a local data source to read documents from the `data` folder.
+
+#### Local Data Source
+
+The local data source is configured by using map with tag `!pw.io.fs.read`. Then set `path` to denote the path to a folder with files to be indexed.
+
+#### Google Drive Data Source
+
+The Google Drive data source is enabled by using map with tag `!pw.io.gdrive.read`. The map must contain two main parameters:
+- `object_id`, containing the ID of the folder that needs to be indexed. It can be found from the URL in the web interface, where it's the last part of the address. For example, the publicly available demo folder in Google Drive has the URL `https://drive.google.com/drive/folders/1cULDv2OaViJBmOfG5WB0oWcgayNrGtVs`. Consequently, the last part of this address is `1cULDv2OaViJBmOfG5WB0oWcgayNrGtVs`, hence this is the `object_id` you would need to specify.
+- `service_user_credentials_file`, containing the path to the credentials files for the Google [service account](https://cloud.google.com/iam/docs/service-account-overview). To get more details on setting up the service account and getting credentials, you can also refer to [this tutorial](https://pathway.com/developers/user-guide/connectors/gdrive-connector#setting-up-google-drive).
+
+Besides, to speed up the indexing process you may want to specify the `refresh_interval` parameter, denoted by an integer number of seconds. It corresponds to the frequency between two sequential folder scans. If unset, it defaults to 30 seconds.
+
+For the full list of the available parameters, please refer to the Google Drive connector [documentation](https://pathway.com/developers/api-docs/pathway-io/gdrive#pathway.io.gdrive.read).
+
+#### SharePoint Data Source
+
+This data source requires Scale or Enterprise [license key](https://pathway.com/pricing) - you can obtain free Scale key on [Pathway website](https://pathway.com/get-license).
+
+To use it, set the map tag to be `!pw.xpacks.connectors.sharepoint.read`, and then provide values of `url`, `tenant`, `client_id`, `cert_path`, `thumbprint` and `root_path`. To read about the meaning of these arguments, check the Sharepoint connector [documentation](https://pathway.com/developers/api-docs/pathway-xpacks-sharepoint#pathway.xpacks.connectors.sharepoint.read).
+
+## Running the app
+To run the app, depending on the configuration, you may need to set up environmntal variables with LLM provider keys. By default, this template  uses OpenAI API, so to run it you need to set `OPENAI_API_KEY` environmental key or create an `.env` file in this directory with your key: `OPENAI_API_KEY=sk-...`. If you modify the code to use another LLM provider, you may need to set a relevant API key.
+
+### With Docker
+In order to let the pipeline get updated with each change in local files, you need to mount the folder onto the docker. The following commands show how to do that.
+
+```bash
+# Build the image in this folder
+docker build -t adaptiverag .
+
+# Run the image, mount the `data` folder into image 
+# -e is used to pass value of OPENAI_API_KEY environmental variable
+docker run -v ./data:/app/data -e OPENAI_API_KEY -p 8000:8000 adaptiverag
+```
+
+### Locally
+To run locally you need to install the Pathway app with LLM dependencies using:
+```bash
+pip install pathway[all]
+```
+
+Then change your directory in the terminal to this folder and run the app:
+```bash
+python app.py
+```
+
+## Using the app
+
+Finally, query the application with;
+
+```bash
+curl -X 'POST'   'http://0.0.0.0:8000/v2/answer'   -H 'accept: */*'   -H 'Content-Type: application/json'   -d '{
+  "prompt": "What is the start date of the contract?" 
+}'
+```
+> `{"response": "December 21, 2015 [6]"}`