Update demo-intrinsic to build granite3-rag

.gitignore

@@ -165,4 +165,7 @@ src/pdl/_version.py
 *~
 
 # Mac
-.DS_Store
+.DS_Store
+
+# Model files
+*.gguf

examples/intrinsics/demo-hallucination.pdl

@@ -1,11 +1,86 @@
 # Granite.runtime Intrinsics Demo with PDL
 
-# Provide document(s) to Granite3-rag model with hallucination_tag
-# Granite3-rag model: Base model granite3-dense:8b setup with Granite RAG LoRA (Low-Rank Adaption) on ollama
+# Provide document(s) to granite3-rag model with hallucination_tag
+# granite3-rag model: Base model granite3-dense:8b setup with Granite RAG LoRA (Low-Rank Adaption) on Ollama
 
-# The model responds to a query providing a hallucination score
+# The model responds to a query and provides a hallucination score
 
 text:
+
+# Builds granite3-rag:8b
+- contribute: []
+  lang: python
+  code: |
+    import os
+    import shutil
+    import subprocess
+    import huggingface_hub
+    import ollama
+
+    granite3_rag_model = 'granite3-rag:8b'
+
+    # Check if rag model exists
+    found = False
+    result = ""
+    local_ollama_models = ollama.list().models
+    for model in local_ollama_models:
+        if model.model == granite3_rag_model:
+            found = True
+            result = f"Model {granite3_rag_model} found in Ollama, skipping build"
+            break
+
+    if not found:
+        # Install dependencies
+        # Define the repository and directory
+        llama_cpp_repo = "https://github.com/ggerganov/llama.cpp.git"
+        llama_cpp_dir_name = "llama.cpp"
+
+        # Clone the repository if it doesn't already exist
+        if not os.path.isdir(llama_cpp_dir_name):
+            subprocess.run(["git", "clone", "--depth", "1", llama_cpp_repo], check=True)
+
+        # Install the required packages
+        subprocess.run([
+            "pip", "install",
+            "git+https://github.com/ibm-granite-community/utils.git",
+            "huggingface_hub",
+            "langchain_community",
+            "langchain_ollama",
+            "langchain-milvus",
+            "docling",
+            "-r", f"{llama_cpp_dir_name}/requirements.txt"
+        ], check=True)
+
+        # Ollama pull base model
+        granite3_base_model = 'granite3-dense:8b'
+        ollama.pull(granite3_base_model)
+
+        # HG: download lora adapter
+        lora_folder = huggingface_hub.snapshot_download(repo_id="ibm-granite/granite-rag-3.0-8b-lora")
+        base_folder = huggingface_hub.snapshot_download(repo_id="ibm-granite/granite-3.0-8b-instruct", allow_patterns="*.json")
+        lora_gguf = "granite-rag-3.0-8b-lora-fp16.gguf"
+        subprocess.run(f"python3 llama.cpp/convert_lora_to_gguf.py --outtype f16 --outfile {lora_gguf} --base {base_folder} -- {lora_folder}",
+                        shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
+
+        # Write Modelfile for granite-rag:8b for Ollama
+        model_file = "Modelfile"
+        with open(model_file, "w") as modelfile:
+            modelfile.write(f"""\
+    FROM {granite3_base_model}
+    ADAPTER {lora_gguf}
+    """)
+
+        # Ideally we should use ollama.create() but there is no option to pass a modelfile
+        subprocess.run(f"ollama create {granite3_rag_model} -f Modelfile",
+                    shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
+
+        # Rm dependencies
+        os.remove(model_file)
+        if os.path.isdir(llama_cpp_dir_name):
+            shutil.rmtree(llama_cpp_dir_name)
+
+        result = f"Model {granite3_rag_model} not found on Ollama. Finished building {granite3_rag_model}"
+
 - role: system
   contribute: [context]
   text:

pdl-live-react/demos/run.sh

@@ -17,3 +17,4 @@ EOF
 pdl --trace "$T"/demo5.json "$TOP"/examples/talk/6-code-json.pdl
 pdl --trace "$T"/demo6.json "$UI"/demos/error.pdl || true
 pdl --trace "$T"/demo7.json "$TOP"/examples/talk/4-function.pdl
+pdl --trace "$T"/demo8.json "$TOP"/examples/intrinsics/demo-hallucination.pdl

pdl-live-react/src/demos/demo1.json

@@ -1 +1 @@
-{"kind": "text", "pdl__id": "text", "description": "Simple LLM interaction", "defs": {}, "text": [{"kind": "data", "pdl__id": "text.0", "defs": {}, "data": "write a hello world example, and explain to me how to run it", "pdl__result": "write a hello world example, and explain to me how to run it", "pdl__timing": {"start_nanos": 1741307359368839000, "end_nanos": 1741307359369389000, "timezone": "EST"}, "pdl__is_leaf": true}, {"kind": "model", "pdl__id": "text.1.model", "context": [{"role": "user", "content": "write a hello world example, and explain to me how to run it", "defsite": "text.0"}], "defs": {}, "platform": "litellm", "model": "ollama_chat/granite3.2:2b", "pdl__result": "Sure, I'd be happy to help with that! Here's a simple \"Hello, World!\" program in Python, which is one of the most popular programming languages:\n\n```python\nprint(\"Hello, World!\")\n```\n\nTo run this code, follow these steps:\n\n1. **Install Python**: If you haven't already installed Python on your computer, download it from the official website (https://www.python.org/downloads/). Choose the latest version of Python 3 for Windows, macOS, or Linux. During installation, make sure to check the box that says \"Add Python to PATH\" (for Windows) or ensure that the correct interpreter is selected during setup (for macOS and Linux).\n\n2. **Open a Text Editor**: Use any text editor like Notepad (for Windows), TextEdit (for macOS), or Visual Studio Code, Sublime Text, Atom, etc. for writing your Python code.\n\n3. **Write the Code**: Copy and paste the following line into your text editor:\n\n   ```python\n   print(\"Hello, World!\")\n   ```\n\n4. **Save the File**: Save this file with a `.py` extension, such as `hello_world.py`. For example, if you're using Notepad on Windows, save it as `hello_world.py`.\n\n5. **Open Command Prompt/Terminal**: On Windows, press `Win + R`, type `cmd`, and hit Enter. On macOS or Linux, open Terminal from the Applications > Utilities folder or by searching for \"Terminal\" in Spotlight.\n\n6. **Navigate to Your File's Location**: Use the `cd` command (for Command Prompt on Windows) or `cd` followed by your file path (for Terminal on macOS and Linux). For instance, if you saved the file in the Desktop folder:\n\n   ```\n   cd Desktop\n   ```\n\n7. **Run the Python Script**: Type `python hello_world.py` (for Command Prompt on Windows) or `python3 hello_world.py` (for Terminal on macOS and Linux), then press Enter. You should see the output \"Hello, World!\" displayed in your terminal window.\n\nThat's it! You've just run a simple Python program. This example demonstrates how to print text using the built-in `print()` function.", "pdl__timing": {"start_nanos": 1741307359418667000, "end_nanos": 1741307365945797000, "timezone": "EST"}, "pdl__is_leaf": true}], "pdl__result": "write a hello world example, and explain to me how to run itSure, I'd be happy to help with that! Here's a simple \"Hello, World!\" program in Python, which is one of the most popular programming languages:\n\n```python\nprint(\"Hello, World!\")\n```\n\nTo run this code, follow these steps:\n\n1. **Install Python**: If you haven't already installed Python on your computer, download it from the official website (https://www.python.org/downloads/). Choose the latest version of Python 3 for Windows, macOS, or Linux. During installation, make sure to check the box that says \"Add Python to PATH\" (for Windows) or ensure that the correct interpreter is selected during setup (for macOS and Linux).\n\n2. **Open a Text Editor**: Use any text editor like Notepad (for Windows), TextEdit (for macOS), or Visual Studio Code, Sublime Text, Atom, etc. for writing your Python code.\n\n3. **Write the Code**: Copy and paste the following line into your text editor:\n\n   ```python\n   print(\"Hello, World!\")\n   ```\n\n4. **Save the File**: Save this file with a `.py` extension, such as `hello_world.py`. For example, if you're using Notepad on Windows, save it as `hello_world.py`.\n\n5. **Open Command Prompt/Terminal**: On Windows, press `Win + R`, type `cmd`, and hit Enter. On macOS or Linux, open Terminal from the Applications > Utilities folder or by searching for \"Terminal\" in Spotlight.\n\n6. **Navigate to Your File's Location**: Use the `cd` command (for Command Prompt on Windows) or `cd` followed by your file path (for Terminal on macOS and Linux). For instance, if you saved the file in the Desktop folder:\n\n   ```\n   cd Desktop\n   ```\n\n7. **Run the Python Script**: Type `python hello_world.py` (for Command Prompt on Windows) or `python3 hello_world.py` (for Terminal on macOS and Linux), then press Enter. You should see the output \"Hello, World!\" displayed in your terminal window.\n\nThat's it! You've just run a simple Python program. This example demonstrates how to print text using the built-in `print()` function.", "pdl__timing": {"start_nanos": 1741307359368794000, "end_nanos": 1741307365945859000, "timezone": "EST"}, "pdl__is_leaf": false}
+{"kind": "text", "id": "text", "pdl__timing": {"start_nanos": 1741364950729177000, "end_nanos": 1741364958625818000, "timezone": "EST"}, "description": "Simple LLM interaction", "defs": {}, "text": [{"kind": "data", "id": "text.0", "pdl__timing": {"start_nanos": 1741364950729210000, "end_nanos": 1741364950729736000, "timezone": "EST"}, "defs": {}, "data": "write a hello world example, and explain to me how to run it", "result": "write a hello world example, and explain to me how to run it"}, {"kind": "model", "id": "text.1.model", "context": [{"role": "user", "content": "write a hello world example, and explain to me how to run it", "defsite": "text.0"}], "pdl__timing": {"start_nanos": 1741364950767923000, "end_nanos": 1741364958625777000, "timezone": "EST"}, "defs": {}, "platform": "litellm", "model": "ollama_chat/granite3.2:2b", "result": "Sure, I'll provide a simple \"Hello, World!\" program in three different programming languages: Python, JavaScript (using Node.js), and Java. After each code snippet, I'll explain how to run the example on your local machine or an online platform.\n\n**Python:**\n\n```python\nprint(\"Hello, World!\")\n```\n\nTo run this Python program:\n\n1. Save it in a file with a `.py` extension, e.g., `hello_world.py`.\n2. Open your terminal or command prompt.\n3. Navigate to the directory where you saved `hello_world.py`.\n4. Run the following command:\n\n   ```\n   python hello_world.py\n   ```\n\n**JavaScript (Node.js):**\n\n```javascript\nconsole.log(\"Hello, World!\");\n```\n\nTo run this JavaScript code using Node.js:\n\n1. Ensure you have Node.js installed on your machine. You can download it from https://nodejs.org/.\n2. Open a terminal or command prompt.\n3. Navigate to the directory where you saved `hello_world.js`.\n4. Run the following command:\n\n   ```\n   node hello_world.js\n   ```\n\n**Java:**\n\n```java\npublic class HelloWorld {\n    public static void main(String[] args) {\n        System.out.println(\"Hello, World!\");\n    }\n}\n```\n\nTo run this Java program, you'll need to follow these steps:\n\n1. Save the code in a file named `HelloWorld.java`.\n2. Install the JDK (Java Development Kit). You can download it from https://adoptopenjdk.net/. Follow the instructions for your operating system.\n3. Set up an environment variable called `JAVA_HOME` and add its path to your `PATH`. This helps Java find the JVM (Java Virtual Machine) on your machine.\n   - For Windows:\n     ```cmd\n     setx /M JAVA_HOME \"path\\to\\your\\jdk\"\n     set PATH=%JAVA_HOME%\\bin;%PATH%\n     ```\n   - For macOS/Linux:\n     ```bash\n     export JAVA_HOME=\"/path/to/your/jdk\"\n     export PATH=\"$JAVA_HOME/bin:$PATH\"\n     ```\n4. Open a terminal or command prompt.\n5. Navigate to the directory where you saved `HelloWorld.java`.\n6. Run the following command:\n\n   ```\n   javac HelloWorld.java\n   ```\n   This step compiles your Java code into bytecode (.class files).\n7. After successful compilation, run this command:\n\n   ```\n   java HelloWorld\n   ```\n\nNow, you should see \"Hello, World!\" printed in the terminal or command prompt."}], "result": "write a hello world example, and explain to me how to run itSure, I'll provide a simple \"Hello, World!\" program in three different programming languages: Python, JavaScript (using Node.js), and Java. After each code snippet, I'll explain how to run the example on your local machine or an online platform.\n\n**Python:**\n\n```python\nprint(\"Hello, World!\")\n```\n\nTo run this Python program:\n\n1. Save it in a file with a `.py` extension, e.g., `hello_world.py`.\n2. Open your terminal or command prompt.\n3. Navigate to the directory where you saved `hello_world.py`.\n4. Run the following command:\n\n   ```\n   python hello_world.py\n   ```\n\n**JavaScript (Node.js):**\n\n```javascript\nconsole.log(\"Hello, World!\");\n```\n\nTo run this JavaScript code using Node.js:\n\n1. Ensure you have Node.js installed on your machine. You can download it from https://nodejs.org/.\n2. Open a terminal or command prompt.\n3. Navigate to the directory where you saved `hello_world.js`.\n4. Run the following command:\n\n   ```\n   node hello_world.js\n   ```\n\n**Java:**\n\n```java\npublic class HelloWorld {\n    public static void main(String[] args) {\n        System.out.println(\"Hello, World!\");\n    }\n}\n```\n\nTo run this Java program, you'll need to follow these steps:\n\n1. Save the code in a file named `HelloWorld.java`.\n2. Install the JDK (Java Development Kit). You can download it from https://adoptopenjdk.net/. Follow the instructions for your operating system.\n3. Set up an environment variable called `JAVA_HOME` and add its path to your `PATH`. This helps Java find the JVM (Java Virtual Machine) on your machine.\n   - For Windows:\n     ```cmd\n     setx /M JAVA_HOME \"path\\to\\your\\jdk\"\n     set PATH=%JAVA_HOME%\\bin;%PATH%\n     ```\n   - For macOS/Linux:\n     ```bash\n     export JAVA_HOME=\"/path/to/your/jdk\"\n     export PATH=\"$JAVA_HOME/bin:$PATH\"\n     ```\n4. Open a terminal or command prompt.\n5. Navigate to the directory where you saved `HelloWorld.java`.\n6. Run the following command:\n\n   ```\n   javac HelloWorld.java\n   ```\n   This step compiles your Java code into bytecode (.class files).\n7. After successful compilation, run this command:\n\n   ```\n   java HelloWorld\n   ```\n\nNow, you should see \"Hello, World!\" printed in the terminal or command prompt."}