Merge branch 'main' into pdl-182

Author: mandel

.github/workflows/build.yml

@@ -39,7 +39,7 @@ jobs:
     - name: pre-commit checks
       run: pre-commit run -a
     - name: run tests
-      run: py.test -v --capture=tee-sys tests
+      run: py.test -v --capture=tee-sys --ignore=tests/test_examples_run.py tests
 
   viewer:
     name: Build PDL live viewer

.github/workflows/run-examples.yml

@@ -0,0 +1,47 @@
+---
+name: Run examples
+
+on:
+  schedule:
+    - cron:  '0 1 * * *'
+  workflow_dispatch:
+
+jobs:
+  tests:
+    name: Exeution tests
+    runs-on: ubuntu-latest
+    strategy:
+      fail-fast: false
+      matrix:
+        python-version: ['3.11', '3.12']
+
+    steps:
+    - uses: actions/checkout@v4
+    - name: Set up Python ${{ matrix.python-version }}
+      uses: actions/setup-python@v5
+      with:
+        python-version: ${{ matrix.python-version }}
+    - name: Cache pip
+      uses: actions/cache@v4
+      with:
+        # This path is specific to Ubuntu
+        path: ${{ env.pythonLocation }}
+        # Look to see if there is a cache hit for the setup file
+        key: ${{ runner.os }}-pip-new3-${{ env.pythonLocation }}-${{ hashFiles('setup.py') }}
+        restore-keys: |
+          ${{ runner.os }}-pip-new3
+          ${{ runner.os }}-new3
+    - name: Install dependencies
+      run: pip install --upgrade --upgrade-strategy eager .[all]
+    - name: pip list packages
+      run: pip list
+    - name: show pip dependencies
+      run: |
+        pip install pipdeptree
+        pipdeptree -fl
+    - name: run tests
+      env:
+        WATSONX_PROJECT_ID: ${{ secrets.WATSONX_PROJECT_ID }}
+        WATSONX_APIKEY: ${{ secrets.WATSONX_APIKEY }}
+        WATSONX_URL: ${{ secrets.WATSONX_URL }}
+      run: py.test -v --capture=tee-sys tests/test_examples_run.py

README.md

@@ -52,7 +52,8 @@ pdl <path/to/example.pdl>
 
 The folder `examples` contains many examples of PDL programs. Several of these examples have been adapted from the LMQL [paper](https://arxiv.org/abs/2212.06094) by Beurer-Kellner et al. The examples cover a variety of prompting patterns such as CoT, RAG, ReAct, and tool use.
 
-We highly recommend using VSCode to edit PDL YAML files. This project has been configured so that every YAML file is associated with the PDL grammar JSONSchema (see [settings](https://github.com/IBM/prompt-declaration-language/blob/main/.vscode/settings.json) and [schema](https://github.com/IBM/prompt-declaration-language/blob/main/src/pdl/pdl-schema.json)). This enables the editor to display error messages when the yaml deviates from the PDL syntax and grammar. It also provides code completion. You can set up your own VSCode PDL projects similarly using the following `.vscode/settings.json` file:
+We highly recommend to edit PDL programs using an editor that support YAML with JSON Schema validation. For example, you can use VSCode with the [YAML extension](https://marketplace.visualstudio.com/items?itemName=redhat.vscode-yaml) and configure it to use the [PDL schema](https://github.com/IBM/prompt-declaration-language/blob/main/src/pdl/pdl-schema.json). This enables the editor to display error messages when the yaml deviates from the PDL syntax and grammar. It also provides code completion.
+The PDL repository has been configured so that every `*.pdl` file is associated with the PDL grammar JSONSchema (see [settings](https://github.com/IBM/prompt-declaration-language/blob/main/.vscode/settings.json)).  You can set up your own VSCode PDL projects similarly using the following `.vscode/settings.json` file:
 
 ```
 {

docs/README.md

@@ -57,7 +57,8 @@ pdl <path/to/example.pdl>
 
 The folder `examples` contains many examples of PDL programs. Several of these examples have been adapted from the LMQL [paper](https://arxiv.org/abs/2212.06094) by Beurer-Kellner et al. The examples cover a variety of prompting patterns such as CoT, RAG, ReAct, and tool use.
 
-We highly recommend using VSCode to edit PDL YAML files. This project has been configured so that every YAML file is associated with the PDL grammar JSONSchema (see [settings](https://github.com/IBM/prompt-declaration-language/blob/main/.vscode/settings.json) and [schema](https://github.com/IBM/prompt-declaration-language/blob/main/src/pdl/pdl-schema.json)). This enables the editor to display error messages when the yaml deviates from the PDL syntax and grammar. It also provides code completion. You can set up your own VSCode PDL projects similarly using the following `./vscode/settings.json` file:
+We highly recommend to edit PDL programs using an editor that support YAML with JSON Schema validation. For example, you can use VSCode with the [YAML extension](https://marketplace.visualstudio.com/items?itemName=redhat.vscode-yaml) and configure it to use the [PDL schema](https://github.com/IBM/prompt-declaration-language/blob/main/src/pdl/pdl-schema.json). This enables the editor to display error messages when the yaml deviates from the PDL syntax and grammar. It also provides code completion.
+The PDL repository has been configured so that every `*.pdl` file is associated with the PDL grammar JSONSchema (see [settings](https://github.com/IBM/prompt-declaration-language/blob/main/.vscode/settings.json)).  You can set up your own VSCode PDL projects similarly using the following `.vscode/settings.json` file:
 
 ```
 {

docs/tutorial.md

@@ -734,11 +734,15 @@ See examples of PDL being called programmatically in Python
 
 ##  Debugging PDL Programs
 
-We highly recommend to use VSCode to edit PDL YAML files. This project has been configured so that every YAML file is associated with the PDL grammar JSONSchema (see [settings](https://github.com/IBM/prompt-declaration-language//blob/main/.vscode/settings.json) and [schema](https://github.com/IBM/prompt-declaration-language//blob/main/src/pdl/pdl-schema.json)). This enables the editor to display error messages when the yaml deviates from the PDL syntax and grammar. It also provides code completion. You can set up your own VSCode PDL projects similarly using this settings and schema files. The PDL interpreter also provides similar error messages. To make sure that the schema is associated with your PDL files, be sure that `PDL Schemas` appear at the bottom right of your VSCode window, or on top of the editor window. You may also need to install the VSCode extension: `YAML Language Support by Red Hat`.
+We highly recommend to edit PDL programs using an editor that support YAML with JSON Schema validation. For example, you can use VSCode with the [YAML extension](https://marketplace.visualstudio.com/items?itemName=redhat.vscode-yaml) and configure it to use the [PDL schema](https://github.com/IBM/prompt-declaration-language/blob/main/src/pdl/pdl-schema.json). The PDL repository has been configured so that every `*.pdl` file is associated with the PDL grammar JSONSchema (see [settings](https://github.com/IBM/prompt-declaration-language/blob/main/.vscode/settings.json)).
+This enables the editor to display error messages when the yaml deviates from the PDL syntax and grammar. It also provides code completion. The PDL interpreter also provides similar error messages. To make sure that the schema is associated with your PDL files, be sure that `PDL Schemas` appear at the bottom right of your VSCode window, or on top of the editor window.
 
 The interpreter prints out a log by default in the file `log.txt`. This log contains the details of inputs and outputs to every block in the program. It is useful to examine this file when the program is behaving differently than expected.
 
 To change the log filename, you can pass it to the interpreter as follows:
+```
+pdl --log <my-example.log> <my-example>
+```
 
 
 ## Live Document Visualizer
@@ -747,7 +751,7 @@ PDL has a Live Document visualizer to help in program understanding given an exe
 To produce an execution trace consumable by the Live Document, you can run the interpreter with the `--trace` argument and set the value to either `json` or `yaml`:
 
 ```
-pdl --trace <file.json> <my-example>
+pdl --trace <my-example_trace.json> <my-example>
 ```
 
 This produces an additional file named `my-example_trace.json` that can be uploaded to the [Live Document](https://ibm.github.io/prompt-declaration-language/viewer/) visualizer tool. Clicking on different parts of the Live Document will show the PDL code that produced that part 

examples/hello/test.pdl

@@ -6,7 +6,7 @@ text:
   call: react
   args:
     question: ${ question }
-    model: watsonx/ibm/granite-34b-code-instruct
+    model: watsonx/ibm/granite-20b-code-instruct
 
 - "\n\n----- Verifying answer... -----\n\n"
 - def: verified

examples/react/multi-agent.pdl

@@ -4,5 +4,4 @@ text:
   - call: react
     args:
       question: How many years ago was the discoverer of the Hudson River born?
-      model: watsonx/mistralai/mistral-large
-
+      model: watsonx/ibm/granite-34b-code-instruct

examples/react/react_call.pdl

@@ -111,7 +111,7 @@ defs:
                     Act: {"name": "Finish", "arguments": {"topic": "2.869047619047619"}}
       - call: react_inner
         args:
-          context: []
+          pdl_context: []
           examples: ${ examples }
           question: ${ question }
           model: ${ model }

examples/react/react_fun.pdl

@@ -111,7 +111,7 @@ defs:
                     Act: {"name": "Finish", "arguments": {"topic": "2.869047619047619"}}
       - call: react_inner
         args:
-          context: []
+          pdl_context: []
           examples: ${ examples }
           question: ${ question }
           model: ${ model }