Merge branch 'main' into pdl-aggregators

Author: mandel

.github/workflows/mkdocs-gh-pages.yml

@@ -33,6 +33,11 @@ jobs:
     steps:
       - name: Checkout
         uses: actions/checkout@v4
+      # Schema
+      - name: Copy schema
+        run: |
+          mkdir -p ./docs/dist
+          cp ./src/pdl/pdl-schema.json ./docs/dist/pdl-schema.json
       # Docs
       - name: Setup Pages
         uses: actions/configure-pages@v5

.github/workflows/tauri-cli.yml

@@ -25,27 +25,53 @@ jobs:
         with:
           node-version: 22
       - name: Install dependencies
+        # sleep 2 to wait for ollama to be running... hack warning
         run: |
           npm ci & sudo apt update && \
             sudo apt install -y libgtk-3-dev libwebkit2gtk-4.1-dev librsvg2-dev patchelf at-spi2-core && \
-            (curl -fsSL https://ollama.com/install.sh | sudo -E sh && sleep 2 && ollama pull granite3.2:2b)
+            (curl -fsSL https://ollama.com/install.sh | sudo -E sh && sleep 2)
           wait
       - name: Test production build
         run: npm run tauri build -- --bundles deb # Skip testing appimage, is this dangerous? It's slow...
       - name: Setup xvfb for screen 0
         run: Xvfb :1 -screen 0 1600x1200x24 &
 
-      - name: Run production build
+      - name: Test beeai compiler
         env:
           DISPLAY: :1
         run: |
           PATH=./src-tauri/target/release/:$PATH
+
+          for i in ./demos/beeai/*.py
+          do pdl compile beeai $i -g -o /tmp/z.json && jq .description /tmp/z.json
+          done
           
-          # 1. `run` subcommand works without any arguments
-          pdl run | grep Usage
+      - name: Test pdl run against production build
+        env:
+          DISPLAY: :1
+        run: |
+          PATH=./src-tauri/target/release/:$PATH
+          
+          # 1a. `run` subcommand errors due to missing required positional parameter
+          pdl run && (echo "This should have failed" && exit 1) || (echo "Great, expected failure received" && exit 0)
+
+          # 1b.`run` subcommand works without any arguments to print Usage
+          pdl run 2>&1 | grep Usage
 
-          # 2. `run` subcommand works with demo1.pdl
+          # 1c.`run` subcommand works with -h to print Usage
+          pdl run -h 2>&1 | grep Usage
+
+          # 2. `run` subcommand works with UI demos (yaml source)
           pdl run ./demos/demo1.pdl | grep 'write a hello'
 
+          # 3. `run` subcommand works with UI demos (json source)
+          # demo4 depends on user input
+          # demo5,demo6 each depend on an external file, and the interpreter does not currently capture this in the trace
+          # demo8 currently requires building a model which the interpreter does not directly support
+          # demo9 takes forever, so... for now skip it
+          for i in ./src/demos/*.json
+          do if [[ $(basename $i) != "demo4.json" ]] && [[ $(basename $i) != "demo5.json" ]] && [[ $(basename $i) != "demo6.json" ]] && [[ $(basename $i) != "demo8.json" ]] && [[  $(basename $i) != "demo9.json" ]]; then pdl run $i; fi
+          done
+
       - name: Tear down xvfb
-        run: killall Xvfb  
+        run: killall Xvfb || true

docs/README.md

@@ -106,6 +106,8 @@ We can pass initial data to the interpreter to populate variables used in a PDL
 pdl --data <JSON-or-YAML-data> <my-example>
 ```
 
+For an example, see [file](https://github.com/IBM/prompt-declaration-language//blob/main/examples/tutorial/free_variables.pdl).
+
 This can also be done by passing a JSON or YAML file:
 
 ```

docs/assets/pdl-ui-3.png

@@ -72,7 +72,7 @@ Using the `input` field, we can also give a directly an array of messages (`role
 --8<-- "./examples/tutorial/calling_llm_with_input_messages.pdl"
 ```
 
-This has the same output as the previous program.
+This has the same output as the previous program. An alternative way of writing this is [this](https://github.com/IBM/prompt-declaration-language//blob/main/examples/tutorial/calling_llm_with_input_messages_var.pdl) program.
 
 ### Parameter defaults for watsonx Granite models
 
@@ -104,7 +104,7 @@ Consider the following example ([file](https://github.com/IBM/prompt-declaration
 ```
 
 Here we assign the output of the model to variable `GEN` using the `def` field. The last line of the program prints out the value of `GEN`. Notice the notation `${ }` for accessing the value of a variable. Any [Jinja](https://jinja.palletsprojects.com/en/3.1.x/) expression is allowed to be used inside these braces. These expressions
-are also used to specify conditions for loops and conditionals. See for example this [file](https://github.com/IBM/prompt-declaration-language//blob/main/examples/tutorial/conditionals_loops.pdl).
+are also used to specify conditions for loops and conditionals. See for example this [file](https://github.com/IBM/prompt-declaration-language//blob/main/examples/tutorial/programs/chatbot.pdl).
 
 When we execute this program, we obtain:
 ```
@@ -115,10 +115,10 @@ GEN is equal to: Hello
 
 ##  Model Chaining
 
-In PDL, we can declaratively chain models together as in the following example ([file](https://github.com/IBM/prompt-declaration-language//blob/main/examples/tutorial/model_chaining.pdl)):
+In PDL, we can declaratively chain models together as in the following example ([file](https://github.com/IBM/prompt-declaration-language//blob/main/examples/tutorial/calling_llm_chaining.pdl)):
 
 ```yaml
---8<-- "./examples/tutorial/model_chaining.pdl"
+--8<-- "./examples/tutorial/calling_llm_chaining.pdl"
 ```
 
 In this program, the first call is to a Granite model with the prompt `"Hello\n"`. The following block in the program prints out the sentence: `"\nDid you just say Hello?\n"`. The final line of the program takes the entire context produced so far and passes it as input to the Granite model. Notice that the input passed to this model is the context up to that point, represented as a conversation. This makes it easy to chain models together and continue building on previous interactions. Notice how the conversational context is accumulated implicitly without requiring the user to explicitly manage messages.
@@ -159,12 +159,18 @@ To reset the context when calling a function, we can pass the special argument:
 
 Notice that the arguments of function calls are expressions and cannot be arbitrary PDL blocks.
 
+A function name can be aliased (see [example](https://github.com/IBM/prompt-declaration-language//blob/main/examples/tutorial/function_alias.pdl)).
+
+The context inherited by a function can be reset at the call site (see [example](https://github.com/IBM/prompt-declaration-language//blob/main/examples/tutorial/function_empty_context.pdl)).
+
+Functions can be declared with optional parameters (see [example](https://github.com/IBM/prompt-declaration-language//blob/main/examples/tutorial/function_optional_params.pdl)).
+
 ##  Grouping Variable Definitions in Defs
 
-In PDL, the above program can be written more neatly by grouping certain variable definitions into a `defs` section, as follows ([file](https://github.com/IBM/prompt-declaration-language//blob/main/examples/tutorial/grouping_definitions.pdl)):
+In PDL, the above program can be written more neatly by grouping certain variable definitions into a `defs` section, as follows ([file](https://github.com/IBM/prompt-declaration-language//blob/main/examples/tutorial/defs.pdl)):
 
 ```yaml
---8<-- "./examples/tutorial/grouping_definitions.pdl"
+--8<-- "./examples/tutorial/defs.pdl"
 ```
 
 
@@ -173,7 +179,7 @@ This program has the same output has the one from the previous section.
 Any block can have a `defs` field defining variables used in that block. Notice it's different than the `def` field which stores the
 result of the block after execution.
 
-
+For another example, see [file](https://github.com/IBM/prompt-declaration-language//blob/main/examples/tutorial/defs-hello.pdl).
 
 ##  Muting Block Output with contribute
 
@@ -310,9 +316,15 @@ Other possible values for `parser` are `yaml`, `jsonl`, or `regex`.
 
 The following example extracts using a regular expression parser the code between triple backtick generated by a model:
 ```yaml
+--8<-- "./examples/tutorial/parser_regex_code.pdl"
+```
+
+Here is another example using a regular expression:
+```yaml
 --8<-- "./examples/tutorial/parser_regex.pdl"
 ```
 
+
 We support the following operations with the`regex` parser (indicated with the `mode` field):
 
 - `fullmatch` (default)
@@ -334,40 +346,43 @@ See [here](https://docs.python.org/3/library/re.html) for more information on ho
 
 ##  Calling code
 
-The following script shows how to execute python code ([file](https://github.com/IBM/prompt-declaration-language//blob/main/examples/tutorial/calling_code.pdl)). The python code is executed locally (or in a containerized way if using `pdl --sandbox`).  In principle, PDL is agnostic of any specific programming language, but we currently only support Python, Jinja, and shell commands. Variables defined in PDL are copied into the global scope of the Python code, so those variables can be used directly in the code. However, mutating variables in Python has no effect on the variables in the PDL program. The result of the code must be assigned to the variable `result` internally to be propagated to the result of the block. A variable `def` on the code block will then be set to this result.
+The following script shows how to execute python code ([file](https://github.com/IBM/prompt-declaration-language//blob/main/examples/tutorial/code_python.pdl)). The python code is executed locally (or in a containerized way if using `pdl --sandbox`).  In principle, PDL is agnostic of any specific programming language, but we currently only support Python, Jinja, and shell commands. Variables defined in PDL are copied into the global scope of the Python code, so those variables can be used directly in the code. However, mutating variables in Python has no effect on the variables in the PDL program. The result of the code must be assigned to the variable `result` internally to be propagated to the result of the block. A variable `def` on the code block will then be set to this result.
 
 In order to define variables that are carried over to the next Python code block, a special variable `PDL_SESSION` can be used, and
 variables assigned to it as fields.
-See for example: ([file](https://github.com/IBM/prompt-declaration-language//blob/main/examples/rag/tfidf_rag.pdl)).
+See for example: ([file](https://github.com/IBM/prompt-declaration-language//blob/main/examples/tutorial/programs/tfidf_rag.pdl)).
 
 ```yaml
---8<-- "./examples/tutorial/calling_code.pdl"
+--8<-- "./examples/tutorial/code_python.pdl"
 ```
 
 This results in the following output (for example):
 ```
 Hello, r!
 ```
 
-PDL also supports Jinja code blocks, as well as PDL code blocks for meta-cycle programming.
+PDL also supports Jinja code blocks, shell commands, as well as PDL code blocks for meta-cycle programming. For more examples, see 
+([Jinja code](https://github.com/IBM/prompt-declaration-language//blob/main/examples/tutorial/code_jinja.pdl)), 
+([shell command](https://github.com/IBM/prompt-declaration-language//blob/main/examples/tutorial/code_command.pdl)), 
+([PDL code](https://github.com/IBM/prompt-declaration-language//blob/main/examples/tutorial/code_pdl.pdl)).
 
 ## Calling REST APIs
 
-PDL programs can contain calls to REST APIs with Python code. Consider a simple weather app ([file](https://github.com/IBM/prompt-declaration-language//blob/main/examples/tutorial/calling_apis.pdl)):
+PDL programs can contain calls to REST APIs with Python code. Consider a simple weather app ([file](https://github.com/IBM/prompt-declaration-language//blob/main/examples/tutorial/programs/weather.pdl)):
 
 ```yaml
---8<-- "./examples/tutorial/calling_apis.pdl"
+--8<-- "./examples/tutorial/programs/weather.pdl"
 ```
 
 In this program, we first define a query about the weather in some location (assigned to variable `QUERY`). The next block is a call to a Granite model with few-shot examples to extract the location, which we assign to variable `LOCATION`. The next block makes an API call with Python (mocked in this example). Here the `LOCATION` is appended to the `url`. The result is a JSON object, which may be hard to interpret for a human user. So we make a final call to an LLM to interpret the JSON in terms of weather. Notice that many blocks have `contribute` set to `[]` to hide intermediate results.
 
 
 ##  Data Block
 
-PDL offers the ability to create JSON data as illustrated by the following example (described in detail in the [Overview](https://ibm.github.io/prompt-declaration-language/#overview) section). The `data` block can gather previously defined variables into a JSON structure. This feature is useful for data generation. Programs such as this one can be generalized to read jsonl files to generate data en masse by piping into another jsonl file ([file](https://github.com/IBM/prompt-declaration-language/blob/main/examples/tutorial/data_block.pdl)).
+PDL offers the ability to create JSON data as illustrated by the following example (described in detail in the [Overview](https://ibm.github.io/prompt-declaration-language/#overview) section). The `data` block can gather previously defined variables into a JSON structure. This feature is useful for data generation. Programs such as this one can be generalized to read jsonl files to generate data en masse by piping into another jsonl file ([file](https://github.com/IBM/prompt-declaration-language/blob/main/examples/tutorial/programs/code-json.pdl)).
 
 ```yaml
---8<-- "./examples/tutorial/data_block.pdl"
+--8<-- "./examples/tutorial/programs/code-json.pdl"
 ```
 
 Notice that in the `data` block the values are interpreted as Jinja expressions. If values need to be PDL programs to be interpreted, then you need to use
@@ -409,11 +424,11 @@ The `import` block means that the PDL code at that file is executed and its scop
 
 ##  Conditionals and Loops
 
-PDL supports conditionals and loops as illustrated in the following example ([file](https://github.com/IBM/prompt-declaration-language//blob/main/examples/tutorial/conditionals_loops.pdl)), which implements a chatbot.
+PDL supports conditionals and loops as illustrated in the following example ([file](https://github.com/IBM/prompt-declaration-language//blob/main/examples/tutorial/programs/chatbot.pdl)), which implements a chatbot.
 
 
 ```yaml
---8<-- "./examples/tutorial/conditionals_loops.pdl"
+--8<-- "./examples/tutorial/programs/chatbot.pdl"
 ```
 
 The first block prompts the user for a query, and this is contributed to the background context. The next
@@ -432,6 +447,8 @@ Notice that the `repeat` and `then` blocks are followed by `text`. This is becau
 The way that the result of each iteration is collated with other iterations can be customized in PDL using
 the `join` feature (see the following section).
 
+Another simple example of using an `if` statement is [this](https://github.com/IBM/prompt-declaration-language//blob/main/examples/tutorial/if.pdl).
+
 ### For Loops
 
 PDL also offers `for` loops over lists.
@@ -536,7 +553,7 @@ as soon as one of the exit conditions is satisfied:
 ### Match block
 
 PDL provides a match block for convenience. 
-Consider the [example](https://github.com/IBM/prompt-declaration-language//blob/main/examples/intrinsics/demo-hallucination.pdl). This shows retrieved RAG documents
+Consider the [example](https://github.com/IBM/prompt-declaration-language//blob/main/examples/tutorial/programs/demo-hallucination.pdl). This shows retrieved RAG documents
 that are then submitted with a query to a RAG Granite model.
 The output contains an answer to the query together with hallucination
 score and possibly a citation.
@@ -565,7 +582,7 @@ The `match` field indicates an expression to match on. The cases follow the `wit
 
 ## Roles and Chat Templates
 
-Consider again the chatbot example ([file](https://github.com/IBM/prompt-declaration-language//blob/main/examples/tutorial/conditionals_loops.pdl)). By default blocks have role `user`, except for model call blocks, which have role `assistant`.
+Consider again the chatbot example ([file](https://github.com/IBM/prompt-declaration-language//blob/main/examples/tutorial/programs/chatbot.pdl)). By default blocks have role `user`, except for model call blocks, which have role `assistant`.
 If we write roles explicitly for the chatbot, we obtain:
 
 
@@ -624,12 +641,12 @@ parameters:
 
 ## Type Checking
 
-Consider the following PDL program ([file](https://github.com/IBM/prompt-declaration-language//blob/main/examples/tutorial/gen-data.pdl)). It first reads the data
-found [here](https://github.com/IBM/prompt-declaration-language//blob/main/examples/tutorial/gen-data.yaml) to form few-shot examples. These demonstrations show how to create
+Consider the following PDL program ([file](https://github.com/IBM/prompt-declaration-language//blob/main/examples/tutorial/type_checking.pdl)). It first reads the data
+found [here](https://github.com/IBM/prompt-declaration-language//blob/main/examples/tutorial/type_checking_data.yaml) to form few-shot examples. These demonstrations show how to create
 some JSON data.
 
 ```yaml
---8<-- "./examples/tutorial/gen-data.pdl"
+--8<-- "./examples/tutorial/type_checking.pdl"
 ```
 
 Upon reading the data we use a parser to parse it into a YAML. The `spec` field indicates the expected type for the
@@ -641,9 +658,9 @@ Similarly, the output of the model call is parsed as YAML, and the `spec` indica
 
 When we run this program, we obtain the output:
 ```
-gen-data.pdl:8 - Type errors during spec checking:
-gen-data.pdl:8 - 30 should be of type <class 'int'>
 {'name': 'John', 'age': '30'}
+type_checking.pdl:9 - Type errors during spec checking:
+type_checking.pdl:9 - twentyfive should be of type <class 'int'>
 ```
 
 Notice that since we asked the age to be produced in letters, we got a string back and this causes a type error indicated above.
@@ -670,9 +687,24 @@ the examples below:
 - `[{question: str, answer: str}]`: same as above
 - `{enum: [red, green, blue]}`: an enumeration
 
+Another example of type checking a list can be found [here](https://github.com/IBM/prompt-declaration-language//blob/main/examples/tutorial/type_list.pdl).
+
 ## Structured Decoding
 
-When a type is specified in a PDL block, it is used for structured decoding with models that support it. The fields `guided_json` and `response_format` are added automatically by the interpreter with a JSON Schema value obtained from the type. Models that support structured decoding will then use this to generate JSON of the correct format.
+When a type is specified in a PDL block, it is used for structured decoding with models that support it. The fields `guided_json` and `response_format` are added automatically by the interpreter with a JSON Schema value obtained from the type. Models on platforms that support structured decoding will then use this to generate JSON of the correct format.
+
+The following [program](https://github.com/IBM/prompt-declaration-language//blob/main/examples/tutorial/structured-decoding.pdl):
+
+```yaml
+--8<-- "./examples/tutorial/structured_decoding.pdl"
+```
+
+produces the output:
+```
+
+What is the color of the sky?
+{'color': 'blue'}
+```
 
 ##  Python SDK
 

docs/tutorial.md

@@ -5,10 +5,9 @@ defs:
     parser: yaml
   TRUTH:
     read: ./ground_truth.txt
-text:
+lastOf:
 - model: ollama_chat/granite3.2:2b
   def: EXPLANATION
-  contribute: []
   input:
      |
       Here is some info about the location of the function in the repo.
@@ -21,10 +20,7 @@ text:
       Explain the following code:
       ```
       ${ CODE.source_code }```
-  parameters:
-    temperature: 0
 - def: EVAL
-  contribute: []
   lang: python
   code:
     |
@@ -35,7 +31,6 @@ text:
     truth = """
     ${ TRUTH }
     """
-    # (In PDL, set `result` to the output you wish for your code block.)
     result = textdistance.levenshtein.normalized_similarity(expl, truth)
 - data:
     input: ${ CODE }