Release

.github/workflows/publish-quay.yaml

@@ -0,0 +1,46 @@
+name: Quay Publish
+
+on:
+  release:
+    types: [published]
+
+permissions:
+  contents: read
+
+env:
+  REGISTRY: quay.io
+  # github.repository as account/repo
+  IMAGE_NAME: project_pdl/pdl 
+
+jobs:
+  # Build and push a multi-platform Docker image to Docker Hub
+  publish-image:
+    runs-on: ubuntu-latest
+    permissions:
+      contents: read
+      packages: write
+
+    steps:
+      - name: Checkout repository
+        uses: actions/checkout@v3
+
+      - name: Docker Setup QEMU
+        uses: docker/setup-qemu-action@v2
+
+      - name: Set up Docker Buildx
+        uses: docker/setup-buildx-action@v2
+
+      - name: Login to Docker Hub
+        uses: docker/login-action@v2
+        with:
+          registry: ${{ env.REGISTRY }}
+          username: ${{ secrets.QUAYIO_USERNAME }}
+          password: ${{ secrets.QUAYIO_TOKEN }}          
+
+      - name: Build and push ${{ github.ref_name }} to ${{ env.REGISTRY }}
+        uses: docker/build-push-action@v4
+        with:
+          context: .
+          platforms: linux/amd64,linux/arm64
+          push: true
+          tags: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:${{ github.ref_name }},${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:latest

.github/workflows/run-examples.yml

@@ -44,4 +44,5 @@ jobs:
         WATSONX_PROJECT_ID: ${{ secrets.WATSONX_PROJECT_ID }}
         WATSONX_APIKEY: ${{ secrets.WATSONX_APIKEY }}
         WATSONX_URL: ${{ secrets.WATSONX_URL }}
+        REPLICATE_API_TOKEN: ${{ secrets.REPLICATE_API_TOKEN }}
       run: py.test -v --capture=tee-sys tests/test_examples_run.py

README.md

@@ -2,7 +2,7 @@
 
 LLMs will continue to change the way we build software systems. They are not only useful as coding assistants, providing snipets of code, explanations, and code transformations, but they can also help replace components that could only previously be achieved with rule-based systems. Whether LLMs are used as coding assistants or software components, reliability remains an important concern. LLMs have a textual interface and the structure of useful prompts is not captured formally. Programming frameworks do not enforce or validate such structures since they are not specified in a machine-consumable way. The purpose of the Prompt Declaration Language (PDL) is to allow developers to specify the structure of prompts and to enforce it, while providing a unified programming framework for composing LLMs with rule-based systems. 
 
-PDL is based on the premise that interactions between users, LLMs and rule-based systems form a *document*. Consider for example the interactions between a user and a chatbot. At each interaction, the exchanges form a document that gets longer and longer. Similarly, chaining models together or using tools for specific tasks result in outputs that together form a document. PDL allows users to specify the shape and contents of such documents in a declarative way (in YAML), and is agnostic of any programming language. Because of its document-oriented nature, it can be used to easily express a variety of data generation tasks (inference, data synthesis, data generation for model training, etc...).
+PDL is based on the premise that interactions between users, LLMs and rule-based systems form a *document*. Consider for example the interactions between a user and a chatbot. At each interaction, the exchanges form a document that gets longer and longer. Similarly, chaining models together or using tools for specific tasks result in outputs that together form a document. PDL allows users to specify the shape of data in such documents in a declarative way (in YAML), and is agnostic of any programming language. Because of its document-oriented nature, it can be used to easily express a variety of data generation tasks (inference, data synthesis, data generation for model training, etc...).
 
 PDL provides the following features:
 - Ability to use any LLM locally or remotely via [LiteLLM](https://www.litellm.ai/), including [IBM's watsonx](https://www.ibm.com/watsonx)
@@ -26,6 +26,7 @@ See below for a quick reference, followed by [installation notes](#interpreter_i
 
 (See also [PDF version](https://github.com/IBM/prompt-declaration-language/blob/main/docs/assets/pdl_quick_reference.pdf).)
 
+Pro Tip: When writing loops and conditionals with `repeat`, `for`, and `if-then-else`, start the body of the loop or conditional (`then`/`else`) with `text` in order to see the results of every block in the body. See for example this [file](https://github.com/IBM/prompt-declaration-language/blob/main/examples/tutorial/conditionals_loops.pdl).
 
 ## Interpreter Installation
 
@@ -39,13 +40,16 @@ pip install prompt-declaration-language
 
 To install the dependencies for development of PDL and execute all the example, execute the command:
 ```
-pip install 'prompt-declaration-language[dev]'
 pip install 'prompt-declaration-language[examples]'
-pip install 'prompt-declaration-language[docs]'
 ```
 
-In order to run the examples that use foundation models hosted on [watsonx](https://www.ibm.com/watsonx) via LiteLLM, you need a watsonx account (a free plan is available) and set up the following environment variables:
-- `WATSONX_URL`, the API url (set to `https://{region}.ml.cloud.ibm.com`) of your watsonx instance
+Most examples in this repository use IBM Granite models on [Replicate](https://replicate.com/).
+In order to run these examples, you need to create a free account
+on Replicate, get an API key and store it in the environment variable:
+- `REPLICATE_API_TOKEN`
+
+In order to use foundation models hosted on [Watsonx](https://www.ibm.com/watsonx) via LiteLLM, you need a WatsonX account (a free plan is available) and set up the following environment variables:
+- `WATSONX_URL`, the API url (set to `https://{region}.ml.cloud.ibm.com`) of your WatsonX instance. The region can be found by clicking in the upper right corner of the Watsonx dashboard (for example a valid region is `us-south` ot `eu-gb`).
 - `WATSONX_APIKEY`, the API key (see information on [key creation](https://cloud.ibm.com/docs/account?topic=account-userapikey&interface=ui#create_user_key))
 - `WATSONX_PROJECT_ID`, the project hosting the resources (see information about [project creation](https://www.ibm.com/docs/en/watsonx/saas?topic=projects-creating-project) and [finding project ID](https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/fm-project-id.html?context=wx)).
 
@@ -117,30 +121,44 @@ In PDL, we can write some YAML to create a prompt and call an LLM:
 ```yaml
 description: Hello world
 text:
-- Hello,
-- model: watsonx/ibm/granite-34b-code-instruct
+- "Hello\n"
+- model: replicate/ibm-granite/granite-20b-code-instruct-8k
   parameters:
-    decoding_method: greedy
-    stop:
-    - '!'
-    include_stop_sequence: true
+    stop_sequences: '!'
+    temperature: 0
 ```
 
-The `description` field is a description for the program. Field `text` contains a list of either strings or *block*s which together form the text to be produced. In this example, the text starts with the string `"Hello"` followed by a block that calls out to a model. In this case, it is model with id `watsonx/ibm/granite-34b-code-instruct` from [watsonx](https://www.ibm.com/watsonx), via LiteLLM, with the indicated parameters: the stop sequence is `!`, which is to be included in the output. The input to the model call is everything that has been produced so far in the document (here `Hello`).
+The `description` field is a description for the program. Field `text` contains a list of either strings or *block*s which together form the text to be produced. In this example, the text starts with the string `"Hello\n"` followed by a block that calls out to a model. In this case, it is model with id `replicate/ibm-granite/granite-20b-code-instruct-8k` on Replicate, via LiteLLM, with the indicated parameters: the stop sequence is `!`, and temperature set to `0`. Stop sequences are provided with a comman separated list of strings. The input to the model call is everything that has been produced so far in the program (here `"Hello\n"`).
 
 When we execute this program using the PDL interpreter:
 
 ```
 pdl examples/hello/hello.pdl
 ```
 
-we obtain the following document:
+we obtain the following:
 
 ```
-Hello, World!
+Hello
+Hello
 ```
 
-where the portion `, World!` was produced by granite. In general, PDL provides blocks for calling models, Python code, and makes it easy to compose them together with control structures (sequencing, conditions, loops).
+where the second `Hello` was produced by Granite. In general, PDL provides blocks for calling models, Python code, and makes it easy to compose them together with control structures (sequencing, conditions, loops).
+
+A similar example on WatsonX would look as follows:
+
+```yaml
+description: Hello world
+text:
+- Hello,
+- model: watsonx/ibm/granite-34b-code-instruct
+  parameters:
+    decoding_method: greedy
+    stop:
+    - '!'
+```
+
+Notice the syntactic differences. Model ids on WatsonX start with `watsonx`. The `decoding_method` can be set to `greedy`, rather than setting the temperature to `0`. Also, `stop_sequences` are indicated with the keyword `stop` instead as a list of strings.
 
 A PDL program computes 2 data structures. The first is a JSON corresponding to the result of the overall program, obtained by aggregating the results of each block. This is what is printed by default when we run the interpreter. The second is a conversational background context, which is a list of role/content pairs, where we implicitly keep track of roles and content for the purpose of communicating with models that support chat APIs. The contents in the latter correspond to the results of each block. The conversational background context is what is used to make calls to LLMs via LiteLLM.
 
@@ -210,7 +228,7 @@ defs:
     parser: yaml
 text:
 - "\n${ CODE.source_code }\n"
-- model: watsonx/ibm/granite-34b-code-instruct
+- model: replicate/ibm-granite/granite-3.0-8b-instruct
   input:
       - |
         Here is some info about the location of the function in the repo.
@@ -248,11 +266,19 @@ public static Map<String, String> deserializeOffsetMap(String lastSourceOffset)
   return offsetMap;
 }
 
+This Java function, `deserializeOffsetMap`, is designed to deserialize a JSON string into a `Map<String, String>`. Here's a breakdown of what it does:
+
+1. It takes a single argument, `lastSourceOffset`, which is expected to be a JSON string.
+
+2. It initializes a `Map<String, String>` called `offsetMap`.
 
+3. If `lastSourceOffset` is either `null` or an empty string, it creates a new `HashMap` and assigns it to `offsetMap`.
 
-The function `deserializeOffsetMap` takes a string as input and returns a map. It first checks if the input string is null or empty. If it is, it creates a new empty map and returns it. Otherwise, it uses the Jackson library to parse the input string into a map and returns it.
+4. If `lastSourceOffset` is not `null` or an empty string, it uses the `JSON_MAPPER` object (which is presumably an instance of a JSON deserialization library like Jackson) to deserialize the JSON string into a `Map<String, String>` and assigns it to `offsetMap`.
 
-The `@SuppressWarnings("unchecked")` annotation is used to suppress the warning that the type of the parsed map is not checked. This is because the Jackson library is used to parse the input string into a map, but the specific type of the map is not known at compile time. Therefore, the warning is suppressed to avoid potential issues.
+5. Finally, it returns the `offsetMap`.
+
+The `@SuppressWarnings("unchecked")` annotation is used to suppress a compile-time warning about the raw use of the `Map` type. This is because the `JSON_MAPPER.readValue` method returns a `Map` object, but the compiler doesn't know that this `Map` will be a `Map<String, String>`. The `unchecked` warning is suppressed to avoid this compile-time warning.
 ```
 
 Notice that in PDL variables are used to templatize any entity in the document, not just textual prompts to LLMs. We can add a block to this document to evaluate the quality of the output using a similarity metric with respect to our [ground truth](https://github.com/IBM/prompt-declaration-language/blob/main/examples/code/ground_truth.txt). See [file](https://github.com/IBM/prompt-declaration-language/blob/main/examples/code/code-eval.pdl):
@@ -267,7 +293,7 @@ defs:
     read: ./ground_truth.txt
 text:
 - "\n${ CODE.source_code }\n"
-- model: watsonx/ibm/granite-34b-code-instruct
+- model: replicate/ibm-granite/granite-3.0-8b-instruct
   def: EXPLANATION
   input: |
       Here is some info about the location of the function in the repo.
@@ -316,14 +342,23 @@ public static Map<String, String> deserializeOffsetMap(String lastSourceOffset)
   return offsetMap;
 }
 
+This Java method, `deserializeOffsetMap`, is designed to convert a JSON string into a `Map<String, String>`. Here's a breakdown of the code:
+
+1. The method takes a single argument, `lastSourceOffset`, which is expected to be a JSON string.
+
+2. It initializes a `Map<String, String>` called `offsetMap`.
 
-The function `deserializeOffsetMap` takes a string as input and returns a map. It first checks if the input string is null or empty. If it is, it creates a new empty map and returns it. Otherwise, it uses the Jackson library to parse the input string into a map and returns it.
+3. If `lastSourceOffset` is either `null` or an empty string, it creates a new `HashMap` and assigns it to `offsetMap`.
 
-The `@SuppressWarnings("unchecked")` annotation is used to suppress the warning that the type of the parsed map is not checked. This is because the Jackson library is used to parse the input string into a map, but the specific type of the map is not known at compile time. Therefore, the warning is suppressed to avoid potential issues.
+4. If `lastSourceOffset` is not `null` or empty, it uses the `JSON_MAPPER` object (which is presumably an instance of `ObjectMapper` from the Jackson library) to convert the JSON string into a `Map<String, String>` and assigns it to `offsetMap`.
+
+5. Finally, it returns the `offsetMap`.
+
+The `@SuppressWarnings("unchecked")` annotation is used to suppress a potential unchecked warning that might occur if the JSON string does not match the expected `Map<String, String>` type.
 
 EVALUATION:
 The similarity (Levenshtein) between this answer and the ground truth is:
-0.9967637540453075
+0.30199115044247793
 ```
 
 PDL allows rapid prototyping of prompts by allowing the user to change prompts and see their immediate effects on metrics. Try it!
@@ -339,7 +374,7 @@ defs:
   TRUTH:
     read: ./ground_truth.txt
 text:
-- model: watsonx/ibm/granite-34b-code-instruct
+- model: replicate/ibm-granite/granite-3.0-8b-instruct
   def: EXPLANATION
   contribute: []
   input:
@@ -401,8 +436,8 @@ This is similar to a spreadsheet for tabular data, where data is in the forefron
 
 ## Additional Notes
 
-When using Granite models on watsonx, we use the following defaults for model parameters (except `granite-20b-code-instruct-r1.1`):
-  - `decoding_method`: `greedy`
+When using Granite models, we use the following defaults for model parameters (except `granite-20b-code-instruct-r1.1`):
+  - `decoding_method`: `greedy`, (`temperature`: 0)
   - `max_new_tokens`: 1024
   - `min_new_tokens`: 1
   - `repetition_penalty`: 1.05