From 68ce1d2a4c45f1c52e7ff2c93c1ba66c4959ae8d Mon Sep 17 00:00:00 2001 From: Louis Mandel Date: Wed, 16 Oct 2024 18:05:13 -0400 Subject: [PATCH 1/5] Fix watsonx capitalization --- README.md | 14 +++++++------- docs/README.md | 14 +++++++------- 2 files changed, 14 insertions(+), 14 deletions(-) diff --git a/README.md b/README.md index 2a184cb99..69f3f531a 100644 --- a/README.md +++ b/README.md @@ -5,7 +5,7 @@ LLMs will continue to change the way we build software systems. They are not onl PDL is based on the premise that interactions between users, LLMs and rule-based systems form a *document*. Consider for example the interactions between a user and a chatbot. At each interaction, the exchanges form a document that gets longer and longer. Similarly, chaining models together or using tools for specific tasks result in outputs that together form a document. PDL allows users to specify the shape and contents of such documents in a declarative way (in YAML), and is agnostic of any programming language. Because of its document-oriented nature, it can be used to easily express a variety of data generation tasks (inference, data synthesis, data generation for model training, etc...). PDL provides the following features: -- Ability to use any LLM locally or remotely via [LiteLLM](https://www.litellm.ai/), including [IBM's Watsonx](https://www.ibm.com/watsonx) +- Ability to use any LLM locally or remotely via [LiteLLM](https://www.litellm.ai/), including [IBM's watsonx](https://www.ibm.com/watsonx) - Ability to templatize not only prompts for one LLM call, but also composition of LLMs with tools (code and APIs). Templates can encompass tasks of larger granularity than a single LLM call - Control structures: variable definitions and use, conditionals, loops, functions - Ability to read from files and stdin, including JSON data @@ -24,7 +24,7 @@ See below for a quick reference, followed by [installation notes](#interpreter_i PDL Quick Reference -(See also [PDF version](https://github.com/IBM/prompt-declaration-language/blob/main/docs/assets/pdl_quick_reference.pdf).] +(See also [PDF version](https://github.com/IBM/prompt-declaration-language/blob/main/docs/assets/pdl_quick_reference.pdf).) ## Interpreter Installation @@ -44,8 +44,8 @@ pip install 'prompt-declaration-language[examples]' pip install 'prompt-declaration-language[docs]' ``` -In order to run the examples that use foundation models hosted on [Watsonx](https://www.ibm.com/watsonx) via LiteLLM, you need a WatsonX account (a free plan is available) and set up the following environment variables: -- `WATSONX_URL`, the API url (set to `https://{region}.ml.cloud.ibm.com`) of your WatsonX instance +In order to run the examples that use foundation models hosted on [watsonx](https://www.ibm.com/watsonx) via LiteLLM, you need a watsonx account (a free plan is available) and set up the following environment variables: +- `WATSONX_URL`, the API url (set to `https://{region}.ml.cloud.ibm.com`) of your watsonx instance - `WATSONX_APIKEY`, the API key (see information on [key creation](https://cloud.ibm.com/docs/account?topic=account-userapikey&interface=ui#create_user_key)) - `WATSONX_PROJECT_ID`, the project hosting the resources (see information about [project creation](https://www.ibm.com/docs/en/watsonx/saas?topic=projects-creating-project) and [finding project ID](https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/fm-project-id.html?context=wx)). @@ -231,7 +231,7 @@ assign the result to variable `CODE`. Next we define a `text`, where the first block is simply a string and writes out the source code. This is done by accessing the variable `CODE`. The syntax `${ var }` means accessing the value of a variable in the scope. Since `CODE` contains YAML data, we can also access fields such as `CODE.source_code`. -The second block calls a granite model on WatsonX via LiteLLM. Here we explicitly provide an `input` field which means that we do not pass the entire text produced so far to the model, but only what is specified in this field. In this case, we specify our template by using the variable `CODE` as shown above. +The second block calls a granite model on watsonx via LiteLLM. Here we explicitly provide an `input` field which means that we do not pass the entire text produced so far to the model, but only what is specified in this field. In this case, we specify our template by using the variable `CODE` as shown above. When we execute this program with the PDL interpreter, we obtain the following text: @@ -401,7 +401,7 @@ This is similar to a spreadsheet for tabular data, where data is in the forefron ## Additional Notes -When using Granite models on Watsonx, we use the following defaults for model parameters (except `granite-20b-code-instruct-r1.1`): +When using Granite models on watsonx, we use the following defaults for model parameters (except `granite-20b-code-instruct-r1.1`): - `decoding_method`: `greedy` - `max_new_tokens`: 1024 - `min_new_tokens`: 1 @@ -417,4 +417,4 @@ For a complete list of issues see [here](https://github.com/IBM/prompt-declarati ## Contributing to the Project -See [Contributing to PDL](https://ibm.github.io/prompt-declaration-language/contrib) +See [Contributing to PDL](https://ibm.github.io/prompt-declaration-language/contrib). diff --git a/docs/README.md b/docs/README.md index 93f04842c..e673543e5 100644 --- a/docs/README.md +++ b/docs/README.md @@ -10,7 +10,7 @@ LLMs will continue to change the way we build software systems. They are not onl PDL is based on the premise that interactions between users, LLMs and rule-based systems form a *document*. Consider for example the interactions between a user and a chatbot. At each interaction, the exchanges form a document that gets longer and longer. Similarly, chaining models together or using tools for specific tasks result in outputs that together form a document. PDL allows users to specify the shape and contents of such documents in a declarative way (in YAML), and is agnostic of any programming language. Because of its document-oriented nature, it can be used to easily express a variety of data generation tasks (inference, data synthesis, data generation for model training, etc...). PDL provides the following features: -- Ability to use any LLM locally or remotely via [LiteLLM](https://www.litellm.ai/), including [IBM's Watsonx](https://www.ibm.com/watsonx) +- Ability to use any LLM locally or remotely via [LiteLLM](https://www.litellm.ai/), including [IBM's watsonx](https://www.ibm.com/watsonx) - Ability to templatize not only prompts for one LLM call, but also composition of LLMs with tools (code and APIs). Templates can encompass tasks of larger granularity than a single LLM call - Control structures: variable definitions and use, conditionals, loops, functions - Ability to read from files and stdin, including JSON data @@ -29,7 +29,7 @@ See below for a quick reference, followed by [installation notes](#interpreter_i PDL Quick Reference -(See also [PDF version](https://github.com/IBM/prompt-declaration-language/blob/main/docs/assets/pdl_quick_reference.pdf).] +(See also [PDF version](https://github.com/IBM/prompt-declaration-language/blob/main/docs/assets/pdl_quick_reference.pdf).) ## Interpreter Installation @@ -49,8 +49,8 @@ pip install 'prompt-declaration-language[examples]' pip install 'prompt-declaration-language[docs]' ``` -In order to run the examples that use foundation models hosted on [Watsonx](https://www.ibm.com/watsonx) via LiteLLM, you need a WatsonX account (a free plan is available) and set up the following environment variables: -- `WATSONX_URL`, the API url (set to `https://{region}.ml.cloud.ibm.com`) of your WatsonX instance +In order to run the examples that use foundation models hosted on [watsonx](https://www.ibm.com/watsonx) via LiteLLM, you need a watsonx account (a free plan is available) and set up the following environment variables: +- `WATSONX_URL`, the API url (set to `https://{region}.ml.cloud.ibm.com`) of your watsonx instance - `WATSONX_APIKEY`, the API key (see information on [key creation](https://cloud.ibm.com/docs/account?topic=account-userapikey&interface=ui#create_user_key)) - `WATSONX_PROJECT_ID`, the project hosting the resources (see information about [project creation](https://www.ibm.com/docs/en/watsonx/saas?topic=projects-creating-project) and [finding project ID](https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/fm-project-id.html?context=wx)). @@ -236,7 +236,7 @@ assign the result to variable `CODE`. Next we define a `text`, where the first block is simply a string and writes out the source code. This is done by accessing the variable `CODE`. The syntax `${ var }` means accessing the value of a variable in the scope. Since `CODE` contains YAML data, we can also access fields such as `CODE.source_code`. -The second block calls a granite model on WatsonX via LiteLLM. Here we explicitly provide an `input` field which means that we do not pass the entire text produced so far to the model, but only what is specified in this field. In this case, we specify our template by using the variable `CODE` as shown above. +The second block calls a granite model on watsonx via LiteLLM. Here we explicitly provide an `input` field which means that we do not pass the entire text produced so far to the model, but only what is specified in this field. In this case, we specify our template by using the variable `CODE` as shown above. When we execute this program with the PDL interpreter, we obtain the following text: @@ -406,7 +406,7 @@ This is similar to a spreadsheet for tabular data, where data is in the forefron ## Additional Notes -When using Granite models on Watsonx, we use the following defaults for model parameters (except `granite-20b-code-instruct-r1.1`): +When using Granite models on watsonx, we use the following defaults for model parameters (except `granite-20b-code-instruct-r1.1`): - `decoding_method`: `greedy` - `max_new_tokens`: 1024 - `min_new_tokens`: 1 @@ -422,4 +422,4 @@ For a complete list of issues see [here](https://github.com/IBM/prompt-declarati ## Contributing to the Project -See [Contributing to PDL](https://ibm.github.io/prompt-declaration-language/contrib) +See [Contributing to PDL](https://ibm.github.io/prompt-declaration-language/contrib). From 1c85e9d8ac081340b5947fc599ed13da1af01112 Mon Sep 17 00:00:00 2001 From: Louis Mandel Date: Wed, 16 Oct 2024 22:24:21 -0400 Subject: [PATCH 2/5] Include examples in the tutorial from actual source --- docs/tutorial.md | 4 +--- mkdocs.yml | 1 + 2 files changed, 2 insertions(+), 3 deletions(-) diff --git a/docs/tutorial.md b/docs/tutorial.md index 3d825a0f6..6820636b4 100644 --- a/docs/tutorial.md +++ b/docs/tutorial.md @@ -11,9 +11,7 @@ All the examples in this tutorial can be found in `examples/tutorial`. The simplest PDL program is one that generates a small text ([file](https://github.com/IBM/prompt-declaration-language/blob/main/examples/tutorial/simple_program.pdl)): ```yaml -description: Hello world! -text: - Hello, world! +--8<-- "./examples/tutorial/simple_program.pdl" ``` This program has a `description` field, which contains a title. The `description` field is optional. It also has a `text` field, which can be either a string, a *block*, or a list of strings and blocks. A block is a recipe for how to obtain data (e.g., model call, code call, etc...). In this case, there are no calls to an LLM or other tools, and `text` consists of a simple string. diff --git a/mkdocs.yml b/mkdocs.yml index 2e48c6faf..6ba1f9143 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -85,3 +85,4 @@ markdown_extensions: - name: mermaid class: mermaid format: !!python/name:pymdownx.superfences.fence_code_format + - pymdownx.snippets From 14091d581999e180811e58c39258a7529dbde4bd Mon Sep 17 00:00:00 2001 From: Louis Mandel Date: Thu, 17 Oct 2024 08:49:22 -0400 Subject: [PATCH 3/5] Update tutorial --- docs/tutorial.md | 38 +++++++------------------------------- 1 file changed, 7 insertions(+), 31 deletions(-) diff --git a/docs/tutorial.md b/docs/tutorial.md index 6820636b4..7e1b5faa0 100644 --- a/docs/tutorial.md +++ b/docs/tutorial.md @@ -33,19 +33,11 @@ Hello, world! ## Calling an LLM ```yaml -description: Hello world calling a model -text: -- Hello, -- model: watsonx/ibm/granite-34b-code-instruct - parameters: - decoding_method: greedy - stop: - - '!' - include_stop_sequence: true +--8<-- "./examples/tutorial/calling_llm.pdl" ``` -In this program ([file](https://github.com/IBM/prompt-declaration-language//blob/main/examples/tutorial/calling_llm.pdl)), the `text` starts with the word `Hello,`, and we call a model (`watsonx/ibm/granite-34b-code-instruct`) with this as input prompt. Notice the Watsonx model id on LiteLLM. -The model is passed some parameters including the `decoding_method` and `stop`, which corresponds to the `stop_sequences` parameter in Watsonx. The stop sequences are to be included in the output. +In this program ([file](https://github.com/IBM/prompt-declaration-language//blob/main/examples/tutorial/calling_llm.pdl)), the `text` starts with the word `Hello,`, and we call a model (`watsonx/ibm/granite-34b-code-instruct`) with this as input prompt. Notice the watsonx model id on LiteLLM. +The model is passed some parameters including the `decoding_method` and `stop`, which corresponds to the `stop_sequences` parameter in watsonx. The stop sequences are to be included in the output. A PDL program computes 2 data structures. The first is a JSON corresponding to the result of the overall program, obtained by aggregating the results of each block. This is what is printed by default when we run the interpreter. The second is a conversational background context, which is a list of role/content pairs, where we implicitly keep track of roles and content for the purpose of communicating with models that support chat APIs. The contents in the latter correspond to the results of each block. The conversational background context is what is used to make calls to LLMs via LiteLLM. @@ -62,12 +54,7 @@ where the portion ` world!` has been generated by Granite. Here's another of model call that includes an `input` field ([file](https://github.com/IBM/prompt-declaration-language//blob/main/examples/tutorial/calling_llm_with_input.pdl)): ```yaml -description: Hello world calling a model -text: -- "Hello," -- model: watsonx/ibm/granite-20b-multilingual - input: - Translate the word 'world' to French +--8<-- "./examples/tutorial/calling_llm_with_input.pdl" ``` In this case, we make a call to the granite multilingual model, and the input passed to the model is the sentence: `Translate the word 'world' to French` and nothing else from the surrounding document. When we execute this program, we obtain: @@ -78,9 +65,9 @@ Le mot 'world' en français est 'monde'. ``` where everything after the `:` including it were generated by the model. -### Parameter defaults for Watsonx Granite models +### Parameter defaults for watsonx Granite models -PDL provides the following defaults for Watsonx Granite models, when the following parameters are missing: +PDL provides the following defaults for watsonx Granite models, when the following parameters are missing: - `decoding_method`: `greedy` - `max_new_tokens`: 1024 - `min_new_tokens`: 1 @@ -128,18 +115,7 @@ GEN is equal to: world! In PDL, we can declaratively chain models together as in the following example ([file](https://github.com/IBM/prompt-declaration-language//blob/main/examples/tutorial/model_chaining.pdl)): ```yaml -description: Model chaining -text: -- Hello, -- model: watsonx/ibm/granite-34b-code-instruct - parameters: - stop: ["!"] - include_stop_sequence: true -- "\nTranslate this to French\n" -- model: watsonx/ibm/granite-20b-multilingual - parameters: - stop: ["!"] - include_stop_sequence: true +--8<-- "./examples/tutorial/model_chaining.pdl" ``` In this program, the first call is to a granite model to complete the sentence `Hello, world!`. The following block in the document prints out the sentence: `Translate this to French`. The final line of the program takes the entire document produced so far and passes it as input to the granite multilingual model. Notice that the input passed to this model is the document up to that point, represented as a conversation. This makes it easy to chain models together and continue building on previous interactions. From 5533a0ec8d157235a6e5ed7cb9e645584d6a4ee0 Mon Sep 17 00:00:00 2001 From: Louis Mandel Date: Thu, 17 Oct 2024 09:10:33 -0400 Subject: [PATCH 4/5] Add an example of messages --- docs/tutorial.md | 6 ++++++ examples/tutorial/calling_llm_with_input.pdl | 1 - examples/tutorial/calling_llm_with_input_messages.pdl | 10 ++++++++++ 3 files changed, 16 insertions(+), 1 deletion(-) create mode 100644 examples/tutorial/calling_llm_with_input_messages.pdl diff --git a/docs/tutorial.md b/docs/tutorial.md index 7e1b5faa0..ee6d1f7ac 100644 --- a/docs/tutorial.md +++ b/docs/tutorial.md @@ -65,6 +65,12 @@ Le mot 'world' en français est 'monde'. ``` where everything after the `:` including it were generated by the model. + +Using the `input` field, we can also give a directly an array of messages (`role`/`content`) to the model ([file](https://github.com/IBM/prompt-declaration-language//blob/main/examples/tutorial/calling_llm_with_input_messages.pdl)): +```yaml +--8<-- "./examples/tutorial/calling_llm_with_input_messages.pdl" +``` + ### Parameter defaults for watsonx Granite models PDL provides the following defaults for watsonx Granite models, when the following parameters are missing: diff --git a/examples/tutorial/calling_llm_with_input.pdl b/examples/tutorial/calling_llm_with_input.pdl index 83c9b19e2..f9134b9ed 100644 --- a/examples/tutorial/calling_llm_with_input.pdl +++ b/examples/tutorial/calling_llm_with_input.pdl @@ -4,4 +4,3 @@ text: - model: watsonx/ibm/granite-20b-multilingual input: Translate the word 'world' to French - \ No newline at end of file diff --git a/examples/tutorial/calling_llm_with_input_messages.pdl b/examples/tutorial/calling_llm_with_input_messages.pdl new file mode 100644 index 000000000..8fb02679c --- /dev/null +++ b/examples/tutorial/calling_llm_with_input_messages.pdl @@ -0,0 +1,10 @@ +description: Hello world calling a model +text: +- "Hello, " +- model: watsonx/ibm/granite-20b-multilingual + input: + array: + - role: system + content: You are a helpful assistant that is fluent in French. + - role: user + content: Translate the word 'world' to French From 49b138ea5a434ba26192e553e0810965b4ed06ee Mon Sep 17 00:00:00 2001 From: Louis Mandel Date: Thu, 17 Oct 2024 13:45:52 -0400 Subject: [PATCH 5/5] Update tutorial --- docs/tutorial.md | 14 ++------------ 1 file changed, 2 insertions(+), 12 deletions(-) diff --git a/docs/tutorial.md b/docs/tutorial.md index ee6d1f7ac..e9d15007d 100644 --- a/docs/tutorial.md +++ b/docs/tutorial.md @@ -90,22 +90,12 @@ The user can override these defaults by explicitly including them in the model c ## Variable Definition and Use -Any block can have a variable definition using a `def: ` field. This means that the output of that block is assigned to the variable ``, which may be reused at a later point in the document. +Any block can define a variable using a `def: ` field. This means that the output of that block is assigned to the variable ``, which may be reused at a later point in the document. Consider the following example ([file](https://github.com/IBM/prompt-declaration-language//blob/main/examples/tutorial/variable_def_use.pdl)): ```yaml -description: Hello world with variable def and use -text: -- Hello, -- model: watsonx/ibm/granite-34b-code-instruct - def: GEN - parameters: - decoding_method: greedy - stop: - - '!' - include_stop_sequence: true -- "\nGEN is equal to: ${ GEN }" +--8<-- "./examples/tutorial/variable_def_use.pdl" ``` Here we assign the output of the model to variable `GEN` using the `def` field. The last line of the program prints out the value of `GEN`. Notice the notation `${ }` for accessing the value of a variable.