Merge branch 'main' into pdl-199

Author: mandel

.github/workflows/build.yml

@@ -10,7 +10,7 @@ jobs:
     strategy:
       fail-fast: false
       matrix:
-        python-version: ['3.11', '3.12']
+        python-version: ['3.11', '3.12', '3.13']
 
     steps:
     - uses: actions/checkout@v4

.github/workflows/run-examples.yml

@@ -13,7 +13,7 @@ jobs:
     strategy:
       fail-fast: false
       matrix:
-        python-version: ['3.11', '3.12']
+        python-version: ['3.11', '3.12', '3.13']
 
     steps:
     - uses: actions/checkout@v4

.pre-commit-config.yaml

@@ -45,6 +45,12 @@ repos:
             "-sn", # Don't display the score
             "--rcfile=pylintrc"
           ]
+  # run the bandit security linter
+  - repo: https://github.com/PyCQA/bandit
+    rev: 1.7.10
+    hooks:
+      - id: bandit
+        args: [-c, bandit.yaml]
   # Type check the Python code with MyPy
   - repo: https://github.com/pre-commit/mirrors-mypy
     rev: 'v1.11.2'

README.md

@@ -0,0 +1,9 @@
+---
+# This is the configuration file for the bandit python static analysis tool
+exclude_dirs:
+  - build
+# We are less worried about tests, as they are not a part of the library meant to be used by users
+# with untrusted inputs.
+  - test
+skips:
+  - B101 # allow the use of assert

bandit.yaml

@@ -35,7 +35,7 @@ Pro Tip: When writing loops and conditionals with `repeat`, `for`, and `if-then-
 
 ## Interpreter Installation
 
-The interpreter has been tested with Python versions **3.11** and **3.12**, on macOS and Linux. For Windows, please use WSL.
+The interpreter has been tested with Python versions **3.11**, **3.12**, and **3.13**, on macOS and Linux. For Windows, please use WSL.
 
 To install the requirements for `pdl`, execute the command:
 
@@ -438,6 +438,28 @@ in the right pane.
 
 This is similar to a spreadsheet for tabular data, where data is in the forefront and the user can inspect the formula that generates the data in each cell. In the Live Document, cells are not uniform but can take arbitrary extents. Clicking on them similarly reveals the part of the code that produced them.
 
+## Best Practices
+
+1. **Template Organization**:
+   - Keep templates modular and reusable
+   - Use variables for dynamic content
+   - Document template purpose and requirements
+
+2. **Error Handling**:
+   - Validate model inputs/outputs
+   - Include fallback logic
+   - Log intermediate results
+
+3. **Performance**:
+   - Cache frequent LLM calls
+   - Use appropriate temperature settings
+   - Implement retry logic for API calls
+
+4. **Security**:
+   - Enabling sandbox mode for untrusted code
+   - Validate all inputs
+   - Follow API key best practices
+
 
 ## Additional Notes
 

docs/README.md

@@ -4,16 +4,19 @@ You can report issues or open a pull request (PR) to suggest changes.
 
 ## Reporting an issue
 
-To report an issue, or to suggest an idea for a change that you haven't
-had time to write-up yet:
-1.  [Review existing issues](https://github.com/IBM/prompt-declaration-language/issues) to see if a similar issue has been opened or discussed.
-2.  [Open an
-issue](https://github.com/IBM/prompt-declaration-language/issues/new). Be sure to include any helpful information, such as your Kubernetes environment details, error messages, or logs that you might have.
+To report an issue, or to suggest an idea for a change that you haven't had time to write-up yet:
 
+1. [Review existing issues](https://github.com/IBM/prompt-declaration-language/issues) to see if a similar issue has been opened or discussed.
+
+2. [Open an
+issue](https://github.com/IBM/prompt-declaration-language/issues/new). Be sure to include any helpful information, such as version used, error messages, or logs that you might have.
+
+
+To report vulnerabilities privately, you can contact the authors by email (see [pyproject.toml](https://github.com/IBM/prompt-declaration-language/blob/main/pyproject.toml)).
 
 ## Suggesting a change
 
-To suggest a change to this repository, [submit a pull request](https://github.com/IBM/prompt-declaration-language/pulls) with the complete set of changes that you want to suggest. Before creating a PR, make sure that your changes pass all of the tests.
+To suggest a change to this repository, [submit a pull request](https://github.com/IBM/prompt-declaration-language/pulls) with the complete set of changes that you want to suggest. Before creating a PR, make sure that your changes pass all of the tests and add tests for new features.
 
 The test suite can be executed with the following command in the top-level folder:
 ```

docs/contrib.md

@@ -305,7 +305,7 @@ Bob lives at the following address:
 
 ##  Calling code
 
-The following script shows how to execute python code ([file](https://github.com/IBM/prompt-declaration-language//blob/main/examples/tutorial/calling_code.pdl)). The python code is executed locally (or in a containerized way if using `pdl --sandbox`).  In principle, PDL is agnostic of any specific programming language, but we currently only support Python. Variables defined in PDL are copied into the global scope of the Python code, so those variables can be used directly in the code. However, mutating variables in Python has no effect on the variables in the PDL program. The result of the code must be assigned to the variable `result` internally to be propagated to the result of the block. A variable `def` on the code block will then be set to this result.
+The following script shows how to execute python code ([file](https://github.com/IBM/prompt-declaration-language//blob/main/examples/tutorial/calling_code.pdl)). The python code is executed locally (or in a containerized way if using `pdl --sandbox`).  In principle, PDL is agnostic of any specific programming language, but we currently only support Python, Jinja, and shell commands. Variables defined in PDL are copied into the global scope of the Python code, so those variables can be used directly in the code. However, mutating variables in Python has no effect on the variables in the PDL program. The result of the code must be assigned to the variable `result` internally to be propagated to the result of the block. A variable `def` on the code block will then be set to this result.
 
 In order to define variables that are carried over to the next Python code block, a special variable `PDL_SESSION` can be used, and
 variables assigned to it as fields.
@@ -668,6 +668,7 @@ text:
     decoding_method: greedy
 ```
 
+If you want to use an external Ollama instance, the env variable `OLLAMA_API_BASE` should be defined, by default is `http://localhost:11434`.
 
 Alternatively, one could also use Ollama's OpenAI-style endpoint using the `openai/` prefix instead of `ollama_chat/`. In this case, set the `OPENAI_API_BASE`, `OPENAI_API_KEY`, and `OPENAI_ORGANIZATION` (if necessary) environment variables. If you were using the official OpenAI API, you would only have to set the api key and possibly the organization. For local use e.g., using Ollama, this could look like so:
 

docs/tutorial.md

@@ -0,0 +1,4 @@
+description: Hello world showing call out to shell command
+lang: command
+code: |
+  echo "Hello World!"

examples/hello/hello-code-command.pdl

@@ -0,0 +1,6 @@
+description: Hello world showing call out to Jinja
+defs:
+  world: "World"
+lang: jinja
+code: |
+  Hello {{ world }}!