Skip to content

Conversation

@jakelorocco
Copy link
Contributor

Generative slot tests and examples all passed / ran.

Closes #190 and #216

Features:

  • added context/backend version of the function call for generative slots
  • fixed function signature when output as text for the llm; used to have additional parens like def test((one: int, two: bool, three:str) -> int:)
  • fixed string param representation; now string args show as escaped:
    previous: Arguments:... - three: (type: <class 'str'>)
    now: Arguments:... - three: "" (type: <class 'str'>)
  • add post conditions (requirements) to the docstring (note these only get checked if you pass in a SamplingStrategy)
  • you can use sampling strategies but cannot get back a sampling result
  • added preconditions
    • does a basic validation call
    • if preconditions fail, an exception is raised with the validation results as a field
  • added examples / tests for these new functionalities

Semi-related changes and open questions:

  • for this PR, I modified our sampling strategies to not pass a format directive to validate
    • open question: should we even allow passing a format argument to validate? This seems like a major footgun that would mess up basically every form of requirement validation and is unexpected.
    • open question: Requirement validation goes off last output; I think this is unexpected. It also means that requriement validation errors if you don't pass a ModelOutputThunk as the output parameter.

Examples of the templated output:

  • precondition validation prompt:
Please check if the following arguments satisfy the precondition.
Reply with 'yes' if the precondition is satisfied and 'no' otherwise.
Do not include any other text in your response.

Arguments:
- one: 1  (type: <class 'int'>)
- two: False  (type: <class 'bool'>)
- three: "hello"  (type: <class 'str'>)

Precondition: args should all be numbers
  • with postconditions:
Your task is to imitate the output of the following function for the given arguments.
Reply Nothing else but the output of the function.

Function:
def listify(one, two, three) -> list[int]:
    """Create a list from the parameters.

    Postconditions:
        - the list should be sorted
        - each item should be a number between 0 and 10
    """

Arguments:
- one: 1  (type: <class 'int'>)
- two: False  (type: <class 'bool'>)
- three: "hello"  (type: <class 'str'>)

@mergify
Copy link

mergify bot commented Nov 7, 2025

Merge Protections

Your pull request matches the following merge protections and will not be merged until they are valid.

🟢 Enforce conventional commit

Wonderful, this rule succeeded.

Make sure that we follow https://www.conventionalcommits.org/en/v1.0.0/

  • title ~= ^(fix|feat|docs|style|refactor|perf|test|build|ci|chore|revert|release)(?:\(.+\))?:

@jakelorocco jakelorocco marked this pull request as ready for review November 8, 2025 03:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

make the requirement that parameter names are needed when using generative slots more explicit in the documentation

2 participants