Skip to content

Support suspendable feed with dynamic external functions for REPL mode #190

@DouweM

Description

@DouweM

Use case

In Pydantic AI's code execution toolset, the LLM gets a run_code tool that executes Python in Monty. Currently each call creates a fresh Monty instance — variables don't persist between calls. We want to use MontyRepl to let variables survive across run_code calls, so the LLM can build up state incrementally (fetch data in one call, process it in the next, etc).

This requires two things that MontyRepl doesn't currently support.

1. Expose suspendable feed to Python

Pydantic AI wraps other tools as callable Python functions inside Monty. When code calls an external function, Monty needs to pause, Pydantic AI executes the actual tool, and then resumes Monty with the result. This is the Monty.start()MontySnapshot.resume() loop we use today with the one-shot Monty class.

The Rust MontyRepl already has a suspendable start() method that returns ReplProgress (with Complete, FunctionCall, OsCall, ResolveFutures variants), and ReplSnapshot/ReplFutureSnapshot provide .run()/.resume() to continue execution. But the Python bindings only expose the synchronous feed(), which returns the result directly and has no way to pause at external function calls.

We need the suspendable variant exposed to Python, returning the same MontySnapshot | MontyFutureSnapshot | MontyComplete types that Monty.start() returns.

2. Dynamic external functions per feed

Currently MontyRepl fixes external_function_names at create() time. They're baked into the runtime:

  • External functions occupy the first N contiguous slots of the global namespace
  • Each gets an ExtFunctionId (0-based index into Interns)
  • Value::ExtFunction(id) values are stored in namespace slots
  • The field is private with no setter; feed()/start() always clone the same list

In Pydantic AI, the set of tools available to a run_code call can change between calls. The code execution toolset wraps an inner toolset whose get_tools(ctx) is called each model request iteration and can return different tools based on context (e.g. filtering by run step, message history, etc). So across the lifetime of a REPL session, the external function set can change.

We need a way to pass the current set of external functions (and ideally type-check stubs for them) at each feed call, so that:

  • Newly available functions can be called in code
  • Removed functions produce a clear error rather than silently referencing a stale binding
  • Type checking sees the current signatures

Proposed API

Something like:

class MontyRepl:
    # Existing simple feed (pure code, no external function support)
    def feed(self, code: str, *, print_callback=None) -> Any: ...

    # New: suspendable feed with per-call external functions
    def feed_start(
        self,
        code: str,
        *,
        external_functions: list[str] | None = None,  # current set, or None to keep previous
        type_check_stubs: str | None = None,           # current signatures for type checking
        print_callback=None,
        limits: ResourceLimits | None = None,
    ) -> MontySnapshot | MontyFutureSnapshot | MontyComplete: ...

The external_functions parameter would update the namespace before compiling/executing the snippet:

  • New names get Value::ExtFunction(id) entries allocated
  • Names no longer in the list get their namespace binding removed (or tombstoned)
  • Previously-seen names reuse their existing ExtFunctionId

Alternatively, a separate set_external_functions() method called before feed/feed_start would also work — whatever's cleaner on the Rust side.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions