feat: new context, new sampling,. #166

HendrikStrobelt · 2025-09-25T00:12:01Z

No description provided.

Context -> LegacyContext

*Context -> Legacy*Context

without log mechanics...

mergify · 2025-09-25T00:12:36Z

Merge Protections

Your pull request matches the following merge protections and will not be merged until they are valid.

🟢 Enforce conventional commit

Wonderful, this rule succeeded.

Make sure that we follow https://www.conventionalcommits.org/en/v1.0.0/

title ~= ^(fix|feat|docs|style|refactor|perf|test|build|ci|chore|revert)(?:\(.+\))?:

jakelorocco · 2025-09-25T12:58:36Z

mellea/stdlib/base.py

+        self._previous = RootContext()
+        self._data = None


I don't know how I feel about having a specific type of RootContext. I think if self._previous is None, you can assume it's the root of a context. This also lets you abruptly truncate contexts, ie you can detach a child from its parent just by setting self._previous = None.

If we want to store context level things in a root node (like common docs, common vars, etc...), then I think it potentially makes sense to have a specific root node class that handles those things. But I haven't really thought out those design decisions yet.

I'm fine with using "None". I was just inspired by other libraries which also use own keywords instead of reserved keywords for neutral values like NoValue,....

Using RootContext would avoid adding if conditionals at every possible occasion that it could occur, because calling any method on None raises an error. RootContext can have its methods that act transparently and thus is superior.

For example, RootContext.linearize would just return self.

jakelorocco · 2025-09-25T13:04:21Z

mellea/stdlib/base.py

+    def from_previous(cls, previous: Context, data: Component | CBlock) -> Context:
+        """Constructs a new context from an existing context."""
+
+        assert isinstance(previous, Context), (
+            "Cannot create a new context from a non-Context object."
+        )
+        assert data is not None, "Cannot create a new context from None data."
+
+        # if previous is an empty root context, create a new root context with the data.
+        if previous._data is None and isinstance(previous._previous, RootContext):
+            x = cls()
+            x._previous = RootContext()
+            x._data = data
+            return x
+        else:
+            x = cls()
+            x._previous = previous
+            x._data = data
+            return x


I think this logic is a bit weird. It's basically saying the current context node is the root if it has no data and a previous pointer to a RootContext. I think if we have a RootContext object, all context nodes under it should have data.

This also causes a weird issue with extend / from_previous. I think the assumption is that expanding a context should always add a new node to the existing context tree. For example,

ctx = Context() # node1(empty) ctx2 = ctx.expand(CBlock("hello")) # node1(empty) -> node2("hello")

Instead what we currently get is something like:

ctx = Context() # RootContext -> node1(empty) ctx2 = ctx.expand(CBlock("Hello")) # RootContext -> node2("hello")

If you now spontaneously disconnect, it's pretty inconsistent imho:

ctx = Context() # node1(empty) ctx2 = ctx.expand(CBlock("hello")) # node1(empty) -> node2("hello") ctx3 = ctx.expand(CBlock("world")) # node1(empty) -> node2("hello") -> node3("world")

disconnect at 2 leads to:

node2("hello") -> node3("world")

which is different in schema than ctx2

jakelorocco · 2025-09-25T13:06:29Z

mellea/stdlib/base.py

+        return []
+
+
+class LegacyContext(abc.ABC):


Is there a reason to keep around the old contexts? I feel like we might almost want to keep the naming of the old contexts, make sure the user facing interfaces still work (like get_last_prompt), and basically pretend like nothing happened. That way it's not a breaking change and people likely don't need to change their code.

The idea here is to keep everything running and piece by piece replacing LegacyContext with Context, but do it carefully and intentionally. Latest at the last commit in this PR there should be no trace of Legacy* anymore.

jakelorocco · 2025-09-25T13:10:50Z

mellea/stdlib/base.py

+    """A `RootContext` is a special type of `Context` that is empty and is used to represent the root context of a `MelleaSession`."""
+
+
 class Context(abc.ABC):


Making a note here in case we want to add the children field as well:

Appending to a list in python should be atomic, so we wouldn't need a lock for that (since we don't care about order either)

Removing objects from a list is not, but I don't think we actually need to do that

HendrikStrobelt · 2025-09-25T14:34:47Z

mellea/stdlib/base.py

+            return x
+
+    @abc.abstractmethod
+    def expand(self, c: Component | CBlock) -> Context:


Session modifications left open.. (unsafe commit)

removing chat, instruct,.. from mellea/__init__.py

…sessions

nrfulton · 2025-09-26T14:23:08Z

mellea/stdlib/base.py

+        """Returns the data associated with this context."""
+        return self._data
+
+    def full_data_as_list(self) -> list[Component | CBlock]:


This comment is for documentation of decision making.

@jakelorocco comments as_list is a better name because full or complete entails that you are getting the entire tree structure (which you are not). I agree.

But @HendrikStrobelt is giving a more verbose function name here because this function does NOT respect the view.

Aside: Nathan's Proposal

Nathan's Proposal: we separate the functionality of a Context from the functionality of a ContextView on that context.

The protocol for Context contains:

Context: add : Context * Component -> Context data_as_list : Context -> List[Component | CBlock] data : Context -> Component | CBlock | None

The protocol for ContextView contains:

ContextView: view : Context -> List[Component | CBlock]?

We then still have the same ergonomics:

class LinearContext(Context, ContextView)

But implementation of new views are forced to implement their functionality in terms of the Context interface.

Both Hendrik and Jake disagree with Nathan's Context / ContextView proposal :)

TODOs

Change this function name to as_list : Context * int -> list[Component | CBlock], and add a prominent comment explaining that the view constraints are NOT respected by this function.

ctx.as_list(last_n_components..) ctx.render_for_generation() --> ctx.view_for_generation()

2) making rejection-sampling default

- all: convert to new cotext - contextual_session test is commented out with TODO - session_ctx test is removed because logging changed.

HendrikStrobelt · 2025-09-29T14:33:36Z

approved by slack

HendrikStrobelt added 3 commits September 24, 2025 17:30

non-breaking:

d5c837a

Context -> LegacyContext

non-breaking:

1917b40

*Context -> Legacy*Context

Context, SimpleContext, ChatContext (nee:LinearContext)

484ccf4

without log mechanics...

jakelorocco reviewed Sep 25, 2025

View reviewed changes

HendrikStrobelt commented Sep 25, 2025

View reviewed changes

HendrikStrobelt and others added 13 commits September 25, 2025 11:59

expand -> add

a14b422

remove RootContext in favor of _is_root attribute.

2df9fbe

.

2fe2e79

actions_for_available_tools is back

89428c9

all backends adjusted to new context structure.

6359b41

Session modifications left open.. (unsafe commit)

feat: add top level funcs and start sample changes

e64b51b

context adjustments and tests

f3e9137

maybe breaking:

c9ec7df

removing chat, instruct,.. from mellea/__init__.py

untested session and sampling changes

3646edf

backends should add ACTION and MOT.

c2b3c47

as_chat_history adapted

a47dae8

fix: ollama client instantiate to prevent stale event loops in httpx …

849a7ed

…sessions

fix: mypy issues

c3854f4

nrfulton reviewed Sep 26, 2025

View reviewed changes

HendrikStrobelt and others added 10 commits September 26, 2025 11:48

Renaming:

0754a5d

ctx.as_list(last_n_components..) ctx.render_for_generation() --> ctx.view_for_generation()

name cleanup

590a285

adding contexts to sampling

e494305

1) handling sampling results and context

d0f6633

2) making rejection-sampling default

adding warning

5e005bd

name clarity

b94809b

remove unnecessary function

9a2efbf

updated tests:

507d10c

- all: convert to new cotext - contextual_session test is commented out with TODO - session_ctx test is removed because logging changed.

fix: watsonx errors

ede690e

fix: sampling and vision tests

4afa7cf

HendrikStrobelt added 6 commits September 26, 2025 16:21

examples react - error to be looked into by @jal

09a34b6

fixed react - thanks to @jal

3ca56a3

examples for alora (untested) and gen slots (tested)

597332c

more examples updated

6a137f4

more examples updated

5b11c5c

remove old context

81446e9

HendrikStrobelt marked this pull request as ready for review September 26, 2025 21:39

HendrikStrobelt requested review from jakelorocco and nrfulton September 26, 2025 21:39

jakelorocco added 4 commits September 27, 2025 17:52

fix: best of n and misc changes

f483afa

fix: improve runtime of best of n sampling

09a50b2

fix: add tests

5fbf63e

fix: remove references to LinearContext

d1c7fb5

HendrikStrobelt merged commit 4ae6d7c into main Sep 29, 2025
4 checks passed

HendrikStrobelt deleted the all/m_v7 branch September 29, 2025 14:34

		"""A `RootContext` is a special type of `Context` that is empty and is used to represent the root context of a `MelleaSession`."""


		class Context(abc.ABC):

feat: new context, new sampling,. #166

feat: new context, new sampling,. #166

Uh oh!

Conversation

HendrikStrobelt commented Sep 25, 2025

Uh oh!

mergify bot commented Sep 25, 2025

Merge Protections

🟢 Enforce conventional commit

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Aside: Nathan's Proposal

TODOs

Uh oh!

HendrikStrobelt commented Sep 29, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants