Coordinating state between widgets

[JoshKarpel]

Hi! Fair warning, I'm trying to cram some existing mental models into Textual and I've never done any serious GUI/TUI programming, so feel free to "that doesn't make any sense, do this instead" me :)

I'm playing with an application that has some complex state that will be consumed by many different widgets across many different screens. The state can be updated by actions that might happen on different widgets across different screens. My initial thought was "store the state in one place, on the App", something like:

from dataclasses import dataclass

from rich.console import RenderableType
from rich.style import Style
from rich.text import Text
from textual.app import App, ComposeResult
from textual.binding import Binding
from textual.reactive import reactive
from textual.screen import Screen
from textual.widget import Widget
from textual.widgets import Static, Footer


@dataclass
class State:
    msg: str = "Hello!"
    style: Style = Style(color = "red")

    def __rich__(self):
        return self.msg

class DisplayWidget(Widget):
    def render(self) -> RenderableType:
        return Text(self.app.state.msg, style=self.app.state.style)


class ScreenA(Screen):
    def compose(self) -> ComposeResult:
        yield Static(Text("A"))
        yield DisplayWidget()
        yield Footer()

class ScreenB(Screen):
    def compose(self) -> ComposeResult:
        yield Static(Text("B"))
        yield DisplayWidget()
        yield Footer()


class StateApp(App):
    state = reactive(State)

    SCREENS = {
        "a": ScreenA(),
        "b": ScreenB()
    }

    BINDINGS = [
        Binding("a", "switch_screen('a')", "Screen A",),
        Binding("b", "switch_screen('b')", "Screen B",),
        Binding("s", "change_state", "Change State"),
    ]

    def action_change_state(self) -> None:
        self.state = State(msg = self.state.msg.swapcase(), style=Style(bold=not self.state.style.bold))
        self.log(f"{self.state}")

    def on_mount(self) -> None:
        self.switch_screen("a")

if __name__ == "__main__":
    StateApp().run()

Unsurprisingly, this does not work: state changes are not reflected in the screens/widgets because there's nothing happening that would trigger the update (no reactives or otherwise-forced refreshes). Moreover, it's not type-safe because Widget.app is an App, not a StateApp (though that could perhaps be fixed by throwing some generics around inside Textual, but it still wouldn't help with the updates).

Suppose I make DisplayWidget have a reactive state attribute, like

class DisplayWidget(Widget):
    state = reactive(State())

    def render(self) -> RenderableType:
        return Text(self.state.msg, style=self.state.style)

But now I've got a copy of the state that's not stored in the source of truth! So I need to make sure that all widgets have the same state by pumping it down to all widgets that use the state whenever it changes (or maybe I'm totally wrong and this is not the right path at all!):

class StateApp(App):
    ...

    def watch_state(self, new_state: State) -> None:
        for widget in self.query(None):
            if hasattr(widget, "state"):
                widget.state = new_state

(I want to pass the whole State down because (in the real project), State is complex and the widget needs many of its properties, helper methods, etc. to render itself; I feel that breaking it down into "just what this widget needs" would make the coordination problem I'm facing even worse.)

That works for the active Screen, but widgets on screens that haven't been laid out yet, or screens/widgets that might not yet exist, wouldn't know anything about this occurring in the past.

For example, if you run this app:

from dataclasses import dataclass

from rich.console import RenderableType
from rich.style import Style
from rich.text import Text
from textual.app import App, ComposeResult
from textual.binding import Binding
from textual.reactive import reactive
from textual.screen import Screen
from textual.widget import Widget
from textual.widgets import Static, Footer


@dataclass
class State:
    msg: str = "Hello!"
    style: Style = Style(color = "red")

    def __rich__(self):
        return self.msg

class DisplayWidget(Widget):
    state = reactive(State())

    def render(self) -> RenderableType:
        return Text(self.state.msg, style=self.state.style)


class ScreenA(Screen):
    def compose(self) -> ComposeResult:
        yield Static(Text("A"))
        yield DisplayWidget()
        yield Footer()

class ScreenB(Screen):
    def compose(self) -> ComposeResult:
        yield Static(Text("B"))
        yield DisplayWidget()
        yield Footer()


class StateApp(App):
    state = reactive(State)

    SCREENS = {
        "a": ScreenA(),
        "b": ScreenB()
    }

    BINDINGS = [
        Binding("a", "switch_screen('a')", "Screen A",),
        Binding("b", "switch_screen('b')", "Screen B",),
        Binding("s", "change_state", "Change State"),
    ]

    def action_change_state(self) -> None:
        self.state = State(msg = self.state.msg.swapcase(), style=Style(underline=not self.state.style.underline))
        self.log(f"{self.state}")

    def on_mount(self) -> None:
        self.switch_screen("a")

    def watch_state(self, new_state: State) -> None:
        for widget in self.query(None):
            if hasattr(widget, "state"): 
                self.log(f"Updating state on {widget}")
                widget.state = new_state


if __name__ == "__main__":
    StateApp().run()

and press s a few times, then change to screen b, the DisplayWidget on screen b will be created fresh and won't know anything about the previous updates, so it'll display the default style. From that point on changes will be correctly tracked everywhere because both DisplayWidgets now exist and the self.query(None) can find both of them.

I tried overriding various methods like App.switch_screen to also trigger watch_state, but it seems like that happens too early - the first render of the screen doesn't happen until after those methods return, so the screen's widgets aren't in the query and don't get updated. (I suppose that ties in with a more general question, which is: what is the recommended way to set reactive attributes on new instances, given that they aren't added to the __init__() method?)

So I think my question is: with this kind of multi-screen app where it seems like I need to coordinate changes in "global", or at least cross-screen/widget, state, what patterns should be I using to make sure that:

1. The current state is kept in sync everywhere and can be "pushed down" to new screens/widgets as they come into existence. Or perhaps I should not do this, and instead I should make sure that all screens/widgets and laid out before the state can change (in App.on_mount, I suppose)?
2. Widgets are refreshed when the state changes (I'm ok with assuming there's just one big blob of state and any change to any part of it should force a refresh of any component using the state; once I've got a pattern I can split the blob up as needed for performance).

I suspect this question is arising due to a clash of my instinct to do something more like immediate-mode and the retained-mode model that Textual is encouraging, which makes me think that I'm just missing something in the way I'm thinking about this problem.

Sorry for the brain dump, hope it makes some sense. Any advice/examples appreciated!

[davep]

Hey @JoshKarpel, thanks for the comprehensive question. I think I see a couple of main things you're highlighting here; one sort of small but which will inform the approach you take, the other quite big and important.

First off:

Moreover, it's not type-safe because Widget.app is an App, not a StateApp (though that could perhaps be fixed by throwing some generics around inside Textual, but it still wouldn't help with the updates)

This is something I've been seeing myself -- that red underline in my Emacs buffer -- and it's a conversation we've been having and will be giving some thought to. I imagine seeing type warnings is something that has been informing your decision as to if a particular approach is a good one or not? For the moment I'm afraid the only suggestion I have for how to approach this will involve a couple of type warnings around Widget.app.

Now to the issue of handling and reacting to global state: there is no "the Textual way" solution to this at the moment; "Reactive state abstraction" is a thing that is on the roadmap and it's something we very much want to address. The sort of feedback you're giving here is really helpful to deciding how we go about that.

Meanwhile, here's my quick little attempt at making your final example code work. WARNING: It does mean there will be a couple of type warnings from your type checker, presumably depending on how strict it is being (I like to run as strict as possible), and it does use an undocumented function from inside textual.reactive.

from dataclasses import dataclass

from rich.console import RenderableType
from rich.style import Style
from rich.text import Text
from textual.app import App, ComposeResult
from textual.binding import Binding
from textual.reactive import reactive, watch
from textual.screen import Screen
from textual.widget import Widget
from textual.widgets import Static, Footer


@dataclass
class State:
    msg: str = "Hello!"
    style: Style = Style(color = "red")

    def __rich__(self):
        return self.msg

class DisplayWidget(Widget):

    def state_changed(self, _):
        self.refresh()

    def on_mount(self):
        watch(self.app, "state", self.state_changed)

    def render(self) -> RenderableType:
        return Text(self.app.state.msg, style=self.app.state.style)


class ScreenA(Screen):
    def compose(self) -> ComposeResult:
        yield Static(Text("A"))
        yield DisplayWidget()
        yield Footer()

class ScreenB(Screen):
    def compose(self) -> ComposeResult:
        yield Static(Text("B"))
        yield DisplayWidget()
        yield Footer()


class StateApp(App[None]):
    state = reactive(State)

    SCREENS = {
        "a": ScreenA(),
        "b": ScreenB()
    }

    BINDINGS = [
        Binding("a", "switch_screen('a')", "Screen A",),
        Binding("b", "switch_screen('b')", "Screen B",),
        Binding("s", "change_state", "Change State"),
    ]

    def action_change_state(self) -> None:
        self.state = State(msg = self.state.msg.swapcase(), style=Style(underline=not self.state.style.underline))
        self.log(f"{self.state}")

    def on_mount(self) -> None:
        self.switch_screen("a")

if __name__ == "__main__":
    StateApp().run()

Long story short: I've used the on_mount of the DisplayWidget to use watch to wire up the DisplayWidget to the app's state. When you reassign state at the app level DisplayWidget.state_changed will be fired, which will call refresh, which will cause render to be called.

While not the most ideal approach, and likely not the final recommended approach for Textual apps, I think this might be a clean way to do things for now: wire a widget classes up to the parts of the app's global state that they care about, and have them react when things change.

Also, speaking to @willmcgugan about watch, it looks like it won't be going away any time soon, and either it'll end up being documented as something that can be used, or a similar mechanism will be exposed, allowing the same effect.

Does that help?

[danielcaballero88]

Hey @davep, it looks like this function now changed from watch to _watch, so I'm wondering if there was a decision to make it "internal" because there is now a better way to achieve the same result?

[davep]

I believe https://textual.textualize.io/api/dom_node/#textual.dom.DOMNode.watch is the preferred approach these days.

[JoshKarpel]

That is very helpful! textual.reactive.watch does seem like the missing piece: it lets me keep the state on the App (i.e., in one place) but still update things dynamically elsewhere. I'll run with that and see where I get to.

re: type-checking, yep, I'm also a "as strict as possible" kind of person, but I'm fine with throwing some ignores on things if need be. I do try to let it guide my approach to things so that I can avoid ignores as much as possible. Given that users will definitely be subclassing App, I think it could make sense for MessagePump to be generic like MessagePump[App].

[willmcgugan]

I don't think that would work for self.app.

Widgets would have to be generics and take App, but then you don't know what app they will be added to until runtime.

This could work:

app = self.app
assert isinstance(app, MyApp)

But its all that elegant.

We could have a function which returns the appropriate App subclass:

app = self.get_app(MyApp)

Or maybe something like this:

app = MyApp.get(self)

Open to suggestions...

[JoshKarpel]

I think you're right that there's not a way to make it happen purely in the type system in an enforceable way - there would need to be a usage of MyApp somewhere at runtime 🤔

If you're comfortable with not actually enforcing it, this two-minute hack makes mypy happy (but is also not particularly elegant):

class MyWidget(Widget):
    app: MyApp

class RealWidget(MyWidget): ...

Which I think is roughly equivalent to doing

A = TypeVar("A", bound=App)

class MessagePump(Generic[A], metaclass=MessagePumpMeta):
    @property
    def app(self) -> A: ...

inside Textual, but of course you don't have the actual type for A inside MessagePump.app so you can't enforce it.

I guess this ultimately goes back to the original question - a widget doesn't need to know what app it's attached to if it only depend on its own attributes, but then you have the state coordination problem. I guess I'm comfortable (at least for now) with having my custom widgets tied very tightly to my app, since I'm not immediately planning for them to be separately-usable.