Questions related to PanelFrontend contribution

[MarcSkovMadsen]

UPDATE: DISCUSSION MOVED TO #13531.

Context

I would like to contribute an intermediate example of a Panel app. I would like to contribute a PanelFrontend similar to the StreamlitFrontend or GradioFrontend. See #13335 for more background info.

POC

I've created the below POC. It consists of one flow that logs its state (i.e. the global app state) and two components. One CounterWork LightningWork that runs an infinite loop and increments the attribute counter every 5 seconds. One PanelApp LightningWork that runs a Panel server with a submit button that updates the submit attribute when clicked

import datetime
import logging
import sys
import time

import lightning as L
import lightning_app as lapp
import panel as pn
from lightning_app.utilities.app_helpers import pretty_state
from lightning_app.utilities.imports import requires

logger = logging.getLogger("panel")
logger.setLevel(logging.INFO)

handler = logging.StreamHandler(sys.stdout)
formatter = logging.Formatter('%(asctime)s - %(message)s')
handler.setFormatter(formatter)
logger.addHandler(handler)

class PanelWork(lapp.LightningWork):
    
    @requires("panel")
    def __init__(self, render_fn, parallel=True, **kwargs):
        super().__init__(parallel=parallel, **kwargs)
        self._render_fn = render_fn
        self._server = None
        self.requests = 0
        logger.info("init PanelWork")

    def _fast_initial_view(self):
        self.requests += 1
        logger.info("Incremented requests %s", self.requests)
        if self.requests == 1:
            return pn.pane.HTML("<h1>Please refresh the browser to see the app.</h1>")
        else:
            return self._render_fn(self)

    def run(self):
        logger.info("LitPanel run start")
        if not self._server:
            logger.info("LitPanel Starting Server")
            self._server = pn.serve(
                {"/": self._fast_initial_view},
                port=self.port,
                address=self.host,
                websocket_origin="*",
                show=False,
                # threaded=True,
            )
        logger.info("LitPanel run finish")

    def stop(self):
        """Stops the server"""
        if self._server:
            self._server.stop()
            logger.info("Panel server stopped")

def render_fn(self):
    submit = pn.widgets.Button(name="Submit")
    
    @pn.depends(submit, watch=True)
    def update(_):
        self.submits += 1
        logger.info("Incremented submits %s", self.submits)

    @pn.depends(submit)
    def submits(_):
        return self.submits

    @pn.depends(submit)
    def local_state(_):
        return self.state

    @pn.depends(submit)
    def global_state(_):
        "How do I get this?"
        return {"works": "How do I get the global app state?"}
    
    return pn.Column(
        submit,
        "Submits",
        submits,
        "Local State",
        pn.panel(local_state, height=300),
        "Global State",
        pn.panel(global_state, height=300),
    )

class PanelApp(PanelWork):
    def __init__(self):
        super().__init__(render_fn=render_fn)

        self.submits = 0

class CounterWork(L.LightningWork):
    def __init__(self, parallel=True, **kwargs):
        super().__init__(parallel=parallel, **kwargs)
        
        self.counter = 0

    def run(self):
        logger.info("Starting run")
        while True:
            time.sleep(5)
            self.counter += 1
            logger.info("Incremented counter %s", self.counter)

class Flow(L.LightningFlow):
    def __init__(self):
        super().__init__()
        self.w = CounterWork()
        self.lit_panel = PanelApp()
        self._last_update = datetime.datetime.now()

    def run(self):
        self.w.run()
        self.lit_panel.run()
        now = datetime.datetime.now()
        if (now-self._last_update).seconds>=5:
            state_str = f"Global state: {pretty_state(self.state)} \n"
            logger.info(state_str)
            self._last_update=now

    def configure_layout(self):
        tab1 = {"name": "Home", "content": self.lit_panel}
        return tab1

app = L.LightningApp(Flow())

pip install panel lightning
lightning run app script.py

My questions

1. How can the PanelApp access the global app state? I would like to display it in the app. (I can see the StreamlitFrontend can access an AppState. But to me it seems like a cumbersome workaround due to Streamlit specifics.)

• I would also like to be able to get the global app state on demand. For example every 1 secs to provide live updates.

3. How can I communicate the counter value on the CounterWork to the PanelApp? Should I add something like self.lit_panel.run(counter=self.w.counter) to the Flows run method? (I tried but it seems to slow down the app significantly).
4. Which class should a PanelFrontend inherit from? The Frontend would not be a good idea as I see it as it does not hold state. I.e. the submits attribute would not be a part of the global app state. Furthermore as I understand it, a Frontend cannot run on GPUs.
5. Instead of starting the server via pn.serve I could also do something more like the StreamlitFrontend. I.e. use subprocess.Popen to run panel serve some_script.py and use the AppState with a panel_plugin. Would that be a better approach?

[MarcSkovMadsen]

Here is the app running

panel-lightning-app-poc.mp4

[MarcSkovMadsen]

A workaround for sharing global state for display in the Panel app would be to save it. For example as a pickle or in a cache like diskcache.

[MarcSkovMadsen]

Lets move the discussion to #13531.

[tchaton]

Hey @MarcSkovMadsen

1. There are currently 2 ways to access AppStage from another process. Either by connecting to the queues or using the REST API. In StreamLit or Panel, I think it is better to start a subprocess and create an AppState object which makes a request to the Rest API exposing the state. This would also enable multi-tenancy in the future.

2. The AppState objects get the entire state. We are simply removing visibility over it using the affiliation.

3. In your example, changing the counter re-run the run method, which re-creates a server. This isn't a good pattern. Using AppState is better for communicating, especially in the cloud.

4. You are right, the frontend doesn't run on GPU, but I think panel fits quite well as a frontend.

5. I think it is a better way to integrate panel.

I started to work on the following example, but Panel doesn't seem to render. I guess I am doing something wrong.

import datetime
import sys
import time

import lightning as L
import lightning_app as lapp
import panel as pn
from lightning_app.core.flow import LightningFlow
from lightning_app.utilities.app_helpers import pretty_state
from lightning_app.utilities.imports import requires
from lightning.app.frontend.frontend import Frontend
from typing import Callable, Optional
from lightning.app.utilities.state import AppState
import subprocess
import os
import inspect

class PanelFrontend(Frontend):

    @requires("panel")
    def __init__(self, render_fn: Callable) -> None:
        super().__init__()

        if inspect.ismethod(render_fn):
            raise TypeError(
                "The `state` doesn't support `render_fn` being a method. Please, use a pure function."
            )

        self.render_fn = render_fn
        self._process: Optional[subprocess.Popen] = None

    def start_server(self, host: str, port: int) -> None:
        env = os.environ.copy()
        env["LIGHTNING_FLOW_NAME"] = self.flow.name
        env["LIGHTNING_RENDER_FUNCTION"] = self.render_fn.__name__
        env["LIGHTNING_RENDER_MODULE_FILE"] = inspect.getmodule(self.render_fn).__file__
        env["HOST"] = str(host)
        env["PORT"] = str(port)
        self._process = subprocess.Popen(
            [
                sys.executable,
                "-m",
                "panel",
                "serve",
                os.path.join(os.path.dirname(lapp.frontend.__file__), "root.lit_panel.py"),
                "--address",
                str(host),
                "--port",
                str(port),
                "--autoreload",
            ],
            env=env,
        )

    def stop_server(self) -> None:
        if self._process is None:
            raise RuntimeError("Server is not running. Call `StreamlitFrontend.start_server()` first.")
        self._process.kill()

class CounterWork(L.LightningWork):
    def __init__(self, parallel=True, **kwargs):
        super().__init__(parallel=parallel, **kwargs)
        self.counter = 0

    def run(self):
        while self.counter >= 0:
            time.sleep(5)
            self.counter += 1

class PanelApp(LightningFlow):
    def __init__(self):
        super().__init__()
        self.submits = 0

    def configure_layout(self):
        return PanelFrontend(render_fn=render_fn)

def render_fn(state: AppState):
    submit = pn.widgets.Button(name="Submit")

    @pn.depends(submit, watch=True)
    def update(_):
        state.submits += 1

    @pn.depends(submit)
    def submits(_):
        return state.submits

    @pn.depends(submit)
    def local_state(_):
        return state._state

    @pn.depends(submit)
    def global_state(_):
        "How do I get this?"
        return {"works": "How do I get the global app state?"}

    column = pn.Column(
        submit,
        "Submits",
        submits,
        "Local State",
        pn.panel(local_state, height=300),
        "Global State",
        pn.panel(global_state, height=300),
    )

    pn.serve(
        {"/": column},
        websocket_origin="*",
        host=os.getenv("HOST", None),
        port=int(os.getenv("PORT", None)),
        show=False,
    )

class Flow(L.LightningFlow):
    def __init__(self):
        super().__init__()
        self.w = CounterWork()
        self.lit_panel = PanelApp()
        self._last_update = datetime.datetime.now()

    def run(self):
        self.w.run()
        self.lit_panel.run()
        now = datetime.datetime.now()
        if (now - self._last_update).seconds>=5:
            print(f"Global state: {pretty_state(self.state)} \n")
            self._last_update=now

    def configure_layout(self):
        tab1 = {"name": "Home", "content": self.lit_panel}
        return tab1

app = L.LightningApp(Flow())

[tchaton]

"""This file gets run by streamlit, which we launch within Lightning.

From here, we will call the render function that the user provided in ``configure_layout``.
"""
import os
import pydoc
from typing import Callable, Union

from lightning_app.core.flow import LightningFlow
from lightning_app.utilities.app_helpers import StreamLitStatePlugin
from lightning_app.utilities.state import AppState

app_state = AppState()


def _get_render_fn_from_environment() -> Callable:
    render_fn_name = os.environ["LIGHTNING_RENDER_FUNCTION"]
    render_fn_module_file = os.environ["LIGHTNING_RENDER_MODULE_FILE"]
    module = pydoc.importfile(render_fn_module_file)
    return getattr(module, render_fn_name)


def _app_state_to_flow_scope(state: AppState, flow: Union[str, LightningFlow]) -> AppState:
    """Returns a new AppState with the scope reduced to the given flow, as if the given flow as the root."""
    flow_name = flow.name if isinstance(flow, LightningFlow) else flow
    flow_name_parts = flow_name.split(".")[1:]  # exclude root
    flow_state = state
    for part in flow_name_parts:
        flow_state = getattr(flow_state, part)
    return flow_state


def main():
    # Fetch the information of which flow attaches to this streamlit instance
    flow_state = _app_state_to_flow_scope(app_state, flow=os.environ["LIGHTNING_FLOW_NAME"])

    # Call the provided render function.
    # Pass it the state, scoped to the current flow.
    render_fn = _get_render_fn_from_environment()
    render_fn(flow_state)


if __name__ == "__main__":
    main()

[MarcSkovMadsen]

This one demonstrates the principle @tchaton

1. I would like to argue that a render_script is more natural than a render_func. At least to Panel users. Probably also to Streamlit users. Would it be ok to use a render_script instead of a render_func.
2. When I use panel serve name_of_script.py then the app is served at /name_of_script. Thus I had to name the script root.lit_panel.py. I can solve it on the Panel side by using a panel_base.py file that runs pn.serve({"root.lit_panel": name_of_script}, ....). But is there an easier way?

lightning-frontend.mp4

app.py

import datetime
import sys
import time

import lightning as L
import panel as pn
from lightning_app.core.flow import LightningFlow
from lightning_app.utilities.app_helpers import pretty_state
from lightning_app.utilities.imports import requires
from lightning.app.frontend.frontend import Frontend
from typing import Callable, Optional
from lightning.app.utilities.state import AppState
import subprocess
import os

class PanelFrontend(Frontend):

    @requires("panel")
    def __init__(self, render_script="root.lit_panel.py") -> None:
        super().__init__()

        self._render_script = render_script
        self._process: Optional[subprocess.Popen] = None

    def start_server(self, host: str, port: int) -> None:
        env = os.environ.copy()
        env["LIGHTNING_FLOW_NAME"] = self.flow.name
        env["HOST"] = str(host)
        env["PORT"] = str(port)
        self._process = subprocess.Popen(
            [
                sys.executable,
                "-m",
                "panel",
                "serve",
                self._render_script,
                "--address",
                str(host),
                "--port",
                str(port),
                "--autoreload",
            ],
            env=env,
        )

    def stop_server(self) -> None:
        if self._process is None:
            raise RuntimeError("Server is not running. Call `StreamlitFrontend.start_server()` first.")
        self._process.kill()

class CounterWork(L.LightningWork):
    def __init__(self, parallel=True, **kwargs):
        super().__init__(parallel=parallel, **kwargs)
        self.counter = 0

    def run(self):
        while self.counter >= 0:
            time.sleep(1)
            self.counter += 1

class PanelApp(LightningFlow):
    def __init__(self):
        super().__init__()
        self.submits = 0

    def configure_layout(self):
        return PanelFrontend()

class Flow(L.LightningFlow):
    def __init__(self):
        super().__init__()
        self.w = CounterWork()
        self.lit_panel = PanelApp()
        self._last_update = datetime.datetime.now()

    def run(self):
        self.w.run()
        self.lit_panel.run()
        now = datetime.datetime.now()
        if (now - self._last_update).seconds>=5:
            print(f"Global state: {pretty_state(self.state)} \n")
            self._last_update=now

    def configure_layout(self):
        tab1 = {"name": "Home", "content": self.lit_panel}
        return tab1

flow = Flow()
app = L.LightningApp(flow)

root.lit_panel.py

import panel as pn
from lightning.app.utilities.state import AppState
import datetime as dt

from lightning_app.utilities.app_helpers import pretty_state


if "initial_request" not in pn.state.cache:
    pn.state.cache["initial_request"]=False
    
    pn.pane.Markdown("Fast intial response").servable()
else:
    pn.extension(sizing_mode="stretch_width")

    app_state = AppState()

    last_update = pn.pane.Markdown().servable()
    json_pane = pn.pane.JSON(theme="light", depth=4).servable()

    def update():
        last_update.object = str(dt.datetime.now().isoformat())
        app_state._state = None
        app_state._request_state()
        state = pretty_state(app_state._state)
        json_pane.object = state
    
    update()
    pn.state.add_periodic_callback(update, period=1000)

lightning run app app.py