Worker API (#2182)

* worker class

* worker API tests

* tidy

* Decorator and more tests

* type fix

* error order

* more tests

* remove active message

* move worker manager to app

* cancel nodes

* typing fix

* revert change

* typing fixes and cleanup

* revert typing

* test fix

* cancel group

* Added test for worker

* comment

* workers docs

* Added exit_on_error

* changelog

* svg

* refactor test

* remove debug tweaks

* docstrings

* worker test

* fix typing in run

* fix 3.7 tests

* blog post

* fix deadlock test

* words

* words

* words

* workers docs

* blog post

* Apply suggestions from code review

Co-authored-by: Dave Pearson <[email protected]>

* docstring

* fix and docstring

* Apply suggestions from code review

Co-authored-by: Rodrigo Girão Serrão <[email protected]>

* Update src/textual/widgets/_markdown.py

Co-authored-by: Rodrigo Girão Serrão <[email protected]>

* Apply suggestions from code review

Co-authored-by: Rodrigo Girão Serrão <[email protected]>

* Update src/textual/worker.py

Co-authored-by: Rodrigo Girão Serrão <[email protected]>

* Fix black

* docstring

* merge

* changelog

---------

Co-authored-by: Dave Pearson <[email protected]>
Co-authored-by: Rodrigo Girão Serrão <[email protected]>

Author: 3 people

CHANGELOG.md

@@ -5,7 +5,15 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](http://keepachangelog.com/)
 and this project adheres to [Semantic Versioning](http://semver.org/).
 
-## Unreleased
+## [0.18.0] - 2023-04-04
+
+### Added
+
+- Added Worker API https://github.com/Textualize/textual/pull/2182
+
+### Changed
+
+- Markdown.update is no longer a coroutine https://github.com/Textualize/textual/pull/2182
 
 ### [Fixed]
 
@@ -705,6 +713,7 @@ https://textual.textualize.io/blog/2022/11/08/version-040/#version-040
 - New handler system for messages that doesn't require inheritance
 - Improved traceback handling
 
+[0.18.0]: https://github.com/Textualize/textual/compare/v0.17.4...v0.18.0
 [0.17.3]: https://github.com/Textualize/textual/compare/v0.17.2...v0.17.3
 [0.17.2]: https://github.com/Textualize/textual/compare/v0.17.1...v0.17.2
 [0.17.1]: https://github.com/Textualize/textual/compare/v0.17.0...v0.17.1

docs/api/worker.md

@@ -0,0 +1 @@
+::: textual.worker

docs/api/worker_manager.md

@@ -0,0 +1 @@
+::: textual._worker_manager

docs/blog/posts/release0-18-0.md

@@ -0,0 +1,41 @@
+---
+draft: false
+date: 2023-04-04
+categories:
+  - Release
+title: "Textual 0.18.0 adds API for managing concurrent workers"
+authors:
+  - willmcgugan
+---
+
+# Textual 0.18.0 adds API for managing concurrent workers
+
+Less than a week since the last release, and we have a new API to show you.
+
+<!-- more -->
+
+This release adds a new [Worker API](../../guide/workers.md) designed to manage concurrency, both asyncio tasks and threads.
+
+An API to manage concurrency may seem like a strange addition to a library for building user interfaces, but on reflection it makes a lot of sense.
+People are building Textual apps to interface with REST APIs, websockets, and processes; and they are running into predictable issues.
+These aren't specifically Textual problems, but rather general problems related to async tasks and threads.
+It's not enough for us to point users at the asyncio docs, we needed a better answer.
+
+The new `run_worker` method provides an easy way of launching "Workers" (a wrapper over async tasks and threads) which also manages their lifetime.
+
+One of the challenges I've found with tasks and threads is ensuring that they are shut down in an orderly manner. Interestingly enough, Textual already implemented an orderly shutdown procedure to close the tasks that power widgets: children are shut down before parents, all the way up to the App (the root node).
+The new API piggybacks on to that existing mechanism to ensure that worker tasks are also shut down in the same order.
+
+!!! tip
+
+    You won't need to worry about this [gnarly issue](https://textual.textualize.io/blog/2023/02/11/the-heisenbug-lurking-in-your-async-code/) with the new Worker API.
+
+
+I'm particularly pleased with the new `@work` decorator which can turn a coroutine OR a regular function into a Textual Worker object, by scheduling it as either an asyncio task or a thread.
+I suspect this will solve 90% of the concurrency issues we see with Textual apps.
+
+See the [Worker API](../../guide/workers.md) for the details.
+
+## Join us
+
+If you want to talk about this update or anything else Textual related, join us on our [Discord server](https://discord.gg/Enf6Z3qhVr).

docs/examples/guide/workers/weather.css

@@ -0,0 +1,16 @@
+Input {
+    dock: top;
+    width: 100%;
+}
+
+#weather-container {
+    width: 100%;
+    height: 1fr;
+    align: center middle;
+    overflow: auto;
+}
+
+#weather {
+    width: auto;
+    height: auto;
+}

docs/examples/guide/workers/weather01.py

@@ -0,0 +1,40 @@
+import httpx
+from rich.text import Text
+
+from textual.app import App, ComposeResult
+from textual.containers import VerticalScroll
+from textual.widgets import Input, Static
+
+
+class WeatherApp(App):
+    """App to display the current weather."""
+
+    CSS_PATH = "weather.css"
+
+    def compose(self) -> ComposeResult:
+        yield Input(placeholder="Enter a City")
+        with VerticalScroll(id="weather-container"):
+            yield Static(id="weather")
+
+    async def on_input_changed(self, message: Input.Changed) -> None:
+        """Called when the input changes"""
+        await self.update_weather(message.value)
+
+    async def update_weather(self, city: str) -> None:
+        """Update the weather for the given city."""
+        weather_widget = self.query_one("#weather", Static)
+        if city:
+            # Query the network API
+            url = f"https://wttr.in/{city}"
+            async with httpx.AsyncClient() as client:
+                response = await client.get(url)
+                weather = Text.from_ansi(response.text)
+                weather_widget.update(weather)
+        else:
+            # No city, so just blank out the weather
+            weather_widget.update("")
+
+
+if __name__ == "__main__":
+    app = WeatherApp()
+    app.run()

docs/examples/guide/workers/weather02.py

@@ -0,0 +1,40 @@
+import httpx
+from rich.text import Text
+
+from textual.app import App, ComposeResult
+from textual.containers import VerticalScroll
+from textual.widgets import Input, Static
+
+
+class WeatherApp(App):
+    """App to display the current weather."""
+
+    CSS_PATH = "weather.css"
+
+    def compose(self) -> ComposeResult:
+        yield Input(placeholder="Enter a City")
+        with VerticalScroll(id="weather-container"):
+            yield Static(id="weather")
+
+    async def on_input_changed(self, message: Input.Changed) -> None:
+        """Called when the input changes"""
+        self.run_worker(self.update_weather(message.value), exclusive=True)
+
+    async def update_weather(self, city: str) -> None:
+        """Update the weather for the given city."""
+        weather_widget = self.query_one("#weather", Static)
+        if city:
+            # Query the network API
+            url = f"https://wttr.in/{city}"
+            async with httpx.AsyncClient() as client:
+                response = await client.get(url)
+                weather = Text.from_ansi(response.text)
+                weather_widget.update(weather)
+        else:
+            # No city, so just blank out the weather
+            weather_widget.update("")
+
+
+if __name__ == "__main__":
+    app = WeatherApp()
+    app.run()

docs/examples/guide/workers/weather03.py

@@ -0,0 +1,42 @@
+import httpx
+from rich.text import Text
+
+from textual import work
+from textual.app import App, ComposeResult
+from textual.containers import VerticalScroll
+from textual.widgets import Input, Static
+
+
+class WeatherApp(App):
+    """App to display the current weather."""
+
+    CSS_PATH = "weather.css"
+
+    def compose(self) -> ComposeResult:
+        yield Input(placeholder="Enter a City")
+        with VerticalScroll(id="weather-container"):
+            yield Static(id="weather")
+
+    async def on_input_changed(self, message: Input.Changed) -> None:
+        """Called when the input changes"""
+        self.update_weather(message.value)
+
+    @work(exclusive=True)
+    async def update_weather(self, city: str) -> None:
+        """Update the weather for the given city."""
+        weather_widget = self.query_one("#weather", Static)
+        if city:
+            # Query the network API
+            url = f"https://wttr.in/{city}"
+            async with httpx.AsyncClient() as client:
+                response = await client.get(url)
+                weather = Text.from_ansi(response.text)
+                weather_widget.update(weather)
+        else:
+            # No city, so just blank out the weather
+            weather_widget.update("")
+
+
+if __name__ == "__main__":
+    app = WeatherApp()
+    app.run()

docs/examples/guide/workers/weather04.py

@@ -0,0 +1,47 @@
+import httpx
+from rich.text import Text
+
+from textual import work
+from textual.app import App, ComposeResult
+from textual.containers import VerticalScroll
+from textual.widgets import Input, Static
+from textual.worker import Worker
+
+
+class WeatherApp(App):
+    """App to display the current weather."""
+
+    CSS_PATH = "weather.css"
+
+    def compose(self) -> ComposeResult:
+        yield Input(placeholder="Enter a City")
+        with VerticalScroll(id="weather-container"):
+            yield Static(id="weather")
+
+    async def on_input_changed(self, message: Input.Changed) -> None:
+        """Called when the input changes"""
+        self.update_weather(message.value)
+
+    @work(exclusive=True)
+    async def update_weather(self, city: str) -> None:
+        """Update the weather for the given city."""
+        weather_widget = self.query_one("#weather", Static)
+        if city:
+            # Query the network API
+            url = f"https://wttr.in/{city}"
+            async with httpx.AsyncClient() as client:
+                response = await client.get(url)
+                weather = Text.from_ansi(response.text)
+                weather_widget.update(weather)
+        else:
+            # No city, so just blank out the weather
+            weather_widget.update("")
+
+    def on_worker_state_changed(self, event: Worker.StateChanged) -> None:
+        """Called when the worker state changes."""
+        self.log(event)
+
+
+if __name__ == "__main__":
+    app = WeatherApp()
+    app.run()

docs/examples/guide/workers/weather05.py

@@ -0,0 +1,52 @@
+from urllib.request import Request, urlopen
+
+from rich.text import Text
+
+from textual import work
+from textual.app import App, ComposeResult
+from textual.containers import VerticalScroll
+from textual.widgets import Input, Static
+from textual.worker import Worker, get_current_worker
+
+
+class WeatherApp(App):
+    """App to display the current weather."""
+
+    CSS_PATH = "weather.css"
+
+    def compose(self) -> ComposeResult:
+        yield Input(placeholder="Enter a City")
+        with VerticalScroll(id="weather-container"):
+            yield Static(id="weather")
+
+    async def on_input_changed(self, message: Input.Changed) -> None:
+        """Called when the input changes"""
+        self.update_weather(message.value)
+
+    @work(exclusive=True)
+    def update_weather(self, city: str) -> None:
+        """Update the weather for the given city."""
+        weather_widget = self.query_one("#weather", Static)
+        worker = get_current_worker()
+        if city:
+            # Query the network API
+            url = f"https://wttr.in/{city}"
+            request = Request(url)
+            request.add_header("User-agent", "CURL")
+            response_text = urlopen(request).read().decode("utf-8")
+            weather = Text.from_ansi(response_text)
+            if not worker.is_cancelled:
+                self.call_from_thread(weather_widget.update, weather)
+        else:
+            # No city, so just blank out the weather
+            if not worker.is_cancelled:
+                self.call_from_thread(weather_widget.update, "")
+
+    def on_worker_state_changed(self, event: Worker.StateChanged) -> None:
+        """Called when the worker state changes."""
+        self.log(event)
+
+
+if __name__ == "__main__":
+    app = WeatherApp()
+    app.run()