Merge pull request #1444 from Textualize/strip-optimization

adds Strip primitive

Author: willmcgugan

CHANGELOG.md

@@ -5,12 +5,19 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](http://keepachangelog.com/)
 and this project adheres to [Semantic Versioning](http://semver.org/).
 
-## [0.8.3] - Unreleased
+## [0.9.0] - Unreleased
 
 ### Added
 
+- Added textual.strip.Strip primitive
+- Added textual._cache.FIFOCache
 - Added an option to clear columns in DataTable.clear() https://github.com/Textualize/textual/pull/1427
 
+### Changed
+
+- Widget.render_line now returns a Strip
+- Fix for slow updates on Windows
+  
 ## [0.8.2] - 2022-12-28
 
 ### Fixed
@@ -308,6 +315,10 @@ https://textual.textualize.io/blog/2022/11/08/version-040/#version-040
 - New handler system for messages that doesn't require inheritance
 - Improved traceback handling
 
+[0.9.0]: https://github.com/Textualize/textual/compare/v0.8.2...v0.9.0
+[0.8.2]: https://github.com/Textualize/textual/compare/v0.8.1...v0.8.2
+[0.8.1]: https://github.com/Textualize/textual/compare/v0.8.0...v0.8.1
+[0.8.0]: https://github.com/Textualize/textual/compare/v0.7.0...v0.8.0
 [0.7.0]: https://github.com/Textualize/textual/compare/v0.6.0...v0.7.0
 [0.6.0]: https://github.com/Textualize/textual/compare/v0.5.0...v0.6.0
 [0.5.0]: https://github.com/Textualize/textual/compare/v0.4.0...v0.5.0

docs/blog/posts/better-sleep-on-windows.md

@@ -0,0 +1,54 @@
+---
+draft: false
+date: 2022-12-30
+categories:
+  - DevLog
+authors:
+  - willmcgugan
+---
+# A better asyncio sleep for Windows to fix animation
+
+I spent some time optimizing Textual on Windows recently, and discovered something which may be of interest to anyone working with async code on that platform.
+
+<!-- more -->
+
+Animation, scrolling, and fading had always been unsatisfactory on Windows. Textual was usable, but the lag when scrolling made it a little unpleasant to use. On macOS and Linux, scrolling is fast enough that it feels close to a native app, and not something running in a terminal. Yet the Windows experience never improved, even as Textual got faster with each release.
+
+I had chalked this up to Windows Terminal being slow to render updates. After all, the classic Windows terminal was (and still is) glacially slow. Perhaps Microsoft just weren't focusing on performance.
+
+In retrospect, that was highly improbable. Like all modern terminals, Windows Terminal uses the GPU to render updates. Even without focussing on performance, it should be fast.
+
+I figured I'd give it once last attempt to speed up Textual on Windows. If I failed, Windows would forever be a third-class platform for Textual apps.
+
+It turned out that it was nothing to do with performance, per se. The issue was with a single asyncio function: `asyncio.sleep`.
+
+Textual has a `Timer` class which creates events at regular intervals. It powers the JS-like `set_interval` and `set_timer` functions. It is also used internally to do animation (such as smooth scrolling). This Timer class calls `asyncio.sleep` to wait the time between one event and the next.
+
+On macOS and Linux, calling `asynco.sleep` is fairly accurate. If you call `sleep(3.14)`, it will return within 1% of 3.14 seconds. This is not the case for Windows, which for historical reasons uses a timer with a granularity of 15 milliseconds. The upshot is that sleep times will be rounded up to the nearest multiple of 15 milliseconds.
+
+This limit appears holds true for all async primitives on Windows. If you wait for something with a timeout, it will return on a multiple of 15 milliseconds. Fortunately there is work in the CPython pipeline to make this more accurate. Thanks to [Steve Dower](https://twitter.com/zooba) for pointing this out.
+
+This lack of accuracy in the timer meant that timer events were created at a far slower rate that intended. Animation was slower because Textual was waiting too long between updates.
+
+Once I had figured that out, I needed an alternative to `asyncio.sleep` for Textual's Timer class. And I found one. The following version of `sleep` is accurate to well within 1%:
+
+```python
+from time import sleep
+from asyncio import get_running_loop
+
+async def sleep(sleep_for: float) -> None:
+    """An asyncio sleep.
+
+    On Windows this achieves a better granularity that asyncio.sleep
+
+    Args:
+        sleep_for (float): Seconds to sleep for.
+    """
+    await get_running_loop().run_in_executor(None, time_sleep, sleep_for)
+```
+
+That is a drop-in replacement for sleep on Windows. With it, Textual runs a *lot* smoother. Easily on par with macOS and Linux.
+
+It's not quite perfect. There is a little *tearing* during full "screen" updates, but performance is decent all round. I suspect when [this bug]( https://bugs.python.org/issue37871) is fixed (big thanks to [Paul Moore](https://twitter.com/pf_moore) for looking in to that), and Microsoft implements [this protocol](https://gist.github.com/christianparpart/d8a62cc1ab659194337d73e399004036) then Textual on Windows will be A+.
+
+This Windows improvement will be in v0.9.0 of [Textual](https://github.com/Textualize/textual), which will be released in a few days.

src/textual/_cache.py

@@ -14,13 +14,14 @@
 
 from __future__ import annotations
 
-from threading import Lock
 from typing import Dict, Generic, KeysView, TypeVar, overload
 
 CacheKey = TypeVar("CacheKey")
 CacheValue = TypeVar("CacheValue")
 DefaultValue = TypeVar("DefaultValue")
 
+__all__ = ["LRUCache", "FIFOCache"]
+
 
 class LRUCache(Generic[CacheKey, CacheValue]):
     """
@@ -37,12 +38,22 @@ class LRUCache(Generic[CacheKey, CacheValue]):
 
     """
 
+    __slots__ = [
+        "_maxsize",
+        "_cache",
+        "_full",
+        "_head",
+        "hits",
+        "misses",
+    ]
+
     def __init__(self, maxsize: int) -> None:
         self._maxsize = maxsize
         self._cache: Dict[CacheKey, list[object]] = {}
         self._full = False
         self._head: list[object] = []
-        self._lock = Lock()
+        self.hits = 0
+        self.misses = 0
         super().__init__()
 
     @property
@@ -60,6 +71,11 @@ def __bool__(self) -> bool:
     def __len__(self) -> int:
         return len(self._cache)
 
+    def __repr__(self) -> str:
+        return (
+            f"<LRUCache maxsize={self._maxsize} hits={self.hits} misses={self.misses}>"
+        )
+
     def grow(self, maxsize: int) -> None:
         """Grow the maximum size to at least `maxsize` elements.
 
@@ -70,10 +86,9 @@ def grow(self, maxsize: int) -> None:
 
     def clear(self) -> None:
         """Clear the cache."""
-        with self._lock:
-            self._cache.clear()
-            self._full = False
-            self._head = []
+        self._cache.clear()
+        self._full = False
+        self._head = []
 
     def keys(self) -> KeysView[CacheKey]:
         """Get cache keys."""
@@ -87,29 +102,28 @@ def set(self, key: CacheKey, value: CacheValue) -> None:
             key (CacheKey): Key.
             value (CacheValue): Value.
         """
-        with self._lock:
-            link = self._cache.get(key)
-            if link is None:
+        link = self._cache.get(key)
+        if link is None:
+            head = self._head
+            if not head:
+                # First link references itself
+                self._head[:] = [head, head, key, value]
+            else:
+                # Add a new root to the beginning
+                self._head = [head[0], head, key, value]
+                # Updated references on previous root
+                head[0][1] = self._head  # type: ignore[index]
+                head[0] = self._head
+            self._cache[key] = self._head
+
+            if self._full or len(self._cache) > self._maxsize:
+                # Cache is full, we need to evict the oldest one
+                self._full = True
                 head = self._head
-                if not head:
-                    # First link references itself
-                    self._head[:] = [head, head, key, value]
-                else:
-                    # Add a new root to the beginning
-                    self._head = [head[0], head, key, value]
-                    # Updated references on previous root
-                    head[0][1] = self._head  # type: ignore[index]
-                    head[0] = self._head
-                self._cache[key] = self._head
-
-                if self._full or len(self._cache) > self._maxsize:
-                    # Cache is full, we need to evict the oldest one
-                    self._full = True
-                    head = self._head
-                    last = head[0]
-                    last[0][1] = head  # type: ignore[index]
-                    head[0] = last[0]  # type: ignore[index]
-                    del self._cache[last[2]]  # type: ignore[index]
+                last = head[0]
+                last[0][1] = head  # type: ignore[index]
+                head[0] = last[0]  # type: ignore[index]
+                del self._cache[last[2]]  # type: ignore[index]
 
     __setitem__ = set
 
@@ -135,31 +149,136 @@ def get(
         """
         link = self._cache.get(key)
         if link is None:
+            self.misses += 1
             return default
-        with self._lock:
-            if link is not self._head:
-                # Remove link from list
-                link[0][1] = link[1]  # type: ignore[index]
-                link[1][0] = link[0]  # type: ignore[index]
-                head = self._head
-                # Move link to head of list
-                link[0] = head[0]
-                link[1] = head
-                self._head = head[0][1] = head[0] = link  # type: ignore[index]
+        if link is not self._head:
+            # Remove link from list
+            link[0][1] = link[1]  # type: ignore[index]
+            link[1][0] = link[0]  # type: ignore[index]
+            head = self._head
+            # Move link to head of list
+            link[0] = head[0]
+            link[1] = head
+            self._head = head[0][1] = head[0] = link  # type: ignore[index]
+        self.hits += 1
+        return link[3]  # type: ignore[return-value]
+
+    def __getitem__(self, key: CacheKey) -> CacheValue:
+        link = self._cache.get(key)
+        if link is None:
+            self.misses += 1
+            raise KeyError(key)
+        if link is not self._head:
+            link[0][1] = link[1]  # type: ignore[index]
+            link[1][0] = link[0]  # type: ignore[index]
+            head = self._head
+            link[0] = head[0]
+            link[1] = head
+            self._head = head[0][1] = head[0] = link  # type: ignore[index]
+        self.hits += 1
+        return link[3]  # type: ignore[return-value]
+
+    def __contains__(self, key: CacheKey) -> bool:
+        return key in self._cache
+
+
+class FIFOCache(Generic[CacheKey, CacheValue]):
+    """A simple cache that discards the first added key when full (First In First Out).
+
+    This has a lower overhead than LRUCache, but won't manage a working set as efficiently.
+    It is most suitable for a cache with a relatively low maximum size that is not expected to
+    do many lookups.
+
+    Args:
+        maxsize (int): Maximum size of the cache.
+    """
+
+    __slots__ = [
+        "_maxsize",
+        "_cache",
+        "hits",
+        "misses",
+    ]
+
+    def __init__(self, maxsize: int) -> None:
+        self._maxsize = maxsize
+        self._cache: dict[CacheKey, CacheValue] = {}
+        self.hits = 0
+        self.misses = 0
+
+    def __bool__(self) -> bool:
+        return bool(self._cache)
+
+    def __len__(self) -> int:
+        return len(self._cache)
+
+    def __repr__(self) -> str:
+        return (
+            f"<FIFOCache maxsize={self._maxsize} hits={self.hits} misses={self.misses}>"
+        )
+
+    def clear(self) -> None:
+        """Clear the cache."""
+        self._cache.clear()
+
+    def keys(self) -> KeysView[CacheKey]:
+        """Get cache keys."""
+        # Mostly for tests
+        return self._cache.keys()
+
+    def set(self, key: CacheKey, value: CacheValue) -> None:
+        """Set a value.
+
+        Args:
+            key (CacheKey): Key.
+            value (CacheValue): Value.
+        """
+        if key not in self._cache and len(self._cache) >= self._maxsize:
+            for first_key in self._cache:
+                self._cache.pop(first_key)
+                break
+        self._cache[key] = value
 
-            return link[3]  # type: ignore[return-value]
+    __setitem__ = set
+
+    @overload
+    def get(self, key: CacheKey) -> CacheValue | None:
+        ...
+
+    @overload
+    def get(self, key: CacheKey, default: DefaultValue) -> CacheValue | DefaultValue:
+        ...
+
+    def get(
+        self, key: CacheKey, default: DefaultValue | None = None
+    ) -> CacheValue | DefaultValue | None:
+        """Get a value from the cache, or return a default if the key is not present.
+
+        Args:
+            key (CacheKey): Key
+            default (Optional[DefaultValue], optional): Default to return if key is not present. Defaults to None.
+
+        Returns:
+            Union[CacheValue, Optional[DefaultValue]]: Either the value or a default.
+        """
+        try:
+            result = self._cache[key]
+        except KeyError:
+            self.misses += 1
+            return default
+        else:
+            self.hits += 1
+            return result
 
     def __getitem__(self, key: CacheKey) -> CacheValue:
-        link = self._cache[key]
-        with self._lock:
-            if link is not self._head:
-                link[0][1] = link[1]  # type: ignore[index]
-                link[1][0] = link[0]  # type: ignore[index]
-                head = self._head
-                link[0] = head[0]
-                link[1] = head
-                self._head = head[0][1] = head[0] = link  # type: ignore[index]
-            return link[3]  # type: ignore[return-value]
+        try:
+            result = self._cache[key]
+        except KeyError:
+            self.misses += 1
+            raise KeyError(key) from None
+        else:
+            self.hits += 1
+            return result
 
     def __contains__(self, key: CacheKey) -> bool:
         return key in self._cache