03: Playwright, interceptor, fixtures. Improve TestClient tests with beautifulsoup4.

Author: pauleveritt

docs/building/playwright.md

@@ -0,0 +1,112 @@
+# Playwright Interceptors
+
+PyScript testing needs a real browser -- web components which load Pyodide and execute Python, then change the DOM.
+[Playwright](https://playwright.dev/python/) provides this, but we'd like more convenience in the testing:
+
+- Don't actually launch a web server to fetch the examples
+- Make it easier to write examples and tests by having some automation
+
+In this step we bring in Playwright, but don't yet use PyScript.
+Here's the big idea: we do _not_ run a web server.
+Instead, we write a Playwright interceptor as a pytest fixture.
+
+## Install Playwright
+
+We need to add Playwright to PSC.
+In the current repo, this is done as part of a Makefile rule, which also copies the examples to a relative directory (bleh).
+
+Instead, we'll just make it a development dependency.
+If a Contributor wants to write an example, they just need to clone the repo and do a Poetry install with dev dependencies.
+Kind of normal Python dev workflow.
+To make it work in CI using Nox, we added this dependency to the `noxfile.py`.
+
+This still requires running `playwright install` manually, to get the Playwright browsers globally installed.
+That has to be added to PSC Contributor documentation.
+
+## Fixture
+
+With Playwright installed, now it is time to make it easier to write/run the tests for examples.
+In the previous step, we did "shallow" testing of an example, using `TestClient` to ensure the HTML was returned.
+We didn't actually load the HTML into a DOM, certainly didn't evaluate the PyScript web components, and _definitely_ didn't run some Python in Pyodide.
+
+The current Collective uses Playwright's `page` fixture directly: you provide a URL, it tells the browser to make an HTTP request.
+This means it needs an HTTP server running.
+The repo fires up and shuts down a Python `SimpleHTTPServer` running in a thread, as part of test running.
+
+If something gets hung...ouch.
+You have to wait for the thread to time out.
+
+PSC changes this by not running an HTTP server for testing the examples.
+Instead, we use [Playwright interceptors](https://playwright.dev/python/docs/network#modify-responses).
+When the URL comes in, our Python code runs and returns a response...quite like `TestClient` pretends to run an ASGI server.
+Our "interceptor" looks at the URL, and if it is to the "fake" server, it reads/returns the path from disk.
+
+This fixture is software, so we'll make a file at `src/psc/fixtures.py` and a test at `test_fixtures.py`.
+We also need to make `tests/conftest.py` to load this as a pytest plugin.
+The test file has dummy objects for the Playwright request/response/page/route etc.
+The tests exercise the main code paths we need for the interceptor:
+
+- A request but _not_ to the fake server URL should just be passed-through to an HTTP request
+- A request to the fake server URL should extract the path
+  - If that path exists in the project, read the file and return it
+  - If not, raise a value error
+
+With that in place, we write a `fixtures.fake_page` fixture function.
+It asks `pytest` to inject the real `page`.
+It then installs the interceptor by calling a helper function.
+This helper function is what we actually write the fixture test for.
+
+## Serve Up Examples
+
+We aren't going to test by fetching examples from an HTTP server.
+But our Viewers will look at examples from the HTTP server we made in the previous step.
+Let's add that to `app.py` with another `Mount`, this time pointing `/examples` at `src/psc/examples`.
+Also, add `first.html` with some dummy text as an "example".
+
+Before the implementation, we add `test_app.test_first_example` as a failing test.
+Then, once `app.py` is fixed, the test will pass.
+
+## First Test
+
+Our fixture is now in place, with a test that has good coverage.
+We have a dummy example in `first.html`.
+Let's write a test that uses Playwright and the interceptor.
+
+We just added a `TestClient` test -- a kind of "shallow" test -- for `first.html`.
+In `test_app.py` we add `test_first_example_full` as a Playwright test.
+
+When we first run it, we see `fixture 'fake_page' not found`.
+This is because `conftest.py` needs to load the `psc.fixtures`.
+With that line added, the tests pass.
+
+## Shallow vs. Full Markers
+
+These Playwright tests are SLOW.
+When we get a bunch of examples, it's going to be a pain.
+As such, we'll want to emphasize unit tests and the shallow `TestClient` tests.
+
+To make this first-class, we'll add 3 pytest markers to the project: unit, shallow, and full.
+We do so in `pyproject.toml` along with the option to warn if someone uses an undefined customer marker.
+
+With this in place, we add decorators such as `@pytest.mark.full` to our tests.
+Later, we can run `pytest -m "not full"` to skip the Playwright tests.
+
+## Better `TestClient` Testing
+
+In this step we also improved the `TestClient` tests which makes sure our Starlette app serves what we expect.
+Part of that means testing HTML, so we installed `beatifulsoup4` and made a fixture that lets us issue a request and get back `BeautifulSoup`.
+These have rich CSS selector locators, so very convenient to use in tests.
+This fixture also raise an exception if it doesn't get a `200` so you don't have to test that any more.
+
+Just to be clear: `TestClient` tests are nice when you do *not* need to get into the JS/Pyodide side.
+They are fast and zero mystery.
+
+Of course, we added tests for those fixtures.
+
+## QA
+
+Cleaned up everything for pre-commit, mypy, nox, etc.
+Coverage is still 100%.
+
+Along the way, Typeguard got mad at the introduction of the marker.
+I skipped investigation and just disabled Typeguard from the noxfile for now.

noxfile.py

@@ -161,6 +161,8 @@ def mypy(session: Session) -> None:
         "jinja2",
         "markdown-it-py",
         "python-frontmatter",
+        "requests",
+        "types-requests",
     )
     session.run("mypy", *args)
 

poetry.lock

@@ -56,6 +56,7 @@ beautifulsoup4 = "^4.11.1"
 html5lib = "^1.1"
 types-beautifulsoup4 = "^4.11.4"
 Jinja2 = "^3.1.2"
+types-requests = "^2.28.10"
 
 [tool.poetry.scripts]
 psc = "psc.__main__:main"

pyproject.toml

@@ -1 +1,9 @@
 """PyScript Collective."""
+
+from pathlib import Path
+
+
+# Paths that can be referenced anywhere and get the right target.
+
+SRC = Path(__file__).parent
+STATIC = SRC / "static"

src/psc/__init__.py

@@ -1,5 +1,4 @@
 """Provide a web server to browse the examples."""
-from pathlib import Path
 
 from starlette.applications import Starlette
 from starlette.requests import Request
@@ -8,8 +7,7 @@
 from starlette.routing import Route
 from starlette.staticfiles import StaticFiles
 
-
-HERE = Path(__file__).parent
+from . import STATIC
 
 
 def index_page(request: Request) -> HTMLResponse:
@@ -19,7 +17,7 @@ def index_page(request: Request) -> HTMLResponse:
 
 routes = [
     Route("/", index_page),
-    Mount("/static", StaticFiles(directory=HERE / "static")),
+    Mount("/static", StaticFiles(directory=STATIC)),
 ]
 
 app = Starlette(debug=True, routes=routes)

src/psc/app.py

@@ -0,0 +1,182 @@
+"""Automate some testing."""
+from __future__ import annotations
+
+from dataclasses import dataclass
+from dataclasses import field
+from mimetypes import guess_type
+from typing import Callable
+from urllib.parse import urlparse
+
+import pytest
+from bs4 import BeautifulSoup
+from playwright.sync_api import Page
+from playwright.sync_api import Route
+from requests.models import Response
+from starlette.testclient import TestClient
+
+from psc import SRC
+from psc.app import app
+
+
+@dataclass
+class MockTestClient:
+    """Pretend to be Starlette ``TestClient``."""
+
+    test_status_code: int = 200
+    test_url: str | None = None
+    test_content: bytes | None = None
+
+    def get(self, url: str) -> Response:
+        """Fake the TestClient response getting."""
+        self.test_url = url
+        response = Response()
+        if self.test_url == "/broken":
+            # This is a flag to test base_page exception raising
+            response.status_code = 404
+        else:
+            response.status_code = self.test_status_code
+        response._content = (
+            self.test_content if self.test_content is not None else b"Test Result"
+        )
+        return response
+
+
+PageT = Callable[..., BeautifulSoup]
+
+
+@pytest.fixture
+def test_client() -> TestClient:
+    """Get a TestClient for the app."""
+    return TestClient(app)
+
+
+def _base_page(client: TestClient | MockTestClient) -> PageT:
+    """Automate ``TestClient`` to return BeautifulSoup.
+
+    Along the way, default to raising an exception if the status
+    code isn't 200.
+    """
+    # Allow passing in a fake TestClient, for testing this fixture.
+
+    def _page(url: str, *, enforce_status: bool = True) -> BeautifulSoup:
+        """Callable that retrieves and returns soup."""
+        response = client.get(url)
+        if enforce_status and response.status_code != 200:
+            raise ValueError(
+                f"Request to {url} resulted in status code {response.status_code}"
+            )
+        return BeautifulSoup(response.text, "html5lib")
+
+    return _page
+
+
+def mocked_client_page() -> PageT:
+    """Get a fake test client, to allow testing the logic in base_page."""
+    client = MockTestClient()
+    return _base_page(client)
+
+
+@pytest.fixture()
+def client_page(test_client: TestClient) -> PageT:
+    """Main fixture for getting BeautifulSoup via TestClient."""
+    return _base_page(test_client)
+
+
+@dataclass
+class DummyResponse:
+    """Fake the Playwright ``Response`` class."""
+
+    dummy_text: str = ""
+    headers: dict[str, object] = field(
+        default_factory=lambda: {"Content-Type": "text/html"}
+    )
+    status: int | None = None
+
+    def text(self) -> str:
+        """Fake the text method."""
+        return self.dummy_text
+
+    def body(self) -> bytes:
+        """Fake the text method."""
+        return bytes(self.dummy_text, "utf-8")
+
+
+@dataclass
+class DummyRequest:
+    """Fake the Playwright ``Request`` class."""
+
+    url: str
+
+    @staticmethod
+    def fetch(request: DummyRequest) -> DummyResponse:
+        """Fake the fetch method."""
+        return DummyResponse(dummy_text="URL Returned Text")
+
+
+@dataclass
+class DummyRoute:
+    """Fake the Playwright ``Route`` class."""
+
+    request: DummyRequest
+    body: bytes | None = None
+    status: str | None = None
+    headers: dict[str, object] | None = None
+
+    def fulfill(self, body: bytes, headers: dict[str, object], status: int) -> None:
+        """Stub the Playwright ``route.fulfill`` method."""
+        self.body = body
+        self.headers = headers
+        self.status = str(status)
+
+
+@dataclass
+class DummyPage:
+    """Fake the Playwright ``Page`` class."""
+
+    request: DummyRequest
+
+
+def route_handler(page: Page, route: Route) -> None:
+    """Called from the interceptor to get the data off disk."""
+    this_url = urlparse(route.request.url)
+    this_path = this_url.path[1:]
+    is_fake = this_url.hostname == "fake"
+    headers = dict()
+    if is_fake:
+        # We should read something from the filesystem
+        this_fs_path = SRC / this_path
+        if this_fs_path.exists():
+            status = 200
+            mime_type = guess_type(this_fs_path)[0]
+            if mime_type:
+                headers = {"Content-Type": mime_type}
+            body = this_fs_path.read_bytes()
+        else:
+            status = 404
+            body = b""
+    else:
+        # This is to a non-fake server. Only for cases where the
+        # local HTML asked for something out in the big wide world.
+        response = page.request.fetch(route.request)
+        status = response.status
+        body = response.body()
+        headers = response.headers
+
+    route.fulfill(body=body, headers=headers, status=status)
+
+
+@pytest.fixture
+def fake_page(page: Page) -> Page:  # pragma: no cover
+    """On the fake server, intercept and return from fs."""
+
+    def _route_handler(route: Route) -> None:
+        """Instead of doing this inline, call to a helper for easier testing."""
+        route_handler(page, route)
+
+    # Use Playwright's route method to intercept any URLs pointed at the
+    # fake server and run through the interceptor instead.
+    page.route("**", _route_handler)
+
+    # Don't spend 30 seconds on timeout
+    page.set_default_timeout(4000)
+    return page