Add pytest-textual-snapshot and integration tests (#259)

* Add pytest-textual-snapshot and integration tests

- Add pytest-textual-snapshot for visual regression testing of Textual UI
- Create snapshot tests for ExitModal, InputField, OpenHandsApp, ConfirmationModal
- Add integration test fixtures for real LLM testing via env vars
- Create integration tests that send messages to real LLM
- Register 'integration' pytest marker in pyproject.toml
- Document snapshot and integration testing in repo.md

Co-authored-by: openhands <openhands@all-hands.dev>

* Fix CI failures: format code and mock WORK_DIR in snapshot test

- Apply ruff formatting to tests/conftest.py and tests/integration/test_cli_with_real_llm.py
- Mock WORK_DIR in test_openhands_app_splash_screen to ensure consistent snapshots across environments
- Update snapshot with deterministic /test/workspace path instead of environment-specific path

Co-authored-by: openhands <openhands@all-hands.dev>

* Remove tests not related to OpenHands CLI's TUI

- Remove TestSimpleWidgetSnapshots class (generic button grid test)
- Remove integration tests directory (tests headless mode, not TUI)
- Remove integration test fixtures from conftest.py
- Update repo.md documentation

Co-authored-by: openhands <openhands@all-hands.dev>

* Simplify snapshot tests: keep only one real UI test

Remove all mocked tests and keep only TestExitModalSnapshots.test_exit_modal_initial_state
which tests the real ExitConfirmationModal widget without any mocks.

Co-authored-by: openhands <openhands@all-hands.dev>

---------

Co-authored-by: openhands <openhands@all-hands.dev>

Author: xingyaoww

.openhands/microagents/repo.md

@@ -38,4 +38,114 @@ If the user says something along the lines of "update the sha" or "update the ag
 4. Run `./build.sh` to confirm that the build still works
 5. Open a pull request with the changes
 
-If the build fails, still open the pull request and explain what error you're seeing, and the steps you plan to take to fix it; don't fix it yet though.
+If the build fails, still open the pull request and explain what error you're seeing, and the steps you plan to take to fix it; don't fix it yet though.
+
+## Snapshot Testing with pytest-textual-snapshot
+
+The CLI uses [pytest-textual-snapshot](https://github.com/Textualize/pytest-textual-snapshot) for visual regression testing of Textual UI components. Snapshots are SVG screenshots that capture the exact visual state of the application.
+
+### Running Snapshot Tests
+
+```bash
+# Run all snapshot tests
+uv run pytest tests/snapshots/ -v
+
+# Update snapshots when intentional UI changes are made
+uv run pytest tests/snapshots/ --snapshot-update
+```
+
+### Snapshot Test Location
+
+- **Test files**: `tests/snapshots/test_app_snapshots.py`
+- **Generated snapshots**: `tests/snapshots/__snapshots__/test_app_snapshots/*.svg`
+
+### Writing Snapshot Tests
+
+Snapshot tests must be **synchronous** (not async). The `snap_compare` fixture handles async internally:
+
+```python
+from textual.app import App, ComposeResult
+from textual.widgets import Static, Footer
+
+def test_my_widget(snap_compare):
+    """Snapshot test for my widget."""
+    
+    class MyTestApp(App):
+        def compose(self) -> ComposeResult:
+            yield Static("Content")
+            yield Footer()
+    
+    assert snap_compare(MyTestApp(), terminal_size=(80, 24))
+```
+
+#### Using `run_before` for Setup
+
+To interact with the app before taking a screenshot:
+
+```python
+def test_with_interaction(snap_compare):
+    class MyApp(App):
+        def compose(self) -> ComposeResult:
+            yield InputField(id="input")
+    
+    async def setup(pilot):
+        input_field = pilot.app.query_one(InputField)
+        input_field.input_widget.value = "Hello!"
+        await pilot.pause()
+    
+    assert snap_compare(MyApp(), terminal_size=(80, 24), run_before=setup)
+```
+
+#### Using `press` for Key Simulation
+
+```python
+def test_with_focus(snap_compare):
+    assert snap_compare(
+        MyApp(),
+        terminal_size=(80, 24),
+        press=["tab", "tab"],  # Press tab twice to move focus
+    )
+```
+
+### Viewing Snapshots Visually
+
+To view the generated SVG snapshots in a browser:
+
+1. **Start a local HTTP server** in the snapshots directory:
+   ```bash
+   cd tests/snapshots/__snapshots__/test_app_snapshots
+   python -m http.server 12000
+   ```
+
+2. **Open in browser** using the work host URL:
+   ```
+   https://work-1-eidmcsndvfctphkv.prod-runtime.all-hands.dev/<snapshot-name>.svg
+   ```
+   
+   Example snapshot names:
+   - `TestExitModalSnapshots.test_exit_modal_initial_state.svg`
+   - `TestOpenHandsAppSnapshots.test_openhands_app_splash_screen.svg`
+   - `TestInputFieldSnapshots.test_input_field_with_text.svg`
+
+3. **Stop the server** when done:
+   ```bash
+   pkill -f "python -m http.server 12000"
+   ```
+
+### Current Snapshot Tests
+
+| Test Class | Test Name | Description |
+|------------|-----------|-------------|
+| `TestExitModalSnapshots` | `test_exit_modal_initial_state` | Exit confirmation modal initial view |
+| `TestExitModalSnapshots` | `test_exit_modal_with_focus_on_yes` | Exit modal with focus on Yes button |
+| `TestInputFieldSnapshots` | `test_input_field_single_line_mode` | Input field in default state |
+| `TestInputFieldSnapshots` | `test_input_field_with_text` | Input field with typed text |
+| `TestOpenHandsAppSnapshots` | `test_openhands_app_splash_screen` | Main app splash screen (mocked) |
+| `TestConfirmationModalSnapshots` | `test_confirmation_settings_modal` | Confirmation settings modal |
+
+### Best Practices
+
+1. **Mock external dependencies** - Use `unittest.mock.patch` to ensure deterministic snapshots
+2. **Use fixed terminal sizes** - Always specify `terminal_size=(width, height)` for consistent results
+3. **Commit snapshots to git** - SVG files are test artifacts and should be version controlled
+4. **Review snapshot diffs carefully** - When tests fail, examine the visual diff to determine if the change is intentional

pyproject.toml

@@ -44,6 +44,7 @@ dev = [
   "pytest-cov>=5.0.0",
   "pytest-forked>=1.6.0",
   "pytest-xdist>=3.6.1",
+  "pytest-textual-snapshot>=1.1.0",
   "ruff>=0.12.10",
   "pycodestyle>=2.12.0",
   "pyright[nodejs]>=1.1.405",
@@ -106,6 +107,9 @@ python_classes = ["Test*"]
 python_functions = ["test_*"]
 addopts = "-v --tb=short"
 asyncio_mode = "auto"
+markers = [
+    "integration: marks tests as integration tests (deselect with '-m \"not integration\"')",
+]
 
 [tool.coverage.run]
 relative_files = true

tests/snapshots/__init__.py

@@ -0,0 +1 @@
+"""Snapshot tests for OpenHands CLI using pytest-textual-snapshot."""