From 7b6d6d2b9822b558f4667bbfd4c9777679fcb46f Mon Sep 17 00:00:00 2001 From: "copilot-swe-agent[bot]" <198982749+Copilot@users.noreply.github.com> Date: Mon, 3 Nov 2025 04:19:42 +0000 Subject: [PATCH 1/5] Initial plan From 8217ac92836aa024e28e43c4117c91736c227b30 Mon Sep 17 00:00:00 2001 From: "copilot-swe-agent[bot]" <198982749+Copilot@users.noreply.github.com> Date: Mon, 3 Nov 2025 04:23:33 +0000 Subject: [PATCH 2/5] Add comprehensive GitHub Copilot instructions Co-authored-by: saulshanabrook <1186124+saulshanabrook@users.noreply.github.com> --- .github/copilot-instructions.md | 187 ++++++++++++++++++++++++++++++++ 1 file changed, 187 insertions(+) create mode 100644 .github/copilot-instructions.md diff --git a/.github/copilot-instructions.md b/.github/copilot-instructions.md new file mode 100644 index 00000000..1bf52951 --- /dev/null +++ b/.github/copilot-instructions.md @@ -0,0 +1,187 @@ +# GitHub Copilot Instructions for egglog-python + +## Project Overview + +This repository provides Python bindings for the Rust library `egglog`, enabling the use of e-graphs in Python for optimization, symbolic computation, and analysis. It is a hybrid project combining: +- **Python code** in `python/egglog/` - The main Python API and library +- **Rust code** in `src/` - PyO3-based bindings to the egglog Rust library +- **Documentation** in `docs/` - Sphinx-based documentation + +## Repository Structure + +- `python/egglog/` - Main Python package source code +- `python/tests/` - Python test suite (pytest-based) +- `src/` - Rust source code for Python bindings (PyO3) +- `docs/` - Documentation source files (Sphinx) +- `test-data/` - Test data files +- `pyproject.toml` - Python project configuration and dependencies +- `Cargo.toml` - Rust project configuration +- `uv.lock` - Locked dependencies (managed by uv) + +## Build and Development Commands + +### Prerequisites +- **uv** - Package manager (https://github.com/astral-sh/uv) +- **Rust toolchain** - Version 1.79.0 (specified in rust-toolchain.toml) +- **Python** - 3.10+ (see .python-version) + +### Common Commands + +```bash +# Install dependencies +uv sync --extra dev --locked + +# Run tests +uv run pytest --benchmark-disable -vvv --durations=10 + +# Type checking with mypy +make mypy + +# Stub testing +make stubtest + +# Build documentation +make docs + +# Format code (auto-run by pre-commit) +uv run ruff format . + +# Lint code (auto-run by pre-commit) +uv run ruff check --fix . +``` + +## Python Code Standards + +### General Guidelines +- **Line length**: 120 characters maximum +- **Type hints**: Use type annotations for public APIs and functions +- **Formatting**: Use Ruff for code formatting and linting +- **Testing**: Write tests using pytest in `python/tests/` +- **Docstrings**: Use clear, concise docstrings for public functions and classes + +### Ruff Configuration +The project uses Ruff for linting and formatting with specific rules: +- Allows uppercase variable names (N806, N802) +- Allows star imports (F405, F403) +- Allows `exec` and subprocess usage (S102, S307, S603) +- Allows `Any` type annotations (ANN401) +- Test files don't require full type annotations + +See `pyproject.toml` for complete Ruff configuration. + +### Type Checking +- **mypy** is used for static type checking +- Run `make mypy` to type check Python code +- Run `make stubtest` to validate type stubs against runtime behavior +- Exclusions: `__snapshots__`, `_build`, `conftest.py` + +### Testing +- Tests are located in `python/tests/` +- Use pytest with snapshot testing (pytest-inline-snapshot) +- Benchmarks use pytest-benchmark and CodSpeed +- Run tests with: `uv run pytest --benchmark-disable -vvv` + +## Rust Code Standards + +### General Guidelines +- **Edition**: Rust 2024 +- **FFI**: Uses PyO3 for Python bindings +- **Main library**: Uses egglog from git (currently saulshanabrook/egg-smol branch) + +### File Organization +- `src/lib.rs` - Main library entry point +- `src/egraph.rs` - E-graph implementation +- `src/conversions.rs` - Type conversions between Python and Rust +- `src/py_object_sort.rs` - Python object handling +- `src/extract.rs` - Extraction functionality +- `src/error.rs` - Error handling +- `src/serialize.rs` - Serialization support +- `src/termdag.rs` - Term DAG operations +- `src/utils.rs` - Utility functions + +## Code Style Preferences + +1. **Imports**: Follow Ruff's import sorting +2. **Naming**: + - Python: snake_case for functions and variables, PascalCase for classes + - Rust: Follow standard Rust conventions +3. **Comments**: Use clear, explanatory comments for complex logic +4. **Documentation**: Keep docs synchronized with code changes + +## Pre-commit Hooks + +The repository uses pre-commit with: +- `ruff-check` with auto-fix +- `ruff-format` for formatting +- `uv-lock` to keep lockfile updated + +Run `pre-commit install` to enable automatic checking. + +## Dependencies + +### Python Dependencies +- **Core**: typing-extensions, black, graphviz, anywidget +- **Array support**: scikit-learn, array_api_compat, numba, numpy>2 +- **Dev tools**: ruff, pre-commit, mypy, jupyterlab +- **Testing**: pytest, pytest-benchmark, syrupy (inline snapshots) +- **Docs**: sphinx and related packages + +### Rust Dependencies +- **PyO3**: Python bindings framework +- **egglog**: Core e-graph library +- **egraph-serialize**: Serialization support +- **serde_json**: JSON handling + +## Contributing Guidelines + +When making changes: +1. Update or add tests in `python/tests/` for Python changes +2. Run the full test suite before committing +3. Ensure type checking passes with `make mypy` +4. Build documentation if changing public APIs +5. Follow existing code patterns and style +6. Keep changes minimal and focused + +## Common Patterns + +### Python API Design +- Use `@egraph.class_` decorator for e-graph classes +- Use `@egraph.function` for functions +- Use `@egraph.method` for methods +- Leverage type annotations for better IDE support + +### Rust-Python Integration +- Use PyO3's `#[pyclass]` and `#[pymethods]` macros +- Handle errors with appropriate Python exceptions +- Convert between Rust and Python types in `conversions.rs` + +## Documentation + +Documentation is built with Sphinx: +- Source files in `docs/` +- Build with `make docs` +- Output in `docs/_build/html/` +- Hosted on ReadTheDocs + +## Testing Strategy + +1. **Unit tests**: Test individual functions and classes +2. **Integration tests**: Test complete workflows +3. **Snapshot tests**: Use inline snapshots for complex outputs +4. **Benchmarks**: Performance testing with pytest-benchmark +5. **Type checking**: Validate type stubs and annotations + +## Performance Considerations + +- The library uses Rust for performance-critical operations +- Benchmarking is done via CodSpeed for continuous performance monitoring +- Profile with release builds (`cargo build --release`) when needed + +## Continuous Integration + +GitHub Actions workflows in `.github/workflows/`: +- `CI.yml` - Main testing, type checking, benchmarks, and docs +- `version.yml` - Version management +- `update-changelog.yml` - Changelog automation + +Tests run on Python 3.10, 3.11, 3.12, and 3.13. From bff58138444561a3e2e4734c6a1efd5dc5ab2df8 Mon Sep 17 00:00:00 2001 From: "copilot-swe-agent[bot]" <198982749+Copilot@users.noreply.github.com> Date: Mon, 3 Nov 2025 04:26:14 +0000 Subject: [PATCH 3/5] Fix inaccuracies in Copilot instructions based on code review Co-authored-by: saulshanabrook <1186124+saulshanabrook@users.noreply.github.com> --- .github/copilot-instructions.md | 22 +++++++++++++--------- 1 file changed, 13 insertions(+), 9 deletions(-) diff --git a/.github/copilot-instructions.md b/.github/copilot-instructions.md index 1bf52951..b5d683a6 100644 --- a/.github/copilot-instructions.md +++ b/.github/copilot-instructions.md @@ -22,15 +22,18 @@ This repository provides Python bindings for the Rust library `egglog`, enabling ### Prerequisites - **uv** - Package manager (https://github.com/astral-sh/uv) -- **Rust toolchain** - Version 1.79.0 (specified in rust-toolchain.toml) +- **Rust toolchain** - Version 1.89.0 (specified in rust-toolchain.toml) - **Python** - 3.10+ (see .python-version) ### Common Commands ```bash -# Install dependencies +# Install dependencies (for development) uv sync --extra dev --locked +# Install dependencies (for testing) +uv sync --extra test --locked + # Run tests uv run pytest --benchmark-disable -vvv --durations=10 @@ -77,16 +80,16 @@ See `pyproject.toml` for complete Ruff configuration. ### Testing - Tests are located in `python/tests/` -- Use pytest with snapshot testing (pytest-inline-snapshot) +- Use pytest with snapshot testing (syrupy) - Benchmarks use pytest-benchmark and CodSpeed - Run tests with: `uv run pytest --benchmark-disable -vvv` ## Rust Code Standards ### General Guidelines -- **Edition**: Rust 2024 +- **Edition**: Rust 2024 (experimental) - **FFI**: Uses PyO3 for Python bindings -- **Main library**: Uses egglog from git (currently saulshanabrook/egg-smol branch) +- **Main library**: Uses egglog from git (saulshanabrook/egg-smol, clone-cost branch) ### File Organization - `src/lib.rs` - Main library entry point @@ -123,7 +126,7 @@ Run `pre-commit install` to enable automatic checking. - **Core**: typing-extensions, black, graphviz, anywidget - **Array support**: scikit-learn, array_api_compat, numba, numpy>2 - **Dev tools**: ruff, pre-commit, mypy, jupyterlab -- **Testing**: pytest, pytest-benchmark, syrupy (inline snapshots) +- **Testing**: pytest, pytest-benchmark, pytest-codspeed, pytest-xdist, syrupy (snapshot testing) - **Docs**: sphinx and related packages ### Rust Dependencies @@ -167,9 +170,10 @@ Documentation is built with Sphinx: 1. **Unit tests**: Test individual functions and classes 2. **Integration tests**: Test complete workflows -3. **Snapshot tests**: Use inline snapshots for complex outputs -4. **Benchmarks**: Performance testing with pytest-benchmark -5. **Type checking**: Validate type stubs and annotations +3. **Snapshot tests**: Use syrupy for snapshot testing of complex outputs +4. **Benchmarks**: Performance testing with pytest-benchmark and pytest-codspeed +5. **Parallel testing**: Use pytest-xdist for faster test runs +6. **Type checking**: Validate type stubs and annotations ## Performance Considerations From dd6db0e9f4e151e91cd7b3d2a5c9b31406e7465c Mon Sep 17 00:00:00 2001 From: "copilot-swe-agent[bot]" <198982749+Copilot@users.noreply.github.com> Date: Mon, 3 Nov 2025 04:46:19 +0000 Subject: [PATCH 4/5] Rename to AGENTS.md and address review feedback Co-authored-by: saulshanabrook <1186124+saulshanabrook@users.noreply.github.com> --- .../{copilot-instructions.md => AGENTS.md} | 93 ++++++++++--------- 1 file changed, 48 insertions(+), 45 deletions(-) rename .github/{copilot-instructions.md => AGENTS.md} (64%) diff --git a/.github/copilot-instructions.md b/.github/AGENTS.md similarity index 64% rename from .github/copilot-instructions.md rename to .github/AGENTS.md index b5d683a6..8286f4d6 100644 --- a/.github/copilot-instructions.md +++ b/.github/AGENTS.md @@ -1,4 +1,6 @@ -# GitHub Copilot Instructions for egglog-python +# Agent Instructions for egglog-python + +This file provides instructions for AI coding agents (including GitHub Copilot) working on this repository. ## Project Overview @@ -22,17 +24,14 @@ This repository provides Python bindings for the Rust library `egglog`, enabling ### Prerequisites - **uv** - Package manager (https://github.com/astral-sh/uv) -- **Rust toolchain** - Version 1.89.0 (specified in rust-toolchain.toml) -- **Python** - 3.10+ (see .python-version) +- **Rust toolchain** - See rust-toolchain.toml for version +- **Python** - See .python-version for version ### Common Commands ```bash -# Install dependencies (for development) -uv sync --extra dev --locked - -# Install dependencies (for testing) -uv sync --extra test --locked +# Install dependencies +uv sync --all-extras # Run tests uv run pytest --benchmark-disable -vvv --durations=10 @@ -91,7 +90,7 @@ See `pyproject.toml` for complete Ruff configuration. - **FFI**: Uses PyO3 for Python bindings - **Main library**: Uses egglog from git (saulshanabrook/egg-smol, clone-cost branch) -### File Organization +### Rust File Organization - `src/lib.rs` - Main library entry point - `src/egraph.rs` - E-graph implementation - `src/conversions.rs` - Type conversions between Python and Rust @@ -102,6 +101,31 @@ See `pyproject.toml` for complete Ruff configuration. - `src/termdag.rs` - Term DAG operations - `src/utils.rs` - Utility functions +### Python File Organization + +#### Public Interface +All public Python APIs are exported from the top-level `egglog` module. Anything that is public should be exported in `python/egglog/__init__.py` at the top level. + +#### Lower-Level Bindings +The `egglog.bindings` module provides lower-level access to the Rust implementation for advanced use cases. + +#### Core Python Files +- `python/egglog/__init__.py` - Top-level module exports, defines the public API +- `python/egglog/egraph.py` - Main EGraph class and e-graph management +- `python/egglog/egraph_state.py` - E-graph state and execution management +- `python/egglog/runtime.py` - Runtime system for expression evaluation and method definitions +- `python/egglog/builtins.py` - Built-in types (i64, f64, String, Vec, etc.) and operations +- `python/egglog/declarations.py` - Class, function, and method declaration decorators +- `python/egglog/conversion.py` - Type conversion between Python and egglog types +- `python/egglog/pretty.py` - Pretty printing for expressions and e-graph visualization +- `python/egglog/deconstruct.py` - Deconstruction of Python values into egglog expressions +- `python/egglog/thunk.py` - Lazy evaluation support +- `python/egglog/type_constraint_solver.py` - Type inference and constraint solving +- `python/egglog/config.py` - Configuration settings +- `python/egglog/ipython_magic.py` - IPython/Jupyter integration +- `python/egglog/visualizer_widget.py` - Interactive visualization widget +- `python/egglog/version_compat.py` - Python version compatibility utilities + ## Code Style Preferences 1. **Imports**: Follow Ruff's import sorting @@ -111,30 +135,6 @@ See `pyproject.toml` for complete Ruff configuration. 3. **Comments**: Use clear, explanatory comments for complex logic 4. **Documentation**: Keep docs synchronized with code changes -## Pre-commit Hooks - -The repository uses pre-commit with: -- `ruff-check` with auto-fix -- `ruff-format` for formatting -- `uv-lock` to keep lockfile updated - -Run `pre-commit install` to enable automatic checking. - -## Dependencies - -### Python Dependencies -- **Core**: typing-extensions, black, graphviz, anywidget -- **Array support**: scikit-learn, array_api_compat, numba, numpy>2 -- **Dev tools**: ruff, pre-commit, mypy, jupyterlab -- **Testing**: pytest, pytest-benchmark, pytest-codspeed, pytest-xdist, syrupy (snapshot testing) -- **Docs**: sphinx and related packages - -### Rust Dependencies -- **PyO3**: Python bindings framework -- **egglog**: Core e-graph library -- **egraph-serialize**: Serialization support -- **serde_json**: JSON handling - ## Contributing Guidelines When making changes: @@ -144,15 +144,27 @@ When making changes: 4. Build documentation if changing public APIs 5. Follow existing code patterns and style 6. Keep changes minimal and focused +7. Add a changelog entry in `docs/changelog.md` under the UNRELEASED section ## Common Patterns ### Python API Design -- Use `@egraph.class_` decorator for e-graph classes -- Use `@egraph.function` for functions -- Use `@egraph.method` for methods +- Define e-graph classes by inheriting from `egglog.Expr` +- Use `@egraph.function` decorator for functions +- Use `@egraph.method` decorator for methods - Leverage type annotations for better IDE support +### Working with Values +- Use `get_literal_value(expr)` or the `.value` property to get Python values from primitives +- Use pattern matching with `match`/`case` for destructuring egglog primitives +- Use `get_callable_fn(expr)` to get the underlying Python function from a callable expression +- Use `get_callable_args(expr)` to get arguments to a callable + +### Parallelism +- The underlying Rust library uses Rayon for parallelism +- Control worker thread count via `RAYON_NUM_THREADS` environment variable +- Defaults to single thread if not set + ### Rust-Python Integration - Use PyO3's `#[pyclass]` and `#[pymethods]` macros - Handle errors with appropriate Python exceptions @@ -180,12 +192,3 @@ Documentation is built with Sphinx: - The library uses Rust for performance-critical operations - Benchmarking is done via CodSpeed for continuous performance monitoring - Profile with release builds (`cargo build --release`) when needed - -## Continuous Integration - -GitHub Actions workflows in `.github/workflows/`: -- `CI.yml` - Main testing, type checking, benchmarks, and docs -- `version.yml` - Version management -- `update-changelog.yml` - Changelog automation - -Tests run on Python 3.10, 3.11, 3.12, and 3.13. From 33f1a3e2fa6b8685e70b997dc71036c820bf392f Mon Sep 17 00:00:00 2001 From: Saul Shanabrook Date: Sun, 2 Nov 2025 21:09:09 -0800 Subject: [PATCH 5/5] Clarify agent setup and workflow details --- .github/AGENTS.md | 17 ++++++++++++++--- 1 file changed, 14 insertions(+), 3 deletions(-) diff --git a/.github/AGENTS.md b/.github/AGENTS.md index 8286f4d6..1e0f18d8 100644 --- a/.github/AGENTS.md +++ b/.github/AGENTS.md @@ -24,8 +24,8 @@ This repository provides Python bindings for the Rust library `egglog`, enabling ### Prerequisites - **uv** - Package manager (https://github.com/astral-sh/uv) -- **Rust toolchain** - See rust-toolchain.toml for version -- **Python** - See .python-version for version +- **Rust toolchain** - Version pinned in `rust-toolchain.toml` +- **Python** - Version pinned in `.python-version` ### Common Commands @@ -33,6 +33,9 @@ This repository provides Python bindings for the Rust library `egglog`, enabling # Install dependencies uv sync --all-extras +# Reinstall the Rust extension after changing code in `src/` +uv sync --reinstall-package egglog --all-extras + # Run tests uv run pytest --benchmark-disable -vvv --durations=10 @@ -45,6 +48,10 @@ make stubtest # Build documentation make docs +# Refresh the bundled visualizer assets +make clean +make + # Format code (auto-run by pre-commit) uv run ruff format . @@ -125,6 +132,10 @@ The `egglog.bindings` module provides lower-level access to the Rust implementat - `python/egglog/ipython_magic.py` - IPython/Jupyter integration - `python/egglog/visualizer_widget.py` - Interactive visualization widget - `python/egglog/version_compat.py` - Python version compatibility utilities +- `python/egglog/examples/` - End-to-end samples and tutorials demonstrating the API +- `python/egglog/exp/` - Experimental Array API integrations and code generation helpers + +The compiled extension artifact `python/egglog/bindings.cpython-*.so` is generated by `uv sync` and should not be edited manually. ## Code Style Preferences @@ -144,7 +155,7 @@ When making changes: 4. Build documentation if changing public APIs 5. Follow existing code patterns and style 6. Keep changes minimal and focused -7. Add a changelog entry in `docs/changelog.md` under the UNRELEASED section +7. Ensure the automatic changelog entry in `docs/changelog.md` (added when opening the PR) accurately reflects your change and add manual notes if additional clarification is needed ## Common Patterns