|
| 1 | +# Agent Instructions for egglog-python |
| 2 | + |
| 3 | +This file provides instructions for AI coding agents (including GitHub Copilot) working on this repository. |
| 4 | + |
| 5 | +## Project Overview |
| 6 | + |
| 7 | +This repository provides Python bindings for the Rust library `egglog`, enabling the use of e-graphs in Python for optimization, symbolic computation, and analysis. It is a hybrid project combining: |
| 8 | +- **Python code** in `python/egglog/` - The main Python API and library |
| 9 | +- **Rust code** in `src/` - PyO3-based bindings to the egglog Rust library |
| 10 | +- **Documentation** in `docs/` - Sphinx-based documentation |
| 11 | + |
| 12 | +## Repository Structure |
| 13 | + |
| 14 | +- `python/egglog/` - Main Python package source code |
| 15 | +- `python/tests/` - Python test suite (pytest-based) |
| 16 | +- `src/` - Rust source code for Python bindings (PyO3) |
| 17 | +- `docs/` - Documentation source files (Sphinx) |
| 18 | +- `test-data/` - Test data files |
| 19 | +- `pyproject.toml` - Python project configuration and dependencies |
| 20 | +- `Cargo.toml` - Rust project configuration |
| 21 | +- `uv.lock` - Locked dependencies (managed by uv) |
| 22 | + |
| 23 | +## Build and Development Commands |
| 24 | + |
| 25 | +### Prerequisites |
| 26 | +- **uv** - Package manager (https://github.com/astral-sh/uv) |
| 27 | +- **Rust toolchain** - Version pinned in `rust-toolchain.toml` |
| 28 | +- **Python** - Version pinned in `.python-version` |
| 29 | + |
| 30 | +### Common Commands |
| 31 | + |
| 32 | +```bash |
| 33 | +# Install dependencies |
| 34 | +uv sync --all-extras |
| 35 | + |
| 36 | +# Reinstall the Rust extension after changing code in `src/` |
| 37 | +uv sync --reinstall-package egglog --all-extras |
| 38 | + |
| 39 | +# Run tests |
| 40 | +uv run pytest --benchmark-disable -vvv --durations=10 |
| 41 | + |
| 42 | +# Type checking with mypy |
| 43 | +make mypy |
| 44 | + |
| 45 | +# Stub testing |
| 46 | +make stubtest |
| 47 | + |
| 48 | +# Build documentation |
| 49 | +make docs |
| 50 | + |
| 51 | +# Refresh the bundled visualizer assets |
| 52 | +make clean |
| 53 | +make |
| 54 | + |
| 55 | +# Format code (auto-run by pre-commit) |
| 56 | +uv run ruff format . |
| 57 | + |
| 58 | +# Lint code (auto-run by pre-commit) |
| 59 | +uv run ruff check --fix . |
| 60 | +``` |
| 61 | + |
| 62 | +## Python Code Standards |
| 63 | + |
| 64 | +### General Guidelines |
| 65 | +- **Line length**: 120 characters maximum |
| 66 | +- **Type hints**: Use type annotations for public APIs and functions |
| 67 | +- **Formatting**: Use Ruff for code formatting and linting |
| 68 | +- **Testing**: Write tests using pytest in `python/tests/` |
| 69 | +- **Docstrings**: Use clear, concise docstrings for public functions and classes |
| 70 | + |
| 71 | +### Ruff Configuration |
| 72 | +The project uses Ruff for linting and formatting with specific rules: |
| 73 | +- Allows uppercase variable names (N806, N802) |
| 74 | +- Allows star imports (F405, F403) |
| 75 | +- Allows `exec` and subprocess usage (S102, S307, S603) |
| 76 | +- Allows `Any` type annotations (ANN401) |
| 77 | +- Test files don't require full type annotations |
| 78 | + |
| 79 | +See `pyproject.toml` for complete Ruff configuration. |
| 80 | + |
| 81 | +### Type Checking |
| 82 | +- **mypy** is used for static type checking |
| 83 | +- Run `make mypy` to type check Python code |
| 84 | +- Run `make stubtest` to validate type stubs against runtime behavior |
| 85 | +- Exclusions: `__snapshots__`, `_build`, `conftest.py` |
| 86 | + |
| 87 | +### Testing |
| 88 | +- Tests are located in `python/tests/` |
| 89 | +- Use pytest with snapshot testing (syrupy) |
| 90 | +- Benchmarks use pytest-benchmark and CodSpeed |
| 91 | +- Run tests with: `uv run pytest --benchmark-disable -vvv` |
| 92 | + |
| 93 | +## Rust Code Standards |
| 94 | + |
| 95 | +### General Guidelines |
| 96 | +- **Edition**: Rust 2024 (experimental) |
| 97 | +- **FFI**: Uses PyO3 for Python bindings |
| 98 | +- **Main library**: Uses egglog from git (saulshanabrook/egg-smol, clone-cost branch) |
| 99 | + |
| 100 | +### Rust File Organization |
| 101 | +- `src/lib.rs` - Main library entry point |
| 102 | +- `src/egraph.rs` - E-graph implementation |
| 103 | +- `src/conversions.rs` - Type conversions between Python and Rust |
| 104 | +- `src/py_object_sort.rs` - Python object handling |
| 105 | +- `src/extract.rs` - Extraction functionality |
| 106 | +- `src/error.rs` - Error handling |
| 107 | +- `src/serialize.rs` - Serialization support |
| 108 | +- `src/termdag.rs` - Term DAG operations |
| 109 | +- `src/utils.rs` - Utility functions |
| 110 | + |
| 111 | +### Python File Organization |
| 112 | + |
| 113 | +#### Public Interface |
| 114 | +All public Python APIs are exported from the top-level `egglog` module. Anything that is public should be exported in `python/egglog/__init__.py` at the top level. |
| 115 | + |
| 116 | +#### Lower-Level Bindings |
| 117 | +The `egglog.bindings` module provides lower-level access to the Rust implementation for advanced use cases. |
| 118 | + |
| 119 | +#### Core Python Files |
| 120 | +- `python/egglog/__init__.py` - Top-level module exports, defines the public API |
| 121 | +- `python/egglog/egraph.py` - Main EGraph class and e-graph management |
| 122 | +- `python/egglog/egraph_state.py` - E-graph state and execution management |
| 123 | +- `python/egglog/runtime.py` - Runtime system for expression evaluation and method definitions |
| 124 | +- `python/egglog/builtins.py` - Built-in types (i64, f64, String, Vec, etc.) and operations |
| 125 | +- `python/egglog/declarations.py` - Class, function, and method declaration decorators |
| 126 | +- `python/egglog/conversion.py` - Type conversion between Python and egglog types |
| 127 | +- `python/egglog/pretty.py` - Pretty printing for expressions and e-graph visualization |
| 128 | +- `python/egglog/deconstruct.py` - Deconstruction of Python values into egglog expressions |
| 129 | +- `python/egglog/thunk.py` - Lazy evaluation support |
| 130 | +- `python/egglog/type_constraint_solver.py` - Type inference and constraint solving |
| 131 | +- `python/egglog/config.py` - Configuration settings |
| 132 | +- `python/egglog/ipython_magic.py` - IPython/Jupyter integration |
| 133 | +- `python/egglog/visualizer_widget.py` - Interactive visualization widget |
| 134 | +- `python/egglog/version_compat.py` - Python version compatibility utilities |
| 135 | +- `python/egglog/examples/` - End-to-end samples and tutorials demonstrating the API |
| 136 | +- `python/egglog/exp/` - Experimental Array API integrations and code generation helpers |
| 137 | + |
| 138 | +The compiled extension artifact `python/egglog/bindings.cpython-*.so` is generated by `uv sync` and should not be edited manually. |
| 139 | + |
| 140 | +## Code Style Preferences |
| 141 | + |
| 142 | +1. **Imports**: Follow Ruff's import sorting |
| 143 | +2. **Naming**: |
| 144 | + - Python: snake_case for functions and variables, PascalCase for classes |
| 145 | + - Rust: Follow standard Rust conventions |
| 146 | +3. **Comments**: Use clear, explanatory comments for complex logic |
| 147 | +4. **Documentation**: Keep docs synchronized with code changes |
| 148 | + |
| 149 | +## Contributing Guidelines |
| 150 | + |
| 151 | +When making changes: |
| 152 | +1. Update or add tests in `python/tests/` for Python changes |
| 153 | +2. Run the full test suite before committing |
| 154 | +3. Ensure type checking passes with `make mypy` |
| 155 | +4. Build documentation if changing public APIs |
| 156 | +5. Follow existing code patterns and style |
| 157 | +6. Keep changes minimal and focused |
| 158 | +7. Ensure the automatic changelog entry in `docs/changelog.md` (added when opening the PR) accurately reflects your change and add manual notes if additional clarification is needed |
| 159 | + |
| 160 | +## Common Patterns |
| 161 | + |
| 162 | +### Python API Design |
| 163 | +- Define e-graph classes by inheriting from `egglog.Expr` |
| 164 | +- Use `@egraph.function` decorator for functions |
| 165 | +- Use `@egraph.method` decorator for methods |
| 166 | +- Leverage type annotations for better IDE support |
| 167 | + |
| 168 | +### Working with Values |
| 169 | +- Use `get_literal_value(expr)` or the `.value` property to get Python values from primitives |
| 170 | +- Use pattern matching with `match`/`case` for destructuring egglog primitives |
| 171 | +- Use `get_callable_fn(expr)` to get the underlying Python function from a callable expression |
| 172 | +- Use `get_callable_args(expr)` to get arguments to a callable |
| 173 | + |
| 174 | +### Parallelism |
| 175 | +- The underlying Rust library uses Rayon for parallelism |
| 176 | +- Control worker thread count via `RAYON_NUM_THREADS` environment variable |
| 177 | +- Defaults to single thread if not set |
| 178 | + |
| 179 | +### Rust-Python Integration |
| 180 | +- Use PyO3's `#[pyclass]` and `#[pymethods]` macros |
| 181 | +- Handle errors with appropriate Python exceptions |
| 182 | +- Convert between Rust and Python types in `conversions.rs` |
| 183 | + |
| 184 | +## Documentation |
| 185 | + |
| 186 | +Documentation is built with Sphinx: |
| 187 | +- Source files in `docs/` |
| 188 | +- Build with `make docs` |
| 189 | +- Output in `docs/_build/html/` |
| 190 | +- Hosted on ReadTheDocs |
| 191 | + |
| 192 | +## Testing Strategy |
| 193 | + |
| 194 | +1. **Unit tests**: Test individual functions and classes |
| 195 | +2. **Integration tests**: Test complete workflows |
| 196 | +3. **Snapshot tests**: Use syrupy for snapshot testing of complex outputs |
| 197 | +4. **Benchmarks**: Performance testing with pytest-benchmark and pytest-codspeed |
| 198 | +5. **Parallel testing**: Use pytest-xdist for faster test runs |
| 199 | +6. **Type checking**: Validate type stubs and annotations |
| 200 | + |
| 201 | +## Performance Considerations |
| 202 | + |
| 203 | +- The library uses Rust for performance-critical operations |
| 204 | +- Benchmarking is done via CodSpeed for continuous performance monitoring |
| 205 | +- Profile with release builds (`cargo build --release`) when needed |
0 commit comments