|
| 1 | +# ISL Wrapper Design |
| 2 | + |
| 3 | +This document captures the architecture plan for `import caten.isl as I`, a high-level wrapper over `islpy` that mirrors the Common Lisp prototype in `caten_isl_ref/*.lisp` while embracing Pythonic ergonomics. |
| 4 | + |
| 5 | +## Goals |
| 6 | +- Provide predictable memory discipline on top of `islpy`, automatically inserting copies for destructive APIs and freeing temporary objects via GC-driven finalizers (context merely scopes legality, not lifetime). |
| 7 | +- Offer an idiomatic Python facade (`I.Set`, `I.UnionSet`, etc.) featuring multi-dispatch arithmetic (`Set + Constraint`, `Set + Set`, …) and convenient construction from strings. |
| 8 | +- Preserve access to destructive operations (e.g., ScheduleTree editing) via explicit opt-in handles such as `I.InPlace(obj)`. |
| 9 | +- Maintain strong typing and abstraction boundaries so higher-level modules (`caten.polyhedral`, schedulers) can rely on predictable behavior. |
| 10 | +- Keep the implementation dense but readable: abstraction first, well-typed, minimal boilerplate, leverage meta-programming where helpful. |
| 11 | + |
| 12 | +## Layered Architecture |
| 13 | +1. **Primitive Layer (`IPrim`)** – Thin, possibly auto-generated bindings around `islpy` that expose raw handles and primitive functions (copy/free/read_from_str/etc.). |
| 14 | +2. **Context Layer (`ISLContext`, `I.context()`)** – Provides thread-local scoping and access to primitive contexts; it validates operations happen under `with I.context()` but defers cleanup to object-level finalizers. |
| 15 | +3. **Object Layer (`ISLObject` and subclasses)** – Python objects that wrap primitive handles, enforce context membership, and surface `copy()/free()` semantics. |
| 16 | +4. **Function Layer (`ISLFunction` decorators)** – Declarative description of argument/return qualifiers (`I.Take`, `I.Give`, `I.Keep`, `I.Param`, `I.Null`, `I.InPlace`) that synthesize wrappers calling the primitive layer safely. |
| 17 | +5. **High-Level API (`Set`, `UnionSet`, …)** – Multi-dispatch algebra, user-facing constructors, and ergonomic helpers. |
| 18 | + |
| 19 | +## Context Management (`caten/isl/context.py`) |
| 20 | +- `ISLContext` implements `__enter__/__exit__`, creating the underlying `islpy.Context` (if needed) and pushing itself onto a thread-local stack (`contextvars.ContextVar`). |
| 21 | +- It intentionally does **not** own object lifetimes—GC/finalizers free handles—so exiting a context only tears down the primitive handle and pops the stack. |
| 22 | +- Helper APIs: |
| 23 | + - `ISLContext.current(required=True)` – fetch current context, failing with a descriptive error when `required`. |
| 24 | + - (Later) `ctx.attach_handle(...)` helpers may exist purely for ergonomics, not lifecycle management. |
| 25 | +- Contexts are strictly necessary: constructors or copies performed without an active context raise, preventing untracked allocations. |
| 26 | + |
| 27 | +## Object Model (`caten/isl/obj.py`) |
| 28 | +- `ISLObject` is an abstract base class with: |
| 29 | + - `handle`: opaque primitive pointer tracked via `weakref.finalize`. |
| 30 | + - Flags such as `_in_place`/`_pending_give` to emulate `__isl_take/__isl_keep` semantics without storing contexts. |
| 31 | + - Abstract hooks `copy(self) -> Self` and class-level `free_handle(handle)` used by GC finalizers; explicit `free()` simply triggers the finalizer early. |
| 32 | +- Class-level metadata (e.g., `_prim_copy`, `_prim_free`, `_prim_make_from_str`) is registered in a metaclass, akin to `define-isl-object`. This metadata allows automatic generation of `from_str`, `to_str`, and list-type helpers. |
| 33 | +- Construction helpers: |
| 34 | + - `ISLObject.from_ptr(cls, handle)` – wraps a raw pointer while inheriting the current context requirement from higher layers. |
| 35 | + - `obj.as_take()` – enforces `__isl_take` semantics: returns self if `no_copy_for_next_take`, otherwise returns `self.copy()`. |
| 36 | + - `obj.as_keep()` – returns self but validates that the object isn’t compromised. |
| 37 | +- `I.InPlace(obj)` is implemented as a lightweight proxy (`ISLInPlaceView`) that flips `no_copy_for_next_take` and tracks compromised state once consumed. |
| 38 | + |
| 39 | +## Function Binding (`caten/isl/func.py`) |
| 40 | +- `ISLFunction` acts both as registry and decorator: |
| 41 | + - `@ISLFunction.redefine("isl_union_map_apply_range")` inspects the wrapped Python function signature and annotations. |
| 42 | + - Qualifier annotations (`I.Take(IPrim.UnionMap)`, `I.Give(...)`, `I.Param(int)`, `I.Null()`) describe how each argument should be handled. |
| 43 | + - During registration, a wrapper is synthesized: it validates runtime types, injects the current context, performs the necessary `.as_take()` / `.as_keep()` / parameter conversions, calls the `islpy` primitive, then post-processes the result via `infer_result_wrapper` (similar to the Lisp macro). |
| 44 | + - Return annotations drive wrapping: `I.Take(IPrim.UnionMap)` means the primitive returns a give-pointer and the wrapper should call `ISLObject.from_ptr` on the corresponding class. |
| 45 | +- The decorator keeps metadata (original name, primitive symbol, argument descriptors) for documentation and potential auto-generation of higher-level operators. |
| 46 | +- Support utilities: |
| 47 | + - `class ISLQualifier`: base for `Take/Give/Keep/Param/Null/Context`. Each exposes `prepare(value, idx, ctx)` and `wrap(result, ctx)` hooks. |
| 48 | + - Error reporting is precise (argument index, expected class, context state) to ease debugging. |
| 49 | + |
| 50 | +## Memory Discipline |
| 51 | +- Every `ISLObject` installs a `weakref.finalize` hook that calls the subclass-provided `free_handle`. Garbage collection is therefore the primary freeing mechanism; `ISLContext` simply enforces scoping and provides primitive handles. |
| 52 | +- Destructive APIs: |
| 53 | + - By default, passing an object into a `Take` slot clones it (`copy()`) before giving the pointer to `islpy`, ensuring the caller retains an uncompromised copy. |
| 54 | + - Users can opt into destructive semantics via `I.InPlace(obj)` or by calling `.compromise()` explicitly; once compromised, the wrapper prohibits further use unless reinitialized. |
| 55 | +- Non-ISL parameters (`I.Param`) are passed through after type validation, enabling functions that accept Python primitives or other wrappers. |
| 56 | + |
| 57 | +## High-Level Pythonic API (`caten/isl/specs/*.py`) |
| 58 | +- Each user-facing class (`Set`, `UnionSet`, `Constraint`, etc.) subclasses `ISLObject` and plugs into the metadata table, exposing constructors such as `Set.from_str("{ S[i] : 0 <= i < n }")`. |
| 59 | +- Operator overloading relies on multi-dispatch: |
| 60 | + - Use `functools.singledispatchmethod` or a custom registry keyed by `(lhs_cls, op, rhs_cls)` to resolve to the correct `ISLFunction` wrapper. |
| 61 | + - Example: `Set.__add__(self, other)` dispatches on type of `other` and selects between `isl_union_set_union`, `isl_set_add_constraint`, etc., automatically handling qualifiers. |
| 62 | +- Chaining with Python objects is supported via `I.Param`: e.g., `Set & "0 <= i < N"` wraps the string into a temporary `Constraint` via the registered constructor. |
| 63 | +- ScheduleTree and other advanced structures will reuse the same layers, benefitting from consistent context enforcement. |
| 64 | + |
| 65 | +## Implementation Roadmap |
| 66 | +1. **Scaffolding** – Implement `ISLContext` (thread-local gating) and minimal `ISLObject` base with GC finalizers. |
| 67 | +2. **Qualifier System** – Define `I.Take/Give/Keep/Param/Null/InPlace` dataclasses plus runtime helpers. |
| 68 | +3. **ISLFunction Decorator** – Build the registry, argument parser, and wrapper synthesis (initially manual, later auto-generated from specs if desired). |
| 69 | +4. **Core Object Types** – Port essential ISL objects (Set, UnionSet, Map, Constraint) using metadata similar to `define-isl-object`. |
| 70 | +5. **Operator Layer** – Introduce multi-dispatch arithmetic and friendly constructors inside `caten/isl/specs/`. |
| 71 | +6. **Advanced Features** – ScheduleTree operations, context-aware auto-GC tuning, optional tracing/logging. |
| 72 | + |
| 73 | +## Open Questions |
| 74 | +- How much of `islpy` should be auto-generated from `caten_isl_ref/{function-specs,object-specs}.lisp` versus hand-written? A script may parse these files to minimize drift. |
| 75 | +- Should contexts be nestable with resource inheritance (child contexts reusing parent handles), or should each context be isolated? Current assumption: stack discipline with inheritance of the primitive `islpy.Context` handle. |
| 76 | +- Error handling strategy when finalizers fail (e.g., double free) – currently logged via `warnings`, but should configurable escalation exist? |
| 77 | +- How to expose tracing/profiling hooks for debugging context leaks without penalizing release builds? |
0 commit comments