Big big refactor

.cursor/rules/graphppl.mdc

@@ -0,0 +1,34 @@
+---
+description: 
+globs: 
+alwaysApply: true
+---
+# GraphPPL.jl Overview
+
+GraphPPL.jl is a probabilistic programming language focused on probabilistic graphical models. It materializes models as factor graphs and provides tools for model specification.
+
+## Main Components
+
+The main entry point is [src/GraphPPL.jl](mdc:src/GraphPPL.jl), which includes all the core modules.
+
+### Core Structure
+- **Core**: Basic functionality and interfaces
+- **Graph**: Factor graph representation and manipulation
+- **Macros**: DSL for model specification via `@model` macro
+- **Model**: Model representation classes
+- **Nodes**: Graph node representation and behavior
+- **Generators**: Code generation tools
+- **Plugins**: Extension system for backend integration
+- **Utils**: Helper functions and utilities
+
+### Testing
+Tests are organized in the [test](mdc:test) directory, mirroring the source code structure. The test suite uses ReTestItems with `@testitem` blocks for self-contained tests. The [test/runtests.jl](mdc:test/runtests.jl) file is the entry point for running all tests.
+
+### Benchmarking
+Performance benchmarks are in the [benchmark](mdc:benchmark) directory, using BenchmarkTools.jl. Run all benchmarks using [benchmark/benchmarks.jl](mdc:benchmark/benchmarks.jl).
+
+## Naming Conventions
+- Use snake_case for function and variable names
+- Use PascalCase for type names (structs and abstract types)
+- Add comprehensive docstrings to functions and types
+- Use `@kwdef` macro for structs to enable keyword constructors

.cursor/rules/julia.mdc

@@ -0,0 +1,106 @@
+---
+description: 
+globs: 
+alwaysApply: true
+---
+You are an expert in Julia language programming, data science, and numerical computing.
+
+Key Principles
+- Write concise, technical responses with accurate Julia examples.
+- Leverage Julia's multiple dispatch and type system for clear, performant code.
+- Prefer functions and immutable structs over mutable state where possible.
+- Use descriptive variable names with auxiliary verbs (e.g., is_active, has_permission).
+- Use lowercase with underscores for directories and files (e.g., src/data_processing.jl).
+- Favor named exports for functions and types.
+- Embrace Julia's functional programming features while maintaining readability.
+
+Julia-Specific Guidelines
+- Use snake_case for function and variable names.
+- Use PascalCase for type names (structs and abstract types).
+- Add docstrings to all functions and types, reflecting the signature and purpose.
+- Keep docstrings up to date with all the changes and argument signatures.
+- Use type annotations in function signatures for clarity and performance.
+- Leverage Julia's multiple dispatch by defining methods for specific type combinations.
+- Use the `@kwdef` macro for structs to enable keyword constructors.
+- Implement custom `show` methods for user-defined types.
+- Use modules to organize code and control namespace.
+
+Function Definitions
+- Use descriptive names that convey the function's purpose.
+- Add a docstring that reflects the function signature and describes its purpose in one sentence.
+- Update docstrings if implementation changed.
+- Describe the return value in the docstring.
+- Example:
+  ```julia
+  """
+      process_data(data::Vector{Float64}, threshold::Float64)
+
+  Process the input `data` by applying a `threshold` filter and return the filtered result.
+  """
+  function process_data(data::Vector{Float64}, threshold::Float64)
+      # Function implementation
+  end
+  ```
+
+Struct Definitions
+- Always use the `@kwdef` macro to enable keyword constructors.
+- Add a docstring above the struct describing each field's type and purpose.
+- Implement a custom `show` method for better struct printing.
+
+Error Handling and Validation
+- Use Julia's exception system for error handling.
+- Create custom exception types for specific error cases.
+- Use guard clauses to handle preconditions and invalid states early.
+- Implement proper error logging and user-friendly error messages.
+- Example:
+  ```julia
+  struct InvalidInputError <: Exception
+      msg::String
+  end
+
+  function process_positive_number(x::Number)
+      x <= 0 && throw(InvalidInputError("Input must be positive"))
+      # Process the number
+  end
+  ```
+
+Performance Optimization
+- Use type annotations where necessary to avoid type instabilities.
+- Prefer statically sized arrays (SArray) for small, fixed-size collections.
+- Use views (@views macro) to avoid unnecessary array copies.
+- Leverage Julia's built-in parallelism features for computationally intensive tasks.
+- Use benchmarking tools (BenchmarkTools.jl) to identify and optimize bottlenecks.
+- We store benchmarks in the benchmark folder in the root of the repository.
+- Create benchmarks for performance sensitive pieces of code.
+
+Testing
+- For each file in the source code create a test file with the `_tests.jl` suffix, e.g. `src/folder/subfoldeer/file.jl` -> `test/folder/subfolder/file_tests.jl`
+- Create small individual tests in `@testitem` blocks
+- test files should only contain `@testitem` blocks and no extra code. All code within `@testitem` blocks are self-contained scopes.
+- Write test cases of increasing difficulty with comments explaining what is being tested.
+- Use individual `@test` calls for each assertion, not for blocks.
+- Example:
+  ```julia
+@testitem "A function can be called" begin
+    import Package: function
+
+    @test function(2) === 3
+end
+  ```
+
+Dependencies
+- Use the built-in package manager (Pkg) for managing dependencies.
+- Specify version constraints in the Project.toml file.
+- Consider using compatibility bounds (e.g., "Package" = "1.2, 2") to balance stability and updates.
+
+Code Organization
+- Use modules to organize related functionality.
+- Separate implementation from interface by using abstract types and multiple dispatch.
+- Use include() to split large modules into multiple files.
+- Follow a consistent project structure (e.g., src/, test/, docs/).
+
+Documentation
+- Write comprehensive docstrings for all public functions and types.
+- Use Julia's built-in documentation system (Documenter.jl) for generating documentation.
+- Include examples in docstrings to demonstrate usage.
+- Keep documentation up-to-date with code changes.

Project.toml

@@ -4,6 +4,7 @@ authors = ["Wouter Nuijten <[email protected]>", "Dmitry Bagaev <bvdmitri@
 version = "4.6.2"
 
 [deps]
+BipartiteFactorGraphs = "b30351d1-6332-4c89-82f4-633fdf76981a"
 BitSetTuples = "0f2f92aa-23a3-4d05-b791-88071d064721"
 DataStructures = "864edb3b-99cc-5e75-8d2d-829cb0a9cfe8"
 Dictionaries = "85a47980-9c8c-11e8-2b9f-f7ca1fa99fb4"
@@ -27,6 +28,7 @@ GraphPPLGraphVizExt = "GraphViz"
 GraphPPLPlottingExt = ["Cairo", "GraphPlot"]
 
 [compat]
+BipartiteFactorGraphs = "1.1.1"
 BitSetTuples = "1.1"
 Cairo = "1.0"
 DataStructures = "0.18"

src/GraphPPL.jl

@@ -1,20 +1,96 @@
 module GraphPPL
 
 using MacroTools
-
-include("resizable_array.jl")
-include("plugins_collection.jl")
-
-include("graph_engine.jl")
-include("model_generator.jl")
-include("model_macro.jl")
-
+using Static
+using NamedTupleTools
+using Dictionaries
+using BipartiteFactorGraphs
+using BipartiteFactorGraphs.Graphs
+using MetaGraphsNext, MetaGraphsNext.JLD2
+using BitSetTuples
+
+import Base: put!, haskey, getindex, getproperty, setproperty!, setindex!, vec, iterate, showerror, Exception
+import BipartiteFactorGraphs.Graphs: neighbors, degree
+
+export as_node, as_variable, as_context, savegraph, loadgraph
+
+include("core/abstract_types.jl")
+
+# Core components - these have minimal dependencies
+include("core/errors.jl")
+include("core/functional_indices.jl")
+include("core/node_types.jl")
+include("core/resizable_array.jl")
+include("graph/node_labels.jl")
+include("plugins/plugins_collection.jl")
+
+# Export from core
+export NotImplementedError
+export FunctionalIndex, FunctionalRange
+export AbstractBackend, AbstractInterfaces, AbstractInterfaceAliases
+
+# Export basic node/edge labels
+export NodeLabel, EdgeLabel, FactorID
+
+# Basic model structure - needed for context and node creation
+include("model/indexed_variable.jl")
+include("model/proxy_label.jl")
+include("model/context.jl")
+
+# Model creation components - needed before node properties
+include("model/node_creation.jl")
+
+# Node handling - now has proper dependencies
+include("nodes/node_properties.jl")
+include("nodes/node_materialization.jl")
+
+# Graph components - uses node properties
+include("graph/interfaces.jl")
+include("graph/node_data.jl")
+
+# Export from graph
+export NodeData, getcontext, getproperties, getextra, is_factor, is_variable
+
+# Export from nodes
+export VariableNodeProperties, FactorNodeProperties
+export VariableKindRandom, VariableKindData, VariableKindConstant, VariableKindUnknown
+export AnonymousVariable
+
+# Export from model basics
+export Model, NodeCreationOptions, Context
+
+# Additional interface types
+export StaticInterfaces, StaticInterfaceAliases
+
+# Remaining model components
+include("model/model.jl")
+include("model/model_filtering.jl")
+include("model/var_dict.jl")
+include("model/variable_ref.jl")
+
+# Export additional model components
+export VariableRef, IndexedVariable
+export path_to_root, factor_nodes, individual_variables, vector_variables, tensor_variables
+export VarDict
+export ProxyLabel, unroll
+
+include("utils/macro_utils.jl")
+
+# Plugins
+include("plugins/plugin_processing.jl")
 include("plugins/node_created_by.jl")
 include("plugins/node_id.jl")
 include("plugins/node_tag.jl")
 include("plugins/variational_constraints/variational_constraints.jl")
 include("plugins/meta/meta.jl")
 
+# Model generation
+include("generators/model_generator.jl")
+
+# Macros
+include("macros/model_macro.jl")
+
+# Backend
 include("backends/default.jl")
 
 """

src/backends/default.jl

@@ -3,7 +3,7 @@
 
 A default backend that is used in the `GraphPPL.@model` macro when no backend is specified explicitly.
 """
-struct DefaultBackend end
+struct DefaultBackend <: AbstractBackend end
 
 function GraphPPL.model_macro_interior_pipelines(::DefaultBackend)
     return (

src/core/abstract_types.jl

@@ -0,0 +1,37 @@
+abstract type AbstractModel end
+
+abstract type AbstractNodeProperties end
+
+abstract type AbstractNodeData end
+
+abstract type AbstractModelFilterPredicate end
+
+abstract type AbstractVariableReference end
+
+"""
+    AbstractBackend
+
+Base type for all probabilistic model backends. Backends define how node types, behavior, and interfaces
+are interpreted during model creation. Different backends can provide different interpretations of the same
+syntax, enabling specialized handling for different inference algorithms.
+"""
+abstract type AbstractBackend end
+
+"""
+    AbstractInterfaces
+
+Base type for all interfaces definition types. Interfaces define the connection points between 
+nodes in a probabilistic graphical model. Different implementations can provide various ways to
+represent and manipulate these interfaces, such as using static types for compile-time reasoning
+or dynamic types for runtime flexibility.
+"""
+abstract type AbstractInterfaces end
+
+"""
+    AbstractInterfaceAliases
+
+Base type for all interface alias definition types. Interface aliases allow different names to refer
+to the same interface, enabling more flexible and intuitive model specifications. Different 
+implementations can provide various ways to represent and manipulate these aliases.
+"""
+abstract type AbstractInterfaceAliases end

src/core/errors.jl

@@ -0,0 +1,5 @@
+struct NotImplementedError <: Exception
+    message::String
+end
+
+showerror(io::IO, e::NotImplementedError) = print(io, "NotImplementedError: " * e.message)