Merge pull request #30 from ChEB-AI/documentation

Documentation for Augmented graphs and GNI

Author: sfluegel05

README.md

@@ -75,3 +75,114 @@ The list can be found in the `configs/data/chebi50_graph_properties.yml` file.
 ```bash
 python -m chebai fit --trainer=configs/training/default_trainer.yml --trainer.logger=configs/training/csv_logger.yml --model=../python-chebai-graph/configs/model/gnn_res_gated.yml --model.train_metrics=configs/metrics/micro-macro-f1.yml --model.test_metrics=configs/metrics/micro-macro-f1.yml --model.val_metrics=configs/metrics/micro-macro-f1.yml --data=../python-chebai-graph/configs/data/chebi50_graph_properties.yml --data.init_args.batch_size=128 --trainer.accumulate_grad_batches=4 --data.init_args.num_workers=10 --model.pass_loss_kwargs=false --data.init_args.chebi_version=241 --trainer.min_epochs=200 --trainer.max_epochs=200 --model.criterion=configs/loss/bce.yml
 ```
+
+## Augmented Graphs
+
+
+Graph Neural Networks (GNNs) often fail to explicitly leverage the chemically meaningful substructures present within molecules (i.e. **functional groups (FGs)**).  To make this implicit information explicitly accessible to GNNs, we augment molecular graphs with **artificial nodes** that represent these substructures. The resulting graph are referred to as **augmented graphs**.
+> Note: Rings are also treated as functional groups in our work.
+
+In these augmented graphs, each functional group node is connected to the atoms that constitute the group. Additionally, two functional group nodes are connected if any atom belonging to one group shares a bond with an atom from the other group. We further introduce a **graph node**, an extra node connected to all functional group nodes.
+
+Among all the connection schemes we evaluated, this configuration delivered the strongest performance. We denote it using the abbreviation **WFG_WFGE_WGN** in our work and is shown in below figure.
+
+<img width="1220" height="668" alt="mol_to_aug_mol" src="https://github.com/user-attachments/assets/0aba6b80-765b-45a6-913a-7d628f14a5db" />
+
+</br>
+</br>
+
+Below is the command for the model and data configuration that achieved the best classification performance using augmented graphs.
+
+```bash
+python -m chebai fit --trainer=configs/training/default_trainer.yml --trainer.logger=configs/training/wandb_logger.yml --model=../python-chebai-graph/configs/model/gat_aug_amgpool.yml --model.train_metrics=configs/metrics/micro-macro-f1.yml --model.test_metrics=configs/metrics/micro-macro-f1.yml --model.val_metrics=configs/metrics/micro-macro-f1.yml --model.config.v2=True --data=../python-chebai-graph/configs/data/chebi50_aug_prop_as_per_node.yml --data.init_args.batch_size=128 --trainer.accumulate_grad_batches=4 --data.init_args.num_workers=10 --model.pass_loss_kwargs=false --data.init_args.chebi_version=241 --trainer.min_epochs=200 --trainer.max_epochs=200 --model.criterion=configs/loss/bce.yml --trainer.logger.init_args.name=gatv2_amg_s0
+```
+
+### Model Hyperparameters
+
+#### **GAT Architecture**
+
+To use a GAT-based model, choose **one** of the following configs:
+
+- **Standard Pooling**: `--model=../python-chebai-graph/configs/model/gat.yml`
+   > Standard pooling sums the learned representations from all the nodes to produce a single representation which is used for classification.  
+
+- **Atom-Augmented Node Pooling**: `--model=../python-chebai-graph/configs/model/gat_aug_aagpool.yml`
+   > With this pooling stratergy, the learned representations are first separated into **two distinct sets**: those from atom nodes and those from all artificial nodes (both functional groups and the graph node). The representations within each set are aggregated separately (using summation) to yield two distinct single vectors. These two resulting vectors are then concatenated before being passed to the classification layer.
+
+- **Atom–Motif–Graph Node Pooling**: `--model=../python-chebai-graph/configs/model/gat_aug_amgpool.yml`
+   >  This approach employs a finer granularity of separation, distinguishing learned representations into **three distinct sets**: atom nodes, Functional Group (FG) nodes, and the single graph node. Summation is performed separately on the atom node set and the FG node set, yielding two vectors. These two vectors are then concatenated along with the single vector corresponding to the graph node before the final linear layer.
+
+#### GAT-specific hyperparameters
+
+- **Number of message-passing layers**: `--model.config.num_layers=5`        (default: 4)
+- **Attention heads**: `--model.config.heads=4`             (default: 8)
+  > **Note**: The number of heads should be divisible by the output channels (or hidden channels if output channels are not specified).
+- **Use GATv2**: `--model.config.v2=True`             (default: False)
+  > **Note**: GATv2 addresses the limitation of static attention in GAT by introducing a dynamic attention mechanism. For further details, please refer to the [original GATv2 paper](https://arxiv.org/abs/2105.14491).
+  
+#### **ResGated Architecture**
+
+To use a ResGated GNN model, choose **one** of the following configs:
+
+- **Atom–Motif–Graph Node Pooling**: `--model=../python-chebai-graph/configs/model/res_aug_amgpool.yml`
+- **Atom-Augmented Node Pooling**: `--model=../python-chebai-graph/configs/model/res_aug_aagpool.yml`
+- **Standard Pooling**: `--model=../python-chebai-graph/configs/model/resgated.yml`
+
+#### **Common Hyperparameters**
+
+These can be used for both GAT and ResGated architectures:
+
+- **Dropout**: `--model.config.dropout=0.1`         (default: 0)
+- **Number of final linear layers**: `--model.n_linear_layers=2`         (default: 1)
+
+## Random Node Initialization
+
+### Static Node Initialization
+
+In this type of node initialization, the node features (and/or edge features) of the given molecular graph are initialized only once during dataset creation with the given initialization scheme.
+
+```bash
+python -m chebai fit --trainer=configs/training/default_trainer.yml --trainer.logger=configs/training/wandb_logger.yml --model=../python-chebai-graph/configs/model/resgated.yml --model.config.in_channels=203 --model.config.edge_dim=11 --model.train_metrics=configs/metrics/micro-macro-f1.yml --model.test_metrics=configs/metrics/micro-macro-f1.yml --model.val_metrics=configs/metrics/micro-macro-f1.yml --data=../python-chebai-graph/configs/data/chebi50_graph_properties.yml --data.pad_node_features=45 --data.pad_edge_features=4 --data.init_args.batch_size=128 --trainer.accumulate_grad_batches=4 --data.init_args.num_workers=10 --data.init_args.persistent_workers=False --model.pass_loss_kwargs=false --data.init_args.chebi_version=241 --trainer.min_epochs=200 --trainer.max_epochs=200 --model.criterion=configs/loss/bce.yml --trainer.logger.init_args.name=gni_res_props+zeros_s0
+```
+
+In the above command, for each node we use the 158 node features (corresponding the node properties defined in `chebi50_graph_properties.yml`) which are retrieved from RDKit and additional 45 additional features (specified by `--data.pad_node_features=45`) drawn from a normal distribution (default).
+
+You can change the distribution from which additional are drawn by using the following config in above command: `--data.distribution=zeros`
+
+Available distributions: `"normal", "uniform", "xavier_normal", "xavier_uniform", "zeros"`
+
+
+Similarly, each edge is initialized with 7 RDKit features and 4 additional features drawn from the given distribution.
+
+
+If you want all node (and edge) features to be drawn from a given distribution (i.e., ignore RDKit features), use: `--data=../python-chebai-graph/configs/data/chebi50_static_gni.yml`
+
+
+Refer to the data class code for details.
+
+
+### Dynamic Node Initialization
+
+In this type of node initialization, the node features (and/or edge features) of the molecular graph are initialized at **each forward pass** of the model using the given initialization scheme.
+
+
+
+Currently, dynamic node initialization is implemented only for the **resgated** architecture by specifying: `--model=../python-chebai-graph/configs/model/resgated_dynamic_gni.yml`
+
+To keep RDKit features and *add* dynamically initialized features use the following config in the command:
+
+```
+--model.config.complete_randomness=False
+--model.config.pad_node_features=45
+```
+
+The additional features are drawn from normal distribution (default). You can change it using:`--model.config.distribution=uniform`
+
+If all features should be initialized from the given distribution, remove the complete_randomness flag (default is True).
+
+
+Please find below the command for a typical dynamic node initialization:
+
+```bash
+python -m chebai fit --trainer=configs/training/default_trainer.yml --trainer.logger=configs/training/wandb_logger.yml --model=../python-chebai-graph/configs/model/resgated_dynamic_gni.yml --model.config.in_channels=203 --model.config.edge_dim=11 --model.config.complete_randomness=False --model.config.pad_node_features=45 --model.config.pad_edge_features=4 --model.train_metrics=configs/metrics/micro-macro-f1.yml --model.test_metrics=configs/metrics/micro-macro-f1.yml --model.val_metrics=configs/metrics/micro-macro-f1.yml --data=../python-chebai-graph/configs/data/chebi50_graph_properties.yml --data.init_args.batch_size=128 --trainer.accumulate_grad_batches=4 --data.init_args.num_workers=10 --data.init_args.persistent_workers=False --model.pass_loss_kwargs=false --data.init_args.chebi_version=241 --trainer.min_epochs=200 --trainer.max_epochs=200 --model.criterion=configs/loss/bce.yml --trainer.logger.init_args.name=gni_dres_props+rand_s0
+```

chebai_graph/models/dynamic_gni.py

@@ -1,3 +1,23 @@
+"""
+ResGatedDynamicGNIGraphPred
+------------------------------------------------
+
+Module providing a ResGated GNN model that applies Random Node Initialization
+(RNI) dynamically at each forward pass. This follows the approach from:
+
+Abboud, R., et al. (2020). "The surprising power of graph neural networks with
+random node initialization." arXiv preprint arXiv:2010.01179.
+
+The module exposes:
+- ResGatedDynamicGNI: a model that can either completely replace node/edge
+  features with random tensors each forward pass or pad existing features with
+  additional random features.
+- ResGatedDynamicGNIGraphPred: a thin wrapper that instantiates the above for
+  graph-level prediction pipelines.
+"""
+
+__all__ = ["ResGatedDynamicGNIGraphPred"]
+
 from typing import Any
 
 import torch
@@ -14,12 +34,37 @@
 
 class ResGatedDynamicGNI(GraphModelBase):
     """
-    Base model class for applying ResGatedGraphConv layers to graph-structured data
-    with dynamic initialization of features for nodes and edges.
-
-    Args:
-        config (dict): Configuration dictionary containing model hyperparameters.
-        **kwargs: Additional keyword arguments for parent class.
+    ResGated GNN with dynamic Random Node Initialization (RNI).
+
+    This model supports two modes controlled by the `config`:
+
+    - complete_randomness (bool-like): If True, **replace** node and edge
+      features entirely with randomly initialized tensors each forward pass.
+      If False, the model **pads** existing features with extra randomly
+      initialized features on-the-fly.
+
+    - pad_node_features (int, optional): Number of random columns to append
+      to each node feature vector when `complete_randomness` is False.
+
+    - pad_edge_features (int, optional): Number of random columns to append
+      to each edge feature vector when `complete_randomness` is False.
+
+    - distribution (str): Distribution for random initialization. Must be one
+      of RandomFeatureInitializationReader.DISTRIBUTIONS.
+
+    Parameters
+    ----------
+    config : Dict[str, Any]
+        Configuration dictionary containing model hyperparameters. Expected keys
+        used by this class:
+            - distribution (optional, default "normal")
+            - complete_randomness (optional, default "True")
+            - pad_node_features (optional, int)
+            - pad_edge_features (optional, int)
+        Keys required by GraphModelBase (e.g., in_channels, hidden_channels,
+        out_channels, num_layers, edge_dim) should also be present.
+    **kwargs : Any
+        Additional keyword arguments forwarded to GraphModelBase.
     """
 
     def __init__(self, config: dict[str, Any], **kwargs: Any):
@@ -96,6 +141,8 @@ def forward(self, batch: dict[str, Any]) -> Tensor:
 
         new_x = None
         new_edge_attr = None
+
+        # If replacing features entirely with random values
         if self.complete_randomness:
             new_x = torch.empty(
                 graph_data.x.shape[0], graph_data.x.shape[1], device=self.device
@@ -110,6 +157,8 @@ def forward(self, batch: dict[str, Any]) -> Tensor:
             RandomFeatureInitializationReader.random_gni(
                 new_edge_attr, self.distribution
             )
+
+        # If padding existing features with additional random columns
         else:
             if self.pad_node_features is not None:
                 pad_node = torch.empty(