Add EGT model and example

[xnuohz]

Feature summary

• Add the EGTConv layer
• Add unit test
• Add an example for graph property prediciton

Issue

• close Add EGT to nn.conv #8258

Highlight

training log

python examples/egt.py
Downloading http://snap.stanford.edu/ogb/data/graphproppred/csv_mol_download/hiv.zip
Downloaded 0.00 GB: 100%|█████████████████████████████████████████████████████████████████████████████████| 3/3 [00:09<00:00,  3.27s/it]
Extracting /home/ubuntu/Projects/pytorch_geometric/examples/../data/OGB/hiv.zip
Processing...
Loading necessary files...
This might take a while.
Processing graphs...
100%|█████████████████████████████████████████████████████████████████████████████████████████| 41127/41127 [00:00<00:00, 263930.86it/s]
Converting graphs into PyG objects...
100%|██████████████████████████████████████████████████████████████████████████████████████████| 41127/41127 [00:00<00:00, 61382.94it/s]
Saving...
Done!
/home/ubuntu/Softwares/anaconda3/envs/pyg/lib/python3.12/site-packages/ogb/graphproppred/dataset_pyg.py:68: FutureWarning: You are using `torch.load` with `weights_only=False` (the current default value), which uses the default pickle module implicitly. It is possible to construct malicious pickle data which will execute arbitrary code during unpickling (See https://github.com/pytorch/pytorch/blob/main/SECURITY.md#untrusted-models for more details). In a future release, the default value for `weights_only` will be flipped to `True`. This limits the functions that could be executed during unpickling. Arbitrary objects will no longer be allowed to be loaded via this mode unless they are explicitly allowlisted by the user via `torch.serialization.add_safe_globals`. We recommend you start setting `weights_only=True` for any use case where you don't have full control of the loaded file. Please open an issue on GitHub for any issues related to this experimental feature.
  self.data, self.slices = torch.load(self.processed_paths[0])
Epoch: 01, Loss: 0.4270, Val: 0.6325, Test: 0.6910
Epoch: 02, Loss: 0.2092, Val: 0.6568, Test: 0.6858
Epoch: 03, Loss: 0.1587, Val: 0.6474, Test: 0.6911
Epoch: 04, Loss: 0.1465, Val: 0.6846, Test: 0.6892
Epoch: 05, Loss: 0.1426, Val: 0.6150, Test: 0.6738
Epoch: 06, Loss: 0.1405, Val: 0.7039, Test: 0.7146
Epoch: 07, Loss: 0.1380, Val: 0.7364, Test: 0.7132
Epoch: 08, Loss: 0.1373, Val: 0.7253, Test: 0.7001
Epoch: 09, Loss: 0.1360, Val: 0.7337, Test: 0.6883
Epoch: 10, Loss: 0.1354, Val: 0.7341, Test: 0.6904
Epoch: 11, Loss: 0.1319, Val: 0.7715, Test: 0.7248
Epoch: 12, Loss: 0.1307, Val: 0.7337, Test: 0.6999
Epoch: 13, Loss: 0.1296, Val: 0.7803, Test: 0.7254
Epoch: 14, Loss: 0.1284, Val: 0.7255, Test: 0.6946
Epoch: 15, Loss: 0.1280, Val: 0.7483, Test: 0.6612
Epoch: 16, Loss: 0.1267, Val: 0.7377, Test: 0.7013
Epoch: 17, Loss: 0.1259, Val: 0.7613, Test: 0.7384
Epoch: 18, Loss: 0.1245, Val: 0.7732, Test: 0.7123
Epoch: 19, Loss: 0.1241, Val: 0.7705, Test: 0.7033
Epoch: 20, Loss: 0.1237, Val: 0.7469, Test: 0.7100
Epoch: 21, Loss: 0.1208, Val: 0.7621, Test: 0.7347
Epoch: 22, Loss: 0.1215, Val: 0.7347, Test: 0.7340
Epoch: 23, Loss: 0.1208, Val: 0.7162, Test: 0.7341
Epoch: 24, Loss: 0.1203, Val: 0.7747, Test: 0.7454
Epoch: 25, Loss: 0.1187, Val: 0.7531, Test: 0.7242
Epoch: 26, Loss: 0.1184, Val: 0.7602, Test: 0.7200
Epoch: 27, Loss: 0.1172, Val: 0.8005, Test: 0.7247
Epoch: 28, Loss: 0.1178, Val: 0.7705, Test: 0.7576
Epoch: 29, Loss: 0.1153, Val: 0.7326, Test: 0.7365
Epoch: 30, Loss: 0.1156, Val: 0.7746, Test: 0.7255

[codecov]

Codecov Report

❌ Patch coverage is 96.93878% with 3 lines in your changes missing coverage. Please review.
✅ Project coverage is 85.95%. Comparing base (c211214) to head (ece0b9c).
⚠️ Report is 179 commits behind head on master.

Files with missing lines	Patch %	Lines
torch_geometric/nn/models/egt.py	88.00%	3 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #8280      +/-   ##
==========================================
- Coverage   86.11%   85.95%   -0.17%     
==========================================
  Files         496      512      +16     
  Lines       33655    36114    +2459     
==========================================
+ Hits        28981    31040    +2059     
- Misses       4674     5074     +400     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[xnuohz]

hi @puririshi98 plz help review this pr when have band-width, which is also a kind of graph transformer layer, thank you!

[puririshi98]

first step is to remove the example and instead integrate this into ogb_train examples like we have done for other graph transformers. Plz also add an EGT model to torch_geometric.nn.models for use in this example. plz follow previous pattern as polynormer etc including the correct uodates to the readme.

[xnuohz]

root@7df2f109d384:/workspace/pytorch_geometric# python examples/ogbn_train.py --model egt -b 2048 --num_layers 1 -e 5
Training ogbn-arxiv with egt model.
The root is:  ./data/ogbn-arxiv
EGT model requires edge features, using node featuresto initialize edge features.
Total time before training begins took 0.8927s
Training...
Epoch 01:   0%|                                                                  | 0/90941 [00:00<?, ?it/s]/workspace/pytorch_geometric/examples/ogbn_train.py:157: UserWarning: Converting a tensor with requires_grad=True to a scalar may lead to unexpected behavior.
Consider using tensor.detach() first. (Triggered internally at /opt/pytorch/pytorch/torch/csrc/autograd/generated/python_variable_methods.cpp:835.)
  total_loss += float(loss)
Epoch 01: 100%|████████████████████████████████████████████████████| 90941/90941 [01:19<00:00, 1142.89it/s]
Epoch 01, Loss: 2.8869,  Train Time: 79.5714s
Val: 31.55%,
lr:
0.003
Epoch 02: 100%|████████████████████████████████████████████████████| 90941/90941 [01:19<00:00, 1145.58it/s]
Epoch 02, Loss: 2.2352,  Train Time: 79.3845s
Val: 38.75%,
lr:
0.003
Epoch 03: 100%|████████████████████████████████████████████████████| 90941/90941 [01:19<00:00, 1144.59it/s]
Epoch 03, Loss: 2.0655,  Train Time: 79.4533s
Val: 38.63%,
lr:
0.003
Epoch 04: 100%|████████████████████████████████████████████████████| 90941/90941 [01:19<00:00, 1144.99it/s]
Epoch 04, Loss: 1.8601,  Train Time: 79.4251s
Val: 43.80%,
lr:
0.003
Epoch 05: 100%|████████████████████████████████████████████████████| 90941/90941 [01:19<00:00, 1146.30it/s]
Epoch 05, Loss: 1.7414,  Train Time: 79.3344s
Val: 48.17%,
lr:
0.003
Average Epoch Time on training: 79.4338s
Average Epoch Time on inference: 24.0079s
Average Epoch Time: 103.4417s
Median Epoch Time: 103.4662s
Best Validation Accuracy: 48.17%
Testing...
Test Accuracy: 44.67%
Total Program Runtime: 555.1059s

[xnuohz]

@puririshi98 ready for next round review