Skip to content

Latest commit

 

History

History
82 lines (58 loc) · 2.46 KB

CHANGELOG.md

File metadata and controls

82 lines (58 loc) · 2.46 KB

Changelog

All notable changes to this project will be documented in this file.

[0.6.1] - 2025-12-23

Model

  • Add contrastive loss pre-training for graph-level datasets like PCQM4M-v2
  • Add inferring graph-level embeddings with model trained with contrastive loss

Code Refactoring

  • Code refactoring for edge-level example ogbl-ppa

[0.6.0] - 2025-11-19

Model

  • Add generation functionality analogous to discrete diffusion LM after pre-trained with pretrain-mlm objective

Code Refactoring

  • Manage configurations using omegaconf, hydra and yaml files.

Dependencies upgrade

  • Upgrade python to 3.10
  • Upgrade pytorch to 2.5.1
  • Upgrade transformers to 4.53.3

[0.5.0] - 2025-05-15

Model

  • Refactor model architectures
  • Release 4 checkpoints for the PCQM4M-v2 dataset in ModelScope

Other

  • Code refactoring.
  • Update paper.

[0.4.0] - 2024-10-13

Dataset

  • Add edge-level example ogbl-citation2 and ogbl-wikikg2
  • Add node-level example ogbn-products

Other

  • Code refactoring.
  • Update README to include details of Eulerian sequence

[0.3.1] - 2024-08-18

Model

  • Add drop path to regularize large models, and it works quite well for deep models

Other

  • Add one package dependency: timm, to implement EMA
  • Update README to include details of Eulerian sequence and cyclic node re-index.
  • Code refactoring.
  • Tokenization config json refactoring.
  • Update vocab by adding some special tokens, e.g., <bos>, <new>, <mask> and etc.
  • Turn off optimizer offload in deepspeed config to boost the training speed.

[0.3.0] - 2024-07-09

Dataset

  • Add toy examples using dataset TUDataset/reddit_threads.
  • Add src/utils/dataset_utils.py::StructureDataset to generate random non-attributed graphs. for structural understanding pre-training

Dataset processing: tokenization

  • Add a new type of node re-indexing: cyclic.

Model

  • Add two new backbones, i.e., backbones from Google's Bert and OpenAI's GPT-2.

Optimization

Optimization Objective

  • Add a new pre-train objective, i.e., Masked Language Modeling from Bert.

Other

  • Code refactoring.
  • Upgrade two package dependencies: deepspeed==0.14.0 and transformers==4.38.2.
  • Add one package dependency: tensorboardX.