Merge pull request #10 from sgrvinod/0.3.0

0.3.0

Author: sgrvinod

CHANGELOG.md

@@ -1,5 +1,26 @@
 # Change Log
 
+## v0.3.0
+
+### Added
+
+* There are 3 new datasets: [ML23c](https://github.com/sgrvinod/chess-transformers#ml23c), [GC22c](https://github.com/sgrvinod/chess-transformers#gc22c), and [ML23d](https://github.com/sgrvinod/chess-transformers#ml23d).
+* A new naming convention for datasets is used. Datasets are now named in the format "[*PGN Fileset*][*Filters*]". For example, *LE1222* is now called [*LE22ct*](https://github.com/sgrvinod/chess-transformers#le22ct), where *LE22* is the name of the PGN fileset from which this dataset was derived, and "*c*", "*t*" are filters for games that ended in checkmates and games that used a specific time control respectively.
+* [*CT-EFT-85*](https://github.com/sgrvinod/chess-transformers#ct-eft-85) is a new trained model with about 85 million parameters.
+* **`chess_transformers.train.utils.get_lr()`** now accepts new arguments, `schedule` and `decay`, to accomodate a new learning rate schedule: exponential decay after warmup.
+* **`chess_transformers.data.prepare_data()`** now handles errors where there is a mismatch between the number of moves and the number of FENs, or when the recorded result in the PGN file was incorrect. Such games are now reported and excluded during dataset creation.
+
+### Changed
+
+* The *LE1222* and *LE1222x* datasets are now renamed to [*LE22ct*](https://github.com/sgrvinod/chess-transformers#le22ct) and [*LE22c*](https://github.com/sgrvinod/chess-transformers#le22c) respectively.
+* All calls to **`chess_transformers.train.utils.get_lr()`** now use the `schedule` and `decay` arguments, even in cases where a user-defined decay is not required.
+* **`chess_transformers.train.datasets.ChessDatasetFT`** was optimized for large datasets. A list of indices for the data split is no longer maintained or indexed in the dataset.
+* Dependencies in [**`setup.py`**](https://github.com/sgrvinod/chess-transformers/blob/main/setup.py) have been updated to newer versions.
+* Fixed an error in **`chess_transformers.play.model_v_model()`** where a move would be attempted by the model playing black even after white won the game with a checkmate.
+* Fixed the `EVAL_GAMES_FOLDER` parameter in the model configuration files pointing to the incorrect folder name **`chess_transformers/eval`** instead of **`chess_transformers/evaluate`**.
+* Fixed an error in **`chess_transformers.evaluate.metrics.elo_delta_margin()`** where the upper limit of the winrate for the confidence interval was not capped at a value of 1.
+* All calls to `torch.load()` now use `weights_only=True` in compliance with its updated API.
+
 ## v0.2.1
 
 ### Changed
@@ -13,7 +34,7 @@
 * **`ChessTransformerEncoderFT`** is an encoder-only transformer that predicts source (*From*) and destination squares (*To*) squares for the next half-move, instead of the half-move in UCI notation.
 * [*CT-EFT-20*](https://github.com/sgrvinod/chess-transformers#ct-eft-20) is a new trained model of this type with about 20 million parameters.
 * **`ChessDatasetFT`** is a PyTorch dataset class for this model type.
-* [**`chess_transformer.data.levels`**](https://github.com/sgrvinod/chess-transformers/blob/main/chess_transformers/data/levels.py) provides a standardized vocabulary (with indices) for oft-used categorical variables. All models and datasets will hereon use this standard vocabulary instead of a dataset-specific vocabulary.
+* [**`chess_transformers.data.levels`**](https://github.com/sgrvinod/chess-transformers/blob/main/chess_transformers/data/levels.py) provides a standardized vocabulary (with indices) for oft-used categorical variables. All models and datasets will hereon use this standard vocabulary instead of a dataset-specific vocabulary.
 
 ### Changed
 

README.md

@@ -0,0 +1,21 @@
+import os
+
+###############################
+############ Name #############
+###############################
+
+NAME = "GC22c"  # name and identifier for this configuration
+
+###############################
+############ Data #############
+###############################
+
+DATA_FOLDER = (
+    os.path.join(os.environ.get("CT_DATA_FOLDER"), NAME)
+    if os.environ.get("CT_DATA_FOLDER")
+    else None
+)  # folder containing all data files
+H5_FILE = NAME + ".h5"  # H5 file containing data
+MAX_MOVE_SEQUENCE_LENGTH = 10  # expected maximum length of move sequences
+EXPECTED_ROWS = 27000000  # expected number of rows, approximately, in the data
+VAL_SPLIT_FRACTION = 0.95  # marker (% into the data) where the validation split begins

chess_transformers/configs/data/GC22c.py

@@ -4,7 +4,7 @@
 ############ Name #############
 ###############################
 
-NAME = "LE1222x"  # name and identifier for this configuration
+NAME = "LE22c"  # name and identifier for this configuration
 
 ###############################
 ############ Data #############

…ess_transformers/configs/data/LE1222x.py chess_transformers/configs/data/LE22c.pychess_transformers/configs/data/LE1222x.py renamed to chess_transformers/configs/data/LE22c.py chess_transformers/configs/data/LE1222x.py renamed to chess_transformers/configs/data/LE22c.py

@@ -4,7 +4,7 @@
 ############ Name #############
 ###############################
 
-NAME = "LE1222"  # name and identifier for this configuration
+NAME = "LE22ct"  # name and identifier for this configuration
 
 ###############################
 ############ Data #############

chess_transformers/configs/data/LE1222.py chess_transformers/configs/data/LE22ct.pychess_transformers/configs/data/LE1222.py renamed to chess_transformers/configs/data/LE22ct.py

@@ -0,0 +1,21 @@
+import os
+
+###############################
+############ Name #############
+###############################
+
+NAME = "ML23c"  # name and identifier for this configuration
+
+###############################
+############ Data #############
+###############################
+
+DATA_FOLDER = (
+    os.path.join(os.environ.get("CT_DATA_FOLDER"), NAME)
+    if os.environ.get("CT_DATA_FOLDER")
+    else None
+)  # folder containing all data files
+H5_FILE = NAME + ".h5"  # H5 file containing data
+MAX_MOVE_SEQUENCE_LENGTH = 10  # expected maximum length of move sequences
+EXPECTED_ROWS = 11000000  # expected number of rows, approximately, in the data
+VAL_SPLIT_FRACTION = 0.925  # marker (% into the data) where the validation split begins

chess_transformers/configs/data/ML23c.py

@@ -0,0 +1,22 @@
+import os
+
+###############################
+############ Name #############
+###############################
+
+NAME = "ML23d"  # name and identifier for this configuration
+
+###############################
+############ Data #############
+###############################
+
+DATA_FOLDER = (
+    os.path.join(os.environ.get("CT_DATA_FOLDER"), NAME)
+    if os.environ.get("CT_DATA_FOLDER")
+    else None
+)  # folder containing all data files
+H5_FILE = NAME + ".h5"  # H5 file containing data
+MAX_MOVE_SEQUENCE_LENGTH = 10  # expected maximum length of move sequences
+EXPECTED_ROWS = 170000000  # expected number of rows, approximately, in the data
+VAL_SPLIT_FRACTION = 0.98  # marker (% into the data) where the validation split begins
+ADD_LOSS_TOKEN = False

chess_transformers/configs/data/ML23d.py

@@ -1 +1 @@
-__all__ = ["LE1222", "LE1222x"]
+__all__ = ["LE22c", "LE22ct", "ML23c", "ML23d", "GC22c"]

chess_transformers/configs/data/__init__.py

@@ -2,7 +2,7 @@
 import pathlib
 
 from chess_transformers.train.utils import get_lr
-from chess_transformers.configs.data.LE1222 import *
+from chess_transformers.configs.data.LE22ct import *
 from chess_transformers.configs.other.stockfish import *
 from chess_transformers.train.datasets import ChessDataset
 from chess_transformers.configs.other.fairy_stockfish import *
@@ -64,10 +64,16 @@
 PRINT_FREQUENCY = 1  # print status once every so many steps
 N_STEPS = 100000  # number of training steps
 WARMUP_STEPS = 8000  # number of warmup steps where learning rate is increased linearly; twice the value in the paper, as in the official transformer repo.
-STEP = 1  # the step number, start from 1 to prevent math error in the next line
+STEP = 1  # the step number, start from 1 to prevent math error in the 'LR' line
+LR_SCHEDULE = "vaswani"  # the learning rate schedule; see utils.py for learning rate schedule
+LR_DECAY = None  # the decay rate for 'exp_decay' schedule
 LR = get_lr(
-    step=STEP, d_model=D_MODEL, warmup_steps=WARMUP_STEPS
-)  # see utils.py for learning rate schedule; twice the schedule in the paper, as in the official transformer repo.
+    step=STEP,
+    d_model=D_MODEL,
+    warmup_steps=WARMUP_STEPS,
+    schedule=LR_SCHEDULE,
+    decay=LR_DECAY,
+)  # see utils.py for learning rate schedule
 START_EPOCH = 0  # start at this epoch
 BETAS = (0.9, 0.98)  # beta coefficients in the Adam optimizer
 EPSILON = 1e-9  # epsilon term in the Adam optimizer
@@ -105,5 +111,5 @@
 ################################
 
 EVAL_GAMES_FOLDER = str(
-    pathlib.Path(__file__).parent.parent.parent.resolve() / "eval" / "games" / NAME
+    pathlib.Path(__file__).parent.parent.parent.resolve() / "evaluate" / "games" / NAME
 )  # folder where evaluation games are saved in PGN files

chess_transformers/configs/models/CT-E-20.py

@@ -2,7 +2,7 @@
 import pathlib
 
 from chess_transformers.train.utils import get_lr
-from chess_transformers.configs.data.LE1222 import *
+from chess_transformers.configs.data.LE22ct import *
 from chess_transformers.configs.other.stockfish import *
 from chess_transformers.train.datasets import ChessDataset
 from chess_transformers.configs.other.fairy_stockfish import *
@@ -64,10 +64,16 @@
 PRINT_FREQUENCY = 1  # print status once every so many steps
 N_STEPS = 100000  # number of training steps
 WARMUP_STEPS = 8000  # number of warmup steps where learning rate is increased linearly; twice the value in the paper, as in the official transformer repo.
-STEP = 1  # the step number, start from 1 to prevent math error in the next line
+STEP = 1  # the step number, start from 1 to prevent math error in the 'LR' line
+LR_SCHEDULE = "vaswani"  # the learning rate schedule; see utils.py for learning rate schedule
+LR_DECAY = None  # the decay rate for 'exp_decay' schedule
 LR = get_lr(
-    step=STEP, d_model=D_MODEL, warmup_steps=WARMUP_STEPS
-)  # see utils.py for learning rate schedule; twice the schedule in the paper, as in the official transformer repo.
+    step=STEP,
+    d_model=D_MODEL,
+    warmup_steps=WARMUP_STEPS,
+    schedule=LR_SCHEDULE,
+    decay=LR_DECAY,
+)  # see utils.py for learning rate schedule
 START_EPOCH = 0  # start at this epoch
 BETAS = (0.9, 0.98)  # beta coefficients in the Adam optimizer
 EPSILON = 1e-9  # epsilon term in the Adam optimizer
@@ -105,5 +111,5 @@
 ################################
 
 EVAL_GAMES_FOLDER = str(
-    pathlib.Path(__file__).parent.parent.parent.resolve() / "eval" / "games" / NAME
+    pathlib.Path(__file__).parent.parent.parent.resolve() / "evaluate" / "games" / NAME
 )  # folder where evaluation games are saved in PGN files