SMILES augmentation

[sfluegel05]

Idea

There are a lot of different SMILES that describe the same molecule. The SMILES you get depends on different factors: where do you start the traversal? Which direction do you use for rings? How do you sort branches?
So far, we only use the SMILES provided by ChEBI. However, we could get more different SMILES out of ChEBI by applying SMILES augmentation, i.e., adding several SMILES to the data set for each molecule. This can potentially reduce overfitting and help generalisation.

Current state

In PR #52 SMILES augmentation has already been implemented and shown to be successful see wandb. PR #52 is however outdated.

Todo

• On a new branch, take the relevant features from Feature/data augmentation #52 and add them to the current chebai
• Use rdkit to generate new SMILES with rootedAtAtom and doRandom: rdkit -> MolToSmiles
• The user should be able to specify the number of different SMILES per molecule (as a maximum value, not all molecules might be able to reach that maximum)

[aditya0by0]

rdkit/rdkit#6489

• RDKit’s internal random number generator (RNG) is seeded once at program start.
• This means:
  • Runs across different sessions are reproducible (you'll get the same set of randomized SMILES every time you run a script).
    • --- This is what we need augmented data reproducibility
  • Within the same session, calling MolToSmiles(..., doRandom=True) multiple times gives different results because the RNG state keeps advancing.
    • --- This randomness works in our favor

from rdkit import Chem


m = Chem.MolFromSmiles("OCCC(F)CC(CCF)CC(O)(CCCCCl)O")
print(len(set([Chem.MolToSmiles(m, canonical=False, doRandom=True) for i in range(100)])))
print(len(set([Chem.MolToSmiles(m, canonical=False, doRandom=True) for i in range(100)])))
print(len(set([Chem.MolToSmiles(m, canonical=False, doRandom=True) for i in range(100)])))
print(len(set([Chem.MolToSmiles(m, canonical=False, doRandom=True) for i in range(100)])))
print(len(set([Chem.MolToSmiles(m, canonical=False, doRandom=True) for i in range(100)])))
print(len(set([Chem.MolToSmiles(m, canonical=False, doRandom=True) for i in range(100)])))
print(len(set([Chem.MolToSmiles(m, canonical=False, doRandom=True) for i in range(100)])))

(gnn3) G:\github-aditya0by0\python-chebai>G:/anaconda3/envs/gnn3/python.exe g:/github-aditya0by0/python-chebai/test.py
87
89
84
84
90
91
87

(gnn3) G:\github-aditya0by0\python-chebai>G:/anaconda3/envs/gnn3/python.exe g:/github-aditya0by0/python-chebai/test.py
87
89
84
84
90
91
87

Also, setting C's random component through below api has no effect on above behaviour
https://www.rdkit.org/docs/source/rdkit.rdBase.html#rdkit.rdBase.SeedRandomNumberGenerator

[aditya0by0]

Setting	Start Atom	Atom Traversal	Output SMILES	Reproducible
canonical=True (default)	Auto	Canonical	Canonical	✅ Yes
doRandom=True	Random	Random	Random	❌ No (in-session)
rootedAtAtom=X	Atom X	Deterministic	Non-canonical	✅ Yes
rootedAtAtom=X, doRandom=True	Atom X	Random	Randomized (starting at X)	❌ No (in-session)

[aditya0by0]

The documentation says If doRandom=True, it overrides canonical, meaning it will generate random SMILES no matter what canonical was set to.
Similarly for rootedAtAtom if non-negative, this forces the SMILES to start at a particular atom. Defaults to -1. If not -1, overrides canonical setting.

[sfluegel05]

That is good to know. In that case, we don't have to worry about canonicalisation here

[aditya0by0]

We need to revisit the data splitting strategy for augmented SMILES. Currently, augmented versions of the same original molecule appear across train, validation, and test sets. This artificially inflates the macro score (as seen here), since the model can exploit overlaps rather than truly generalize. To ensure a fair evaluation, augmented SMILES derived from the same parent molecule should be kept within the same split.

[aditya0by0]

Please Check the augmentation results here: https://wandb.ai/chebai/chebai/reports/Copy-of-sfluegel-s-SMILES-augmentation--VmlldzoxNDIyMzQ3MQ

Please find summary below:

Setting	Seed	Val Macro-F1	Val Micro-F1	Epoch	Train Loss (epoch)	Train Loss (step)	Train Macro-F1	Train Micro-F1	Global Step	Val Loss (epoch)	Val Loss (step)
Augmentation 15	0	0.6850	0.9110	199	0.0007919	0.0009062	0.9315	0.9899	1,313,599	0.0229558	0.0191848
Augmentation 15	12345	0.6817	0.9106	199	0.0009690	0.0003090	0.9145	0.9881	1,313,599	0.0210955	0.0196686
Augmentation 10	12345	0.6959	0.9129	199	0.0007078	0.0004505	0.9390	0.9913	1,362,799	0.0222750	0.0389156
Augmentation 10	0	0.6868	0.9123	199	0.0008145	0.0003438	0.9294	0.9901	1,362,799	0.0219213	0.0530284
Augmentation 5	12345	0.6868	0.9118	199	0.0009964	0.0010712	0.9125	0.9881	747,799	0.0185963	0.0293011
Augmentation 5	0	0.6685	0.9088	199	0.0012363	0.0013085	0.8899	0.9853	747,799	0.0185530	0.0249622
Original (no aug.)	0	0.6031	0.8890	199	0.0014627	0.0007209	0.8564	0.9830	250,999	0.0210648	0.0444179
Original (no aug.)	123456	0.5794	0.8866	199	0.0018911	0.0026047	0.8042	0.9782	250,999	0.0210222	0.0484128

[aditya0by0]

Although the validation scores indicate performance improvements, I believe it may still be somewhat unfair to directly compare the results. Even if the splits remain consistent between the original and augmented versions, the number of samples within each split differs. Therefore, I would recommend that we also evaluate all models on a common test (or validation) split derived from the original dataset to ensure a fair and consistent comparison.