GPT from Scratch !

Following along Andrej Karpathy's video on building GPT from scratch, but with my own experimentation.

Currently working on medium sized and larger data with other tokenization methods. Plan to eventually restructure the GitHub folder with /models, /notes and /plots folders.

Project Structure

• /scripts - Training scripts
• /data - Datasets or download from Tiny Codes dataset

References

🔗 Andrej Karpathy's GPT Video - here
🔗 "Attention Is All You Need" Paper - here