LLM from Scratch

Personal implementation (from scratch) following Build a Large Language Model (From Scratch) by Sebastian Raschka.

Project structure (planned)

• src/: implementation
• scripts/: training/evaluation/generation scripts
• notebooks/: experiments
• configs/: hyperparameter/config files
• data/: datasets (typically gitignored)

Setup

1. Create a virtual environment.
2. Install dependencies (to be defined).

Usage

Commands and scripts will be added as the implementation grows.

References

• Sebastian Raschka, Build a Large Language Model (From Scratch)