Generative Pretrained Transformer (GPT)

This repo contains a naive PyTorch implementation of the GPT model. It is trained on the Tiny-Shakespeare Dataset.

File Structure

├──bigram.py                                --> bigram language model
├──gpt.py                                   --> transformer decoder-only model
└──input.txt                                --> shakespeare training data

Transformer Architecture