README.md

Brainrot GPT

A character-level transformer that generates Italian brainrot names. Based on Karpathy's microGPT, scaled up with PyTorch.

Trained on 2,000 names like Bombardiro Crocodilo, Tralalelo Tralala, and Spaghettosauro Dinosauro.

Files

File	What it does
gpt.py	Karpathy's original pure-Python GPT (no dependencies)
brainrot-gpt.py	PyTorch version: 4 layers, 64-dim embeddings, 210K params
input.txt	Training data: 1,999 Italian brainrot names
italian.txt	Original smaller dataset (1,064 names)
index.html	Browser demo, runs the full transformer in JavaScript
weights.json	Exported model weights for the browser demo

Quick start

Train from scratch:

python brainrot-gpt.py

Generate names from saved checkpoint:

python brainrot-gpt.py --infer -n 30 -t 0.5

Run the browser demo:

python -m http.server 8765
# open http://localhost:8765

Architecture

The PyTorch model (brainrot-gpt.py) follows GPT-2 with small tweaks:

• 4 transformer layers, 4 attention heads
• 64-dim embeddings, 128 max sequence length
• RMSNorm instead of LayerNorm
• GELU activation in MLP
• Weight tying between token embeddings and output head
• Separate BOS and PAD tokens (the original shares BOS for both)
• Repetition penalty during generation

The original Karpathy version (gpt.py) implements everything from scratch in pure Python, including autograd. One layer, 16-dim embeddings, 5,440 parameters. Educational, not performant.

Browser demo

index.html ports the full transformer inference to JavaScript. Loads weights.json (exported via export_weights.py), runs the forward pass with KV caching, and generates names with adjustable temperature. No dependencies, no build step.

Sample output

Bombettone Gorillone
Mozzarello Cammello
Anguillone Bombardone
Tortenini Cappuccini
Il Manteribo Pandatorino
Bam Bam Girancioismo
Salametto Corvone