Add README with submission instructions for Homework 1

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

Author: eugenevinitsky

homeworks/homework_1/README.md

@@ -0,0 +1,60 @@
+# Homework 1: Value-Based Reinforcement Learning
+
+This homework covers tabular and deep Q-learning methods.
+
+## Structure
+
+- `Homework_1_theory.pdf` - Theory questions (submit on Gradescope)
+- `problem_1/` - Tabular Q-iteration for MountainCar
+- `problem_2/` - DQN for Tetris (provided separately)
+
+## Submission Instructions
+
+### Theory (Gradescope)
+Submit your answers to the theory questions as a PDF on Gradescope.
+
+### Programming Problems (Leaderboard)
+
+For each programming problem, submit to the course leaderboard:
+
+**Problem 1: MountainCar Q-Iteration**
+- Submit: `policy.py` and `checkpoint.pt`
+- Your policy must achieve mean reward > -150
+
+**Problem 2: Tetris DQN**
+- Submit: `policy.py` and `checkpoint.pt`
+- Your policy must beat the random baseline
+
+### How to Submit to Leaderboard
+
+1. Go to the course leaderboard website
+2. Select the appropriate problem (MountainCar or Tetris)
+3. Upload your `policy.py` and `checkpoint.pt` files
+4. Wait for evaluation results
+
+### File Requirements
+
+Your `policy.py` must contain:
+- A `Policy` class with a `forward(obs)` method that returns an action
+- A `load_policy(checkpoint_path)` function that returns a Policy instance
+
+Your `checkpoint.pt` must be loadable by your `load_policy` function.
+
+## Getting Started
+
+```bash
+# Install dependencies
+pip install torch numpy gymnasium pufferlib
+
+# Problem 1: Run Q-iteration
+cd problem_1
+python problem_1.py
+
+# Test your policy locally
+python policy.py
+```
+
+## Grading
+
+- Theory questions: See Gradescope rubric
+- Programming problems: Pass/fail based on leaderboard performance thresholds