|
| 1 | +# LMFDB Development Environment Instructions |
| 2 | + |
| 3 | +The L-functions and Modular Forms Database (LMFDB) is a large Flask-based mathematical web application built on SageMath that provides access to mathematical objects including elliptic curves, number fields, modular forms, and L-functions. |
| 4 | + |
| 5 | +Always reference these instructions first and fallback to search or bash commands only when you encounter unexpected information that does not match the info here. |
| 6 | + |
| 7 | +## Essential Setup & Build Commands |
| 8 | + |
| 9 | +**Environment Setup (NEVER CANCEL - takes 20+ minutes):** |
| 10 | +```bash |
| 11 | +# Create conda environment with Sage 10.4 |
| 12 | +conda env create -f .environment.yml # Takes 20+ minutes, set timeout to 60+ minutes |
| 13 | + |
| 14 | +# Activate environment (required for all commands) |
| 15 | +source /usr/share/miniconda/etc/profile.d/conda.sh |
| 16 | +conda activate lmfdb |
| 17 | + |
| 18 | +# Install Python dependencies (NEVER CANCEL - takes 2-3 minutes) |
| 19 | +sage -pip install -r requirements.txt # Set timeout to 10+ minutes |
| 20 | +``` |
| 21 | + |
| 22 | +**Linting & Code Quality (NEVER CANCEL - takes 45+ seconds):** |
| 23 | +```bash |
| 24 | +# Run full linting suite - ALWAYS run before committing |
| 25 | +tox -e lint # Takes 45 seconds, set timeout to 5+ minutes |
| 26 | + |
| 27 | +# Alternative linting commands |
| 28 | +./codestyle.sh # Runs pycodestyle checks |
| 29 | +pyflakes start-lmfdb.py user-manager.py lmfdb/ |
| 30 | +``` |
| 31 | + |
| 32 | +**Testing (Database required - see limitations below):** |
| 33 | +```bash |
| 34 | +# Full test suite (requires database connection) |
| 35 | +./test.sh # Takes 10+ minutes when working, set timeout to 30+ minutes |
| 36 | + |
| 37 | +# Run specific test modules |
| 38 | +sage -python -m pytest lmfdb/hypergm/test_hgm.py -v |
| 39 | +``` |
| 40 | + |
| 41 | +## Database and Runtime Configuration |
| 42 | + |
| 43 | +**Database Connection:** |
| 44 | +- Default configuration uses read-only database at `devmirror.lmfdb.xyz` |
| 45 | +- Local database setup requires PostgreSQL and is complex |
| 46 | +- Application creates `config.ini` automatically on first run |
| 47 | +- Tests and runtime require network access to database |
| 48 | + |
| 49 | +**Server Startup (requires database):** |
| 50 | +```bash |
| 51 | +# Start development server with debug mode |
| 52 | +sage -python start-lmfdb.py --debug # Runs on http://localhost:37777 |
| 53 | + |
| 54 | +# Server options |
| 55 | +sage -python start-lmfdb.py --help # Show all options |
| 56 | +sage -python start-lmfdb.py --port 37778 --debug # Custom port |
| 57 | +``` |
| 58 | + |
| 59 | +**LIMITATION:** The application cannot run locally without database access. Tests and server startup will fail with `psycopg2.OperationalError` if `devmirror.lmfdb.xyz` is unreachable. |
| 60 | + |
| 61 | +## Project Structure & Navigation |
| 62 | + |
| 63 | +**Main Application Code:** |
| 64 | +- `lmfdb/` - Main application directory with mathematical modules |
| 65 | + - `elliptic_curves/` - Elliptic curves over Q and number fields |
| 66 | + - `number_fields/` - Global number fields |
| 67 | + - `modular_forms/` - Classical and Hilbert modular forms |
| 68 | + - `lfunctions/` - L-functions (~2090 lines in main.py) |
| 69 | + - `characters/` - Dirichlet and Hecke characters |
| 70 | + - `galois_groups/` - Galois groups |
| 71 | + - `genus2_curves/` - Genus 2 curves |
| 72 | + - `local_fields/` - Local fields (~1883 lines in main.py) |
| 73 | + |
| 74 | +**Key Files:** |
| 75 | +- `start-lmfdb.py` - Application entry point |
| 76 | +- `lmfdb/website.py` - Main Flask application setup |
| 77 | +- `lmfdb/app.py` - Flask app configuration (25K+ lines) |
| 78 | +- `lmfdb/lmfdb_database.py` - Database interface layer |
| 79 | +- `requirements.txt` - Python dependencies |
| 80 | +- `.environment.yml` - Conda environment specification |
| 81 | + |
| 82 | +**Configuration:** |
| 83 | +- `config.ini` - Auto-generated database and Flask configuration |
| 84 | +- `tox.ini` - Test runner configuration |
| 85 | +- `test.sh` - Main test script |
| 86 | +- `codestyle.sh` - Code style checking |
| 87 | + |
| 88 | +## Validation & CI Requirements |
| 89 | + |
| 90 | +**Before Committing (CRITICAL):** |
| 91 | +```bash |
| 92 | +# ALWAYS run linting - CI will fail without this |
| 93 | +tox -e lint # Must pass before committing |
| 94 | + |
| 95 | +# Code style validation |
| 96 | +./codestyle.sh |
| 97 | + |
| 98 | +# Check pyflakes errors |
| 99 | +pyflakes start-lmfdb.py user-manager.py lmfdb/ |
| 100 | +``` |
| 101 | + |
| 102 | +**GitHub Actions CI:** |
| 103 | +- Uses matrix testing across different database configurations |
| 104 | +- Requires conda environment setup (~20 minutes) |
| 105 | +- Runs pytest with parallel execution using tox |
| 106 | +- Linting must pass (pyflakes, pylint, ruff) |
| 107 | +- Tests require database access (not available in local development) |
| 108 | + |
| 109 | +## Common Development Tasks |
| 110 | + |
| 111 | +**Adding New Mathematical Objects:** |
| 112 | +1. Create new module in `lmfdb/` following existing patterns |
| 113 | +2. Follow URL conventions in Development.md (e.g., `/ModularForm/GL2/Q/holomorphic/`) |
| 114 | +3. Create database tables using `db.create_table()` (see Postgres_FAQ.md) |
| 115 | +4. Add Flask blueprint registration in `lmfdb/website.py` |
| 116 | +5. Create templates in module's `templates/` directory |
| 117 | +6. Add tests in module's `test_*.py` files |
| 118 | + |
| 119 | +**Key Development Files to Check:** |
| 120 | +- `lmfdb/utils/search_parsing.py` - Search functionality (~1892 lines) |
| 121 | +- `lmfdb/classical_modular_forms/main.py` - Example complex module (~1780 lines) |
| 122 | +- `Development.md` - Detailed development guidelines |
| 123 | +- `StyleGuide.md` - HTML/CSS styling conventions |
| 124 | +- `Postgres_FAQ.md` - Database interaction documentation |
| 125 | + |
| 126 | +**Template System:** |
| 127 | +- Uses Flask + Jinja2 templates |
| 128 | +- Base template: `lmfdb/templates/homepage.html` |
| 129 | +- Common template variables: `title`, `properties`, `bread`, `sidebar` |
| 130 | +- CSS in `lmfdb/templates/style.css` |
| 131 | + |
| 132 | +## Technology Stack |
| 133 | + |
| 134 | +**Core Technologies:** |
| 135 | +- **SageMath 10.4** - Mathematical computation system |
| 136 | +- **Flask 3.1.1** - Web framework |
| 137 | +- **PostgreSQL** - Database (via psycodict abstraction) |
| 138 | +- **Jinja2** - Template engine |
| 139 | +- **pytest** - Testing framework |
| 140 | + |
| 141 | +**Key Dependencies:** |
| 142 | +- `psycodict` - Database abstraction layer |
| 143 | +- `psycopg2-binary` - PostgreSQL adapter |
| 144 | +- `flask-login` - User authentication |
| 145 | +- `bcrypt` - Password hashing |
| 146 | +- `pyyaml` - YAML processing |
| 147 | + |
| 148 | +**Build Tools:** |
| 149 | +- `conda` - Environment management |
| 150 | +- `tox` - Test runner and automation |
| 151 | +- `pyflakes`, `pylint`, `ruff` - Code quality tools |
| 152 | + |
| 153 | +## Timing Expectations & Timeouts |
| 154 | + |
| 155 | +**NEVER CANCEL these operations:** |
| 156 | +- `conda env create -f .environment.yml` - 20+ minutes (set timeout: 60+ minutes) |
| 157 | +- `sage -pip install -r requirements.txt` - 2-3 minutes (set timeout: 10+ minutes) |
| 158 | +- `tox -e lint` - 45 seconds (set timeout: 5+ minutes) |
| 159 | +- `./test.sh` - 10+ minutes when working (set timeout: 30+ minutes) |
| 160 | + |
| 161 | +**Quick Operations:** |
| 162 | +- Environment activation: 1-2 seconds |
| 163 | +- `./codestyle.sh`: <5 seconds |
| 164 | +- `sage --version`: <2 seconds |
| 165 | + |
| 166 | +## Repository Statistics |
| 167 | + |
| 168 | +- **51 main modules** in lmfdb/ directory |
| 169 | +- **40+ test files** across the codebase |
| 170 | +- **71,000+ lines** of Python code |
| 171 | +- **Complex mathematical application** with deep domain expertise required |
| 172 | +- **Large contributor base** (see CONTRIBUTORS.yaml) |
| 173 | + |
| 174 | +## Working Effectively |
| 175 | + |
| 176 | +**Daily Development Workflow:** |
| 177 | +1. Always activate conda environment first |
| 178 | +2. Run `tox -e lint` before making changes to ensure clean baseline |
| 179 | +3. Make minimal, surgical code changes |
| 180 | +4. Test changes locally (limited without database access) |
| 181 | +5. Run `tox -e lint` before committing |
| 182 | +6. Check that pyflakes shows no new errors |
| 183 | + |
| 184 | +**Code Style:** |
| 185 | +- Follow existing patterns in similar modules |
| 186 | +- Use mathematical URL conventions from Development.md |
| 187 | +- Create knowls for mathematical terminology |
| 188 | +- Include proper credits and documentation |
| 189 | +- Update related templates and tests |
| 190 | + |
| 191 | +This codebase requires significant mathematical domain knowledge and familiarity with web development patterns. Always consult the existing documentation and similar modules when making changes. |
0 commit comments