|
| 1 | +# Plutus Core Development Instructions |
| 2 | + |
| 3 | +Always reference these instructions first and fallback to search or bash commands only when you encounter unexpected information that does not match the info here. |
| 4 | + |
| 5 | +## Critical Development Environment Requirements |
| 6 | + |
| 7 | +**MANDATORY: You MUST use Nix for development.** All builds and tests require the Nix environment. Do not attempt to build without Nix - it will fail due to missing dependencies and specific toolchain requirements. |
| 8 | + |
| 9 | +### Nix Environment Setup |
| 10 | + |
| 11 | +Install Nix first if not available: |
| 12 | +```bash |
| 13 | +curl -L https://nixos.org/nix/install | sh |
| 14 | +``` |
| 15 | + |
| 16 | +Enter the development shell: |
| 17 | +```bash |
| 18 | +nix develop --no-warn-dirty --accept-flake-config |
| 19 | +``` |
| 20 | + |
| 21 | +Your prompt will change to indicate you're in the Nix shell environment. This shell provides all required tools including: |
| 22 | +- Correct GHC version with all Haskell dependencies |
| 23 | +- Cabal build tool |
| 24 | +- Haskell Language Server (HLS) |
| 25 | +- Pre-commit hooks and formatting tools |
| 26 | +- Documentation build tools |
| 27 | + |
| 28 | +## Building the Project |
| 29 | + |
| 30 | +### Essential Pre-Build Steps |
| 31 | +```bash |
| 32 | +# Always run these before building |
| 33 | +cabal update |
| 34 | +``` |
| 35 | + |
| 36 | +### Primary Build Commands |
| 37 | + |
| 38 | +**CRITICAL BUILD TIMING - NEVER CANCEL BUILDS:** |
| 39 | + |
| 40 | +Build the core library (15-30 minutes): |
| 41 | +```bash |
| 42 | +cabal build plutus-core # NEVER CANCEL - takes 15-30 minutes, timeout: 45+ minutes |
| 43 | +``` |
| 44 | + |
| 45 | +Build all packages (45-90 minutes): |
| 46 | +```bash |
| 47 | +cabal build all # NEVER CANCEL - takes 45-90 minutes, timeout: 120+ minutes |
| 48 | +``` |
| 49 | + |
| 50 | +Build specific packages: |
| 51 | +```bash |
| 52 | +cabal build plutus-tx # 10-20 minutes |
| 53 | +cabal build plutus-ledger-api # 10-15 minutes |
| 54 | +cabal build plutus-tx-plugin # 15-25 minutes |
| 55 | +cabal build cardano-constitution # 5-10 minutes |
| 56 | +cabal build plutus-executables # 10-15 minutes |
| 57 | +``` |
| 58 | + |
| 59 | +### Built Executables |
| 60 | + |
| 61 | +The build produces these command-line tools: |
| 62 | +```bash |
| 63 | +cabal run plutus # Plutus Core evaluator and tools |
| 64 | +cabal run pir # Plutus IR tools |
| 65 | +cabal run plc # Plutus Core compiler |
| 66 | +cabal run uplc # Untyped Plutus Core tools |
| 67 | +``` |
| 68 | + |
| 69 | +Example: Evaluate a Plutus Core program: |
| 70 | +```bash |
| 71 | +cabal run plc evaluate -- -h # Show help for evaluation options |
| 72 | +``` |
| 73 | + |
| 74 | +### Alternative GHC Versions |
| 75 | + |
| 76 | +The project supports multiple GHC versions. Use specific shells: |
| 77 | +```bash |
| 78 | +nix develop --no-warn-dirty --accept-flake-config .#ghc96 # Primary (default) |
| 79 | +nix develop --no-warn-dirty --accept-flake-config .#ghc98 # Alternative |
| 80 | +nix develop --no-warn-dirty --accept-flake-config .#ghc910 # Alternative |
| 81 | +``` |
| 82 | + |
| 83 | +## Testing |
| 84 | + |
| 85 | +### Test Suite Execution |
| 86 | + |
| 87 | +**CRITICAL TESTING TIMING - NEVER CANCEL TESTS:** |
| 88 | + |
| 89 | +Run all tests (30-60 minutes): |
| 90 | +```bash |
| 91 | +cabal test all # NEVER CANCEL - takes 30-60 minutes, timeout: 90+ minutes |
| 92 | +``` |
| 93 | + |
| 94 | +Run specific test suites: |
| 95 | +```bash |
| 96 | +cabal test plutus-core-test # 15-25 minutes |
| 97 | +cabal test untyped-plutus-core-test # 10-15 minutes |
| 98 | +cabal test plutus-ir-test # 10-15 minutes |
| 99 | +cabal test plutus-tx-test # 10-15 minutes |
| 100 | +cabal test plutus-ledger-api-test # 5-10 minutes |
| 101 | +``` |
| 102 | + |
| 103 | +### Golden Test Regeneration |
| 104 | + |
| 105 | +Regenerate all golden test files: |
| 106 | +```bash |
| 107 | +./scripts/regen-goldens.sh # NEVER CANCEL - takes 45-75 minutes, timeout: 120+ minutes |
| 108 | +``` |
| 109 | + |
| 110 | +For plugin tests with multiple GHC versions: |
| 111 | +```bash |
| 112 | +nix develop .#ghc96 --command cabal test plutus-tx-plugin --test-options=--accept |
| 113 | +``` |
| 114 | + |
| 115 | +### Validation Scenarios |
| 116 | + |
| 117 | +ALWAYS run these validation steps after making changes: |
| 118 | + |
| 119 | +1. **Build validation**: |
| 120 | + ```bash |
| 121 | + cabal build all --ghc-options=-Werror # Must pass with no warnings |
| 122 | + ``` |
| 123 | + |
| 124 | +2. **Test validation**: |
| 125 | + ```bash |
| 126 | + cabal test all # All tests must pass |
| 127 | + ``` |
| 128 | + |
| 129 | +3. **Specific functionality tests**: |
| 130 | + ```bash |
| 131 | + # Test Plutus Core evaluator |
| 132 | + cabal run plc evaluate -- --help |
| 133 | + |
| 134 | + # Test conformance (if modifying core evaluator) |
| 135 | + cabal run haskell-conformance -- --help |
| 136 | + cabal run agda-conformance -- --help |
| 137 | + |
| 138 | + # Test benchmarks (if modifying performance-critical code) |
| 139 | + cabal run plutus-benchmark-nofib-tests -- --help |
| 140 | + ``` |
| 141 | + |
| 142 | +4. **Formatting validation**: |
| 143 | + ```bash |
| 144 | + stylish-haskell --config .stylish-haskell.yaml --check <modified-files> |
| 145 | + cabal-fmt --check <modified-cabal-files> |
| 146 | + ``` |
| 147 | + |
| 148 | +### Performance and Conformance Testing |
| 149 | + |
| 150 | +The repository includes specialized test suites: |
| 151 | + |
| 152 | +**Conformance tests** (validate evaluator correctness): |
| 153 | +- `plutus-conformance/` - Compare Haskell vs Agda implementations |
| 154 | +- Run with: `cabal test haskell-conformance` and `cabal test agda-conformance` |
| 155 | + |
| 156 | +**Benchmark tests** (validate performance): |
| 157 | +- `plutus-benchmark/` - Performance regression tests |
| 158 | +- Run with: `cabal test plutus-benchmark-nofib-tests` |
| 159 | + |
| 160 | +## Development Tools and Linting |
| 161 | + |
| 162 | +### Pre-commit Hooks and Formatting |
| 163 | + |
| 164 | +Pre-commit hooks are defined in the Nix shell configuration and run automatically on commit. The Nix environment provides the following formatting tools: |
| 165 | + |
| 166 | +```bash |
| 167 | +# These tools are available in the Nix shell: |
| 168 | +stylish-haskell --config .stylish-haskell.yaml <files> # Haskell formatting |
| 169 | +cabal-fmt <cabal-files> # Cabal file formatting |
| 170 | +fourmolu --mode inplace <files> # Alternative Haskell formatter |
| 171 | +``` |
| 172 | + |
| 173 | +If you commit outside the Nix shell, you may get: |
| 174 | +``` |
| 175 | +pre-commit not found. Did you forget to activate your virtualenv? |
| 176 | +``` |
| 177 | + |
| 178 | +In this case, use `git commit --no-verify` or ensure you're in the Nix environment. |
| 179 | + |
| 180 | +**CRITICAL:** Always run formatting before committing or CI will fail. |
| 181 | + |
| 182 | +### Linting |
| 183 | + |
| 184 | +Two hlint configurations exist: |
| 185 | +- `./.hlint.yaml` - Suggested hints for editors |
| 186 | +- `./.github/.hlint.yaml` - Enforced hints for CI |
| 187 | + |
| 188 | +### Haskell Language Server |
| 189 | + |
| 190 | +HLS is available in the Nix shell as `haskell-language-server` (not `haskell-language-server-wrapper`). Configure your editor to use this binary. |
| 191 | + |
| 192 | +## Documentation Building |
| 193 | + |
| 194 | +### Haddock Documentation |
| 195 | + |
| 196 | +Build API documentation (60-90 minutes): |
| 197 | +```bash |
| 198 | +./scripts/combined-haddock.sh _haddock all # NEVER CANCEL - takes 60-90 minutes, timeout: 120+ minutes |
| 199 | +``` |
| 200 | + |
| 201 | +### Docusaurus Site |
| 202 | + |
| 203 | +Navigate to documentation directory: |
| 204 | +```bash |
| 205 | +cd doc/docusaurus |
| 206 | +``` |
| 207 | + |
| 208 | +Install dependencies and build (10-15 minutes): |
| 209 | +```bash |
| 210 | +yarn # Install dependencies - 2-5 minutes |
| 211 | +yarn build # Build site - 5-10 minutes |
| 212 | +yarn start # Development server |
| 213 | +``` |
| 214 | + |
| 215 | +## Package Structure and Key Locations |
| 216 | + |
| 217 | +### Core Packages |
| 218 | +- **plutus-core/** - Core language implementation and evaluator |
| 219 | +- **plutus-tx/** - Plutus Tx compiler and libraries |
| 220 | +- **plutus-tx-plugin/** - GHC plugin for Plutus Tx compilation |
| 221 | +- **plutus-ledger-api/** - Ledger integration APIs |
| 222 | +- **plutus-metatheory/** - Agda mechanized metatheory (formal proofs) |
| 223 | +- **plutus-executables/** - Command-line tools (plc, uplc, pir) |
| 224 | +- **plutus-benchmark/** - Performance and regression tests |
| 225 | +- **plutus-conformance/** - Evaluator correctness validation |
| 226 | + |
| 227 | +### Agda Metatheory |
| 228 | + |
| 229 | +The `plutus-metatheory/` package contains formal verification in Agda: |
| 230 | +```bash |
| 231 | +# Build the Agda evaluator |
| 232 | +cabal build plutus-metatheory |
| 233 | + |
| 234 | +# Run the Agda-based evaluator (equivalent to plc) |
| 235 | +cabal run plc-agda -- --help |
| 236 | + |
| 237 | +# Test conformance between Haskell and Agda implementations |
| 238 | +cabal test plutus-metatheory |
| 239 | +``` |
| 240 | + |
| 241 | +**Note:** When modifying `.lagda` files, regenerate Haskell modules with: |
| 242 | +```bash |
| 243 | +generate-malonzo-code # Available in Nix shell |
| 244 | +``` |
| 245 | + |
| 246 | +### Important Files |
| 247 | +- **cabal.project** - Main project configuration |
| 248 | +- **CONTRIBUTING.adoc** - Detailed contribution guidelines |
| 249 | +- **scripts/** - Build and utility scripts |
| 250 | +- **.github/workflows/** - CI/CD pipeline definitions |
| 251 | + |
| 252 | +### Common Development Patterns |
| 253 | + |
| 254 | +When modifying core functionality: |
| 255 | +1. **Always check plutus-core/README.md** for package-specific guidance |
| 256 | +2. **Update tests** in corresponding test/ directories |
| 257 | +3. **Regenerate golden files** if evaluator behavior changes |
| 258 | +4. **Check multiple GHC versions** if touching plutus-tx-plugin |
| 259 | + |
| 260 | +## CI Requirements |
| 261 | + |
| 262 | +The CI pipeline requires: |
| 263 | +1. **No compiler warnings** (builds with `-Werror`) |
| 264 | +2. **All tests passing** across all supported GHC versions |
| 265 | +3. **Proper formatting** (stylish-haskell, cabal-fmt) |
| 266 | +4. **No lint violations** per `.github/.hlint.yaml` |
| 267 | + |
| 268 | +## Troubleshooting Common Issues |
| 269 | + |
| 270 | +### Build Failures |
| 271 | +- **"package not found"** → Run `cabal update` in the Nix shell |
| 272 | +- **Network issues with CHaP** → Ensure you're in proper Nix environment with `nix develop` |
| 273 | +- **Missing dependencies** → Exit and re-enter `nix develop --no-warn-dirty --accept-flake-config` |
| 274 | +- **"could not find module"** → Clean and rebuild: `cabal clean && cabal build all` |
| 275 | + |
| 276 | +### Plugin Issues |
| 277 | +- **Cross-compilation problems** → Use GHC version conditionals in .cabal files |
| 278 | +- **GHC version mismatch** → Use appropriate `nix develop .#ghc<version>` shell |
| 279 | +- **"orphan instances" warnings** → Add `-Wno-orphans` to specific modules if safe |
| 280 | + |
| 281 | +### Memory Issues During Build |
| 282 | +- **OOM during compilation** → Add to cabal.project.local: |
| 283 | + ``` |
| 284 | + package plutus-core |
| 285 | + ghc-options: -fexternal-interpreter |
| 286 | + ``` |
| 287 | +- **GHCi link errors with recursion-schemes** → Same fix as above |
| 288 | + |
| 289 | +### Test Failures |
| 290 | +- **Golden test mismatches** → Regenerate with `./scripts/regen-goldens.sh` |
| 291 | +- **Conformance test failures** → Check if changes affect evaluator semantics |
| 292 | +- **Plugin test failures** → Test with all supported GHC versions |
| 293 | + |
| 294 | +### Environment Issues |
| 295 | +- **HLS not working** → Ensure you're using `haskell-language-server` (not `-wrapper`) |
| 296 | +- **Formatting errors in CI** → Run formatting tools before commit: |
| 297 | + ```bash |
| 298 | + stylish-haskell --config .stylish-haskell.yaml -i <files> |
| 299 | + cabal-fmt -i <cabal-files> |
| 300 | + ``` |
| 301 | + |
| 302 | +## Development Workflow Examples |
| 303 | + |
| 304 | +### Adding a New Built-in Function |
| 305 | +1. Modify `plutus-core/plutus-core/src/PlutusCore/Default/Builtins.hs` |
| 306 | +2. Update cost model data in `plutus-core/cost-model/data/` |
| 307 | +3. Regenerate golden tests: `./scripts/regen-goldens.sh` |
| 308 | +4. Test conformance: `cabal test haskell-conformance agda-conformance` |
| 309 | +5. Run benchmarks: `cabal test plutus-benchmark-nofib-tests` |
| 310 | + |
| 311 | +### Modifying the Evaluator |
| 312 | +1. Make changes in `plutus-core/untyped-plutus-core/` |
| 313 | +2. Update corresponding Agda code in `plutus-metatheory/src/` |
| 314 | +3. Regenerate Haskell from Agda: `generate-malonzo-code` |
| 315 | +4. Test conformance extensively: `cabal test plutus-metatheory` |
| 316 | +5. Check performance impact: `cabal run plutus-benchmark` |
| 317 | + |
| 318 | +### Working with Plutus Tx |
| 319 | +1. Changes to `plutus-tx/` may require plugin updates in `plutus-tx-plugin/` |
| 320 | +2. Test with multiple GHC versions: |
| 321 | + ```bash |
| 322 | + nix develop .#ghc96 --command cabal test plutus-tx-plugin |
| 323 | + ``` |
| 324 | +3. Update golden tests for all GHC versions if needed |
| 325 | + |
| 326 | +## Essential Timing Reminders |
| 327 | + |
| 328 | +**NEVER CANCEL ANY BUILD OR TEST COMMAND** |
| 329 | + |
| 330 | +- Full builds: 45-90 minutes (timeout: 120+ minutes) |
| 331 | +- All tests: 30-60 minutes (timeout: 90+ minutes) |
| 332 | +- Haddock generation: 60-90 minutes (timeout: 120+ minutes) |
| 333 | +- Golden test regeneration: 45-75 minutes (timeout: 120+ minutes) |
| 334 | + |
| 335 | +Always set appropriate timeouts and wait for completion. Cancelled builds waste time and may leave the environment in an inconsistent state. |
0 commit comments