-
Notifications
You must be signed in to change notification settings - Fork 7
feat: add factorial benchmark scenario #16
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
6 commits
Select commit
Hold shift + click to select a range
edd6043
feat: add factorial benchmark scenario
Unisay 664dbcd
refactor: merge Main.hs and FactorialMain.hs
Unisay 6acfab3
feat: add Plinth factorial submission
Unisay 047b284
refactor: replace source file copies with README links
Unisay b2367c4
refactor: simplify source README files to only include links
Unisay b4e7e0b
fixup: update metrics.json timestamps and notes
Unisay File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,46 @@ | ||
| {-# LANGUAGE BangPatterns #-} | ||
| {-# LANGUAGE BlockArguments #-} | ||
| {-# LANGUAGE DataKinds #-} | ||
| {-# LANGUAGE LambdaCase #-} | ||
| {-# LANGUAGE MultiParamTypeClasses #-} | ||
| {-# LANGUAGE MultiWayIf #-} | ||
| {-# LANGUAGE NamedFieldPuns #-} | ||
| {-# LANGUAGE OverloadedStrings #-} | ||
| {-# LANGUAGE PatternSynonyms #-} | ||
| {-# LANGUAGE Strict #-} | ||
| {-# LANGUAGE TemplateHaskell #-} | ||
| {-# LANGUAGE ViewPatterns #-} | ||
| {-# LANGUAGE NoImplicitPrelude #-} | ||
| -- | ||
| {-# OPTIONS_GHC -fno-full-laziness #-} | ||
| {-# OPTIONS_GHC -fno-ignore-interface-pragmas #-} | ||
| {-# OPTIONS_GHC -fno-omit-interface-pragmas #-} | ||
| {-# OPTIONS_GHC -fno-spec-constr #-} | ||
| {-# OPTIONS_GHC -fno-specialise #-} | ||
| {-# OPTIONS_GHC -fno-strictness #-} | ||
| {-# OPTIONS_GHC -fno-unbox-small-strict-fields #-} | ||
| {-# OPTIONS_GHC -fno-unbox-strict-fields #-} | ||
| {-# OPTIONS_GHC -fplugin PlutusTx.Plugin #-} | ||
| {-# OPTIONS_GHC -fplugin-opt PlutusTx.Plugin:no-conservative-optimisation #-} | ||
| {-# OPTIONS_GHC -fplugin-opt PlutusTx.Plugin:no-preserve-logging #-} | ||
| {-# OPTIONS_GHC -fplugin-opt PlutusTx.Plugin:remove-trace #-} | ||
| {-# OPTIONS_GHC -fplugin-opt PlutusTx.Plugin:target-version=1.1.0 #-} | ||
|
|
||
| module Factorial (factorialCode, factorial10Code) where | ||
|
|
||
| import PlutusTx | ||
| import PlutusTx.Prelude | ||
|
|
||
| -- | Compiled validator script | ||
| factorialCode :: CompiledCode (Integer -> Integer) | ||
| factorialCode = $$(PlutusTx.compile [||factorial||]) | ||
|
|
||
| -- | The compiled factorial validator for n=10 | ||
| factorial10Code :: CompiledCode Integer | ||
| factorial10Code = factorialCode `unsafeApplyCode` liftCodeDef (10 :: Integer) | ||
|
|
||
| {-# INLINEABLE factorial #-} | ||
| factorial :: Integer -> Integer | ||
| factorial n | ||
| | n <= 0 = 1 | ||
| | otherwise = n * factorial (n - 1) | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,173 @@ | ||
| # Factorial Benchmark Scenario | ||
|
|
||
| The Factorial benchmark is a **synthetic computational scenario** designed to measure the performance characteristics of iterative and recursive algorithms implemented as UPLC programs. This benchmark tests a compiler's ability to optimize mathematical computations, manage stack operations, and handle integer arithmetic efficiently. | ||
|
|
||
| ## TL;DR | ||
|
|
||
| Implement a Factorial function that computes **factorial(10) = 3628800** and compile it as a fully-applied UPLC program. | ||
|
|
||
| **Required Files**: Submit `factorial.uplc`, `metadata.json`, `metrics.json` to `submissions/factorial/{Compiler}_{Version}_{Handle}/` | ||
|
|
||
| **Target**: `factorial(10)` → Expected result: `3628800` | ||
| **Metrics**: CPU units, Memory units, Script size (bytes), Term size | ||
| **Constraints**: Plutus Core 1.1.0, Plutus V3 recommended, CEK machine budget limits | ||
| **Implementation**: Choose recursive or iterative approach | ||
|
|
||
| --- | ||
|
|
||
| ## Exact Task | ||
|
|
||
| Implement a Factorial function and compile it as a **fully-applied UPLC program** that computes the factorial of 10. | ||
|
|
||
| ### Core Requirements | ||
|
|
||
| 1. **Function Implementation**: Create a function that computes factorials using the mathematical definition: | ||
|
|
||
| - `factorial(0) = 1` | ||
| - `factorial(n) = n * factorial(n-1)` for n > 0 | ||
|
|
||
| 2. **Full Application**: The UPLC program must be fully-applied with the target value (10) baked in during compilation, not passed as a parameter. | ||
|
|
||
| 3. **Target Computation**: `factorial(10)` must produce exactly `3628800` | ||
|
|
||
| ### Implementation Approaches | ||
|
|
||
| Choose the approach that works best for your compiler: | ||
|
|
||
| #### Recursive Implementation (Most Direct) | ||
|
|
||
| ```pseudocode | ||
| function factorial_recursive(n): | ||
| if n == 0: | ||
| return 1 | ||
| return n * factorial_recursive(n - 1) | ||
| ``` | ||
|
|
||
| #### Iterative Implementation (More Efficient) | ||
|
|
||
| ```pseudocode | ||
| function factorial_iterative(n): | ||
| if n == 0: | ||
| return 1 | ||
|
|
||
| result = 1 | ||
| for i in range(1, n + 1): | ||
| result = result * i | ||
|
|
||
| return result | ||
| ``` | ||
|
|
||
| --- | ||
|
|
||
| ## Acceptance Criteria | ||
|
|
||
| Your submission passes if: | ||
|
|
||
| - ✅ **Correctness**: Program outputs exactly `3628800` | ||
| - ✅ **Budget Compliance**: Executes within CEK machine CPU and memory limits | ||
| - ✅ **Determinism**: Produces identical results across multiple executions | ||
| - ✅ **Self-Contained**: No external dependencies or parameters | ||
| - ✅ **File Format**: Valid UPLC program that can be executed by the CEK evaluator | ||
|
|
||
| --- | ||
|
|
||
| ## Metrics Recorded | ||
|
|
||
| All submissions are measured on these standardized metrics: | ||
|
|
||
| | Metric | Description | Purpose | | ||
| | --- | --- | --- | | ||
| | **CPU Units** | Total execution units consumed | Computational efficiency | | ||
| | **Memory Units** | Peak memory usage during execution | Memory efficiency | | ||
| | **Script Size** | Compiled UPLC script size in bytes | Code generation efficiency | | ||
| | **Term Size** | UPLC term representation size | Optimization effectiveness | | ||
|
|
||
| **Measurement Environment**: Standard CEK machine evaluator with default budget limits. | ||
|
|
||
| ### Performance Context | ||
|
|
||
| **Why factorial(10)?** | ||
|
|
||
| - **Computationally Manageable**: 10! = 3,628,800 fits comfortably in standard integer types | ||
| - **Budget Safe**: Fits well within CEK machine limits for both recursive and iterative approaches | ||
| - **Optimization Sensitive**: Large enough to show compiler differences in loop vs recursion handling | ||
| - **Practical Scale**: Representative of many real-world mathematical computations | ||
|
|
||
| **Expected Performance Ranges** (approximate): | ||
|
|
||
| - **Recursive**: Higher CPU usage due to function call overhead | ||
| - **Iterative**: Lower CPU usage, more predictable memory patterns | ||
| - **Tail Recursive**: Moderate CPU usage with optimized stack handling | ||
|
|
||
| --- | ||
|
|
||
| ## Submission Checklist | ||
|
|
||
| Before submitting your implementation: | ||
|
|
||
| - [ ] **Verify Result**: Program produces exactly `3628800` when executed | ||
| - [ ] **Test Budget**: Execution completes without budget exhaustion | ||
| - [ ] **Prepare Files**: | ||
| - [ ] `factorial.uplc` - Your compiled UPLC program | ||
| - [ ] `metadata.json` - Compiler info, optimization settings, implementation notes | ||
| - [ ] `metrics.json` - Performance measurements (CPU, memory, script size, term size) | ||
| - [ ] `README.md` - Brief description of your approach | ||
| - [ ] **Directory Structure**: Place in `submissions/factorial/{Compiler}_{Version}_{Handle}/` | ||
| - [ ] **Schema Validation**: Ensure JSON files match required schemas | ||
|
|
||
| ### File Templates | ||
|
|
||
| Use these templates from `submissions/TEMPLATE/`: | ||
|
|
||
| - `metadata-template.json` for compiler and build information | ||
| - `metrics-template.json` for performance measurements | ||
|
|
||
| --- | ||
|
|
||
| ## Local Validation | ||
|
|
||
| 1. **Functional Test**: Execute your UPLC program and verify output is `3628800` | ||
| 2. **Budget Test**: Ensure execution completes within CEK machine limits | ||
| 3. **Consistency Test**: Run multiple times to confirm deterministic behavior | ||
| 4. **Schema Test**: Validate JSON files against schemas in `submissions/TEMPLATE/` | ||
|
|
||
| ### Example Validation Commands | ||
|
|
||
| ```bash | ||
| # Measure your UPLC program (if using the cape tool) | ||
| cape submission measure factorial.uplc | ||
|
|
||
| # Validate submission files | ||
| cape submission validate submissions/factorial/YourCompiler_1.0.0_YourHandle/ | ||
| ``` | ||
|
|
||
| --- | ||
|
|
||
| ## Technical Constraints | ||
|
|
||
| - **Plutus Core Version**: Target Plutus Core 1.1.0 | ||
| - **Plutus Version**: V3 recommended (V1, V2 acceptable) | ||
| - **Budget Limits**: Must complete within standard CEK machine execution limits | ||
| - **No External Dependencies**: Program must be self-contained | ||
| - **Deterministic**: Must produce consistent results | ||
|
|
||
| --- | ||
|
|
||
| ## Verification Points | ||
|
|
||
| ### Correctness Verification | ||
|
|
||
| 1. **Base Case**: `factorial(0) = 1` | ||
| 2. **Small Values**: `factorial(1) = 1`, `factorial(2) = 2`, `factorial(3) = 6`, `factorial(4) = 24` | ||
| 3. **Target Value**: `factorial(10) = 3628800` | ||
|
|
||
| ### Performance Verification | ||
|
|
||
| 1. **CPU Budget Compliance**: Must execute within CEK machine limits | ||
| 2. **Memory Budget Compliance**: Must not exceed memory allocation limits | ||
| 3. **Script Size**: Should be reasonably compact for the computation performed | ||
| 4. **Execution Consistency**: Should produce identical results across multiple runs | ||
|
|
||
| --- | ||
|
|
||
| _This benchmark serves as both a correctness test and performance comparison tool, enabling compiler authors to validate their mathematical computation handling while providing standardized metrics for community comparison._ |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,32 @@ | ||
| # Benchmark Implementation Notes | ||
|
|
||
| **Scenario**: `factorial` | ||
|
|
||
| **Submission ID**: `Plinth_1.52.0.0_Unisay` (Format: `Language_Version_GitHubHandle`) | ||
|
|
||
| ## Implementation Details | ||
|
|
||
| - **Compiler**: `Plinth 1.52.0.0` | ||
| - **Implementation Approach**: `recursive` | ||
| - **Compilation Flags**: Standard Plinth (PlutusTx) optimization flags | ||
|
|
||
| ## Performance Results | ||
|
|
||
| - See [metrics.json](metrics.json) for detailed performance measurements | ||
| - **Result**: `factorial(10) = 3628800` ✓ | ||
| - **CPU Units**: 5,859,917 | ||
| - **Memory Units**: 21,751 | ||
| - **Script Size**: 37 bytes | ||
|
|
||
| ## Reproducibility | ||
|
|
||
| - **Source Available**: `true` | ||
| - **Source Repository**: https://github.com/IntersectMBO/UPLC-CAPE | ||
| - **Compilation Config**: Targeting Plutus Core 1.1.0 with standard PlutusTx plugin configuration | ||
|
|
||
| ## Notes | ||
|
|
||
| - Uses recursive factorial implementation: `factorial(n) = if n <= 0 then 1 else n * factorial(n-1)` | ||
| - Much more efficient than fibonacci due to linear recursion vs exponential | ||
| - Compiled with PlutusTx plugin targeting Plutus Core 1.1.0 | ||
| - Source code available in the `plinth/src/Factorial.hs` module |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,6 @@ | ||
| { | ||
| "comment": "Optional: Include compilation parameters that affect UPLC output", | ||
| "optimization_flags": [], | ||
| "compiler_settings": {}, | ||
| "build_environment": {} | ||
| } |
26 changes: 26 additions & 0 deletions
26
submissions/factorial/Plinth_1.52.0.0_Unisay/factorial.uplc
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,26 @@ | ||
| (program | ||
| 1.1.0 | ||
| [ | ||
| [ | ||
| (lam s-0 [ s-0 s-0 ]) | ||
| (lam | ||
| s-1 | ||
| (lam | ||
| n-2 | ||
| (case | ||
| [ [ (builtin lessThanEqualsInteger) n-2 ] (con integer 0) ] | ||
| [ | ||
| [ (builtin multiplyInteger) n-2 ] | ||
| [ | ||
| [ s-1 s-1 ] | ||
| [ [ (builtin subtractInteger) n-2 ] (con integer 1) ] | ||
| ] | ||
| ] | ||
| (con integer 1) | ||
| ) | ||
| ) | ||
| ) | ||
| ] | ||
| (con integer 10) | ||
| ] | ||
| ) |
29 changes: 29 additions & 0 deletions
29
submissions/factorial/Plinth_1.52.0.0_Unisay/metadata.json
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,29 @@ | ||
| { | ||
| "compiler": { | ||
| "name": "Plinth", | ||
| "version": "1.52.0.0", | ||
| "commit_hash": "800c67d38d01177fd0a36d01aa23ff7fea7bd1ba" | ||
| }, | ||
| "compilation_config": { | ||
| "optimization_level": "Plinth", | ||
| "target": "uplc", | ||
| "flags": ["Plinth"], | ||
| "environment": { | ||
| "dependencies": { | ||
| "plutus-tx": "1.52.0.0", | ||
| "plutus-core": "1.52.0.0", | ||
| "plutus-ledger-api": "1.52.0.0" | ||
| } | ||
| } | ||
| }, | ||
| "contributor": { | ||
| "name": "Unisay", | ||
| "organization": "UPLC-CAPE Project" | ||
| }, | ||
| "submission": { | ||
| "date": "2025-08-18T08:00:00Z", | ||
| "source_available": true, | ||
| "source_repository": "https://github.com/IntersectMBO/UPLC-CAPE", | ||
| "implementation_notes": "Factorial implementation using Plinth (PlutusTx) with parameter 10, targeting Plutus Core 1.1.0. Recursive approach computing factorial(10) = 3628800." | ||
| } | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,15 @@ | ||
| { | ||
| "execution_environment": { | ||
| "evaluator": "PlutusTx.Eval-1.52.0.0" | ||
| }, | ||
| "measurements": { | ||
| "cpu_units": 5859917, | ||
| "memory_units": 21751, | ||
| "script_size_bytes": 37, | ||
| "term_size": 29 | ||
| }, | ||
| "notes": "Generated using UPLC-CAPE measure tool", | ||
| "scenario": "factorial", | ||
| "timestamp": "2025-08-18T08:12:44Z", | ||
| "version": "1.0.0" | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| # Optional: Place your source code files here |
3 changes: 3 additions & 0 deletions
3
submissions/factorial/Plinth_1.52.0.0_Unisay/source/README.md
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,3 @@ | ||
| # Source Code | ||
|
|
||
| **[📁 plinth/src/Factorial.hs](../../../../plinth/src/Factorial.hs)** |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
3 changes: 3 additions & 0 deletions
3
submissions/fibonacci/Plinth_1.49.0.0_Unisay/source/README.md
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,3 @@ | ||
| # Source Code | ||
|
|
||
| **[📁 plinth/src/Fibonacci.hs](../../../../plinth/src/Fibonacci.hs)** |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
3 changes: 3 additions & 0 deletions
3
submissions/fibonacci/Plinth_1.52.0.0_Unisay/source/README.md
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,3 @@ | ||
| # Source Code | ||
|
|
||
| **[📁 plinth/src/Fibonacci.hs](../../../../plinth/src/Fibonacci.hs)** |
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.