-
Notifications
You must be signed in to change notification settings - Fork 3
Implement std::intrinsics::raw_eq support in KMIR #665
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
Stevengre
wants to merge
17
commits into
master
Choose a base branch
from
jh/intrinsic-raw-eq
base: master
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from 11 commits
Commits
Show all changes
17 commits
Select commit
Hold shift + click to select a range
0ed9d98
Add raw_eq_simple test for intrinsic raw_eq functionality
Stevengre e7f3a31
add test to exec_smir
Stevengre a545912
Implement std::intrinsics::raw_eq support in KMIR
Stevengre eb64d3e
remove useless \n
Stevengre 28467a0
docs: Add documentation section for raw_eq intrinsic
Stevengre 8f03fd7
docs: Document current limitations of raw_eq implementation
Stevengre 9f66b04
update expected files
Stevengre 95a6b5d
Update kmir/src/kmir/kdist/mir-semantics/kmir.md
Stevengre 54de2b2
Update kmir/src/kmir/kdist/mir-semantics/kmir.md
Stevengre 7eac083
docs: Improve intrinsic development documentation
Stevengre d05eaa5
update expected files
Stevengre 296e95f
refactor: Keep operands within #execIntrinsic for better indexing
Stevengre 3fb15d5
docs: Remove unnecessary Rust example in intrinsic guide
Stevengre 6d3d53f
docs: Simplify intrinsic implementation examples
Stevengre 1ada2ae
refactor: Simplify black_box intrinsic implementation
Stevengre 105d2aa
docs: Remove redundant code style guideline
Stevengre e4a9bc0
updated expected
Stevengre File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,73 +1,269 @@ | ||
# Adding Intrinsics | ||
|
||
## Overview | ||
|
||
This guide explains how to add support for new intrinsic functions in KMIR. Intrinsics are compiler built-in functions that don't have regular MIR bodies and require special semantic rules. | ||
|
||
## Architecture | ||
|
||
As of PR #665, intrinsics use a "freeze/heat" pattern: | ||
1. **Operand Evaluation (Freeze)**: `#readOperands(ARGS)` evaluates all arguments to values | ||
2. **Intrinsic Execution (Heat)**: `#execIntrinsic(symbol("name"), DEST)` executes with values on K cell | ||
3. **Pattern Matching**: Rules match on specific value patterns for each intrinsic | ||
|
||
## Development Workflow | ||
|
||
### Step 1: Create Test File | ||
Create `tests/rust/intrinsic/your_intrinsic.rs`: | ||
|
||
Create test file in `kmir/src/tests/integration/data/exec-smir/intrinsic/`: | ||
|
||
```rust | ||
// your_intrinsic.rs | ||
#![feature(core_intrinsics)] | ||
|
||
fn main() { | ||
let result = your_intrinsic(args); | ||
assert_eq!(result, expected); | ||
use std::intrinsics::your_intrinsic; | ||
|
||
// Set up test values | ||
let val = 42; | ||
let result = your_intrinsic(&val); | ||
|
||
// Add assertion to verify behavior | ||
assert!(result); | ||
} | ||
``` | ||
|
||
### Step 2: Generate SMIR and Verify Intrinsic Detection | ||
```bash | ||
# Generate SMIR JSON | ||
make generate-tests-smir | ||
### Step 2: Add Test to Integration Suite | ||
|
||
# Update expected outputs and verify intrinsic is detected | ||
make test-unit TEST_ARGS="--update-expected-output" | ||
Edit `kmir/src/tests/integration/test_integration.py` and add entry to `EXEC_DATA`: | ||
|
||
```python | ||
( | ||
'your_intrinsic', | ||
EXEC_DATA_DIR / 'intrinsic' / 'your_intrinsic.smir.json', | ||
EXEC_DATA_DIR / 'intrinsic' / 'your_intrinsic.state', | ||
65, # Start with small depth, increase if needed | ||
), | ||
``` | ||
|
||
Check `tests/expected/unit/test_smir/test_function_symbols/your_intrinsic.expected.json` to confirm the intrinsic appears as `IntrinsicSym`. | ||
The SMIR JSON will be generated automatically when the test runs. | ||
|
||
### Step 3: Generate Initial State (Will Show Stuck Point) | ||
|
||
### Step 3: Run Initial Integration Test | ||
```bash | ||
# Run test and update expected output (will show stuck at intrinsic call) | ||
make test-integration TEST_ARGS="-k your_intrinsic --update-expected-output" | ||
# Generate initial state showing where execution gets stuck | ||
make test-integration TEST_ARGS="-k 'exec_smir and your_intrinsic' --update-expected-output" | ||
|
||
# Backup the initial state for comparison | ||
cp tests/expected/integration/test_exec_smir/intrinsic_your_intrinsic.state \ | ||
tests/expected/integration/test_exec_smir/intrinsic_your_intrinsic.state.backup | ||
# This will create your_intrinsic.state showing execution stuck at: | ||
# #execIntrinsic(symbol("your_intrinsic"), DEST) | ||
|
||
# Save this for comparison | ||
cp kmir/src/tests/integration/data/exec-smir/intrinsic/your_intrinsic.state \ | ||
your_intrinsic.state.initial | ||
``` | ||
|
||
### Step 4: Implement K Rule | ||
### Step 4: Implement K Semantics Rule | ||
|
||
Edit `kmir/src/kmir/kdist/mir-semantics/kmir.md`: | ||
|
||
#### For Simple Value Operations (like `black_box`): | ||
|
||
```k | ||
rule <k> #execIntrinsic(mirString("your_intrinsic"), ARGS, DEST) => | ||
/* your implementation */ | ||
rule <k> ListItem(ARG:Value) ~> #execIntrinsic(symbol("your_intrinsic"), DEST) | ||
=> #setLocalValue(DEST, process(ARG)) | ||
... </k> | ||
``` | ||
|
||
### Step 5: Rebuild and Test | ||
#### For Reference Operations (like `raw_eq`): | ||
|
||
```k | ||
// Handle Reference values | ||
rule <k> ListItem(Reference(_OFFSET, place(LOCAL, PROJ), _MUT, _META):Value) | ||
~> #execIntrinsic(symbol("your_intrinsic"), DEST) | ||
=> #readOperands( | ||
operandCopy(place(LOCAL, projectionElemDeref PROJ)) | ||
.Operands | ||
) ~> #execYourIntrinsic(DEST) | ||
... </k> | ||
|
||
// Helper function to avoid recursion | ||
syntax KItem ::= #execYourIntrinsic(Place) | ||
rule <k> ListItem(VAL:Value) ~> #execYourIntrinsic(DEST) | ||
=> #setLocalValue(DEST, process(VAL)) | ||
... </k> | ||
``` | ||
|
||
### Step 5: Add Documentation | ||
|
||
Add a section in `kmir.md` under "Intrinsic Functions": | ||
|
||
```markdown | ||
#### Your Intrinsic (`std::intrinsics::your_intrinsic`) | ||
|
||
Description of what the intrinsic does and how it's implemented. | ||
|
||
**Current Limitations:** | ||
- Any limitations or unhandled cases | ||
- Future improvements needed | ||
``` | ||
|
||
### Step 6: Rebuild and Test | ||
|
||
```bash | ||
# Rebuild K semantics | ||
make build | ||
|
||
# Run test again | ||
make test-integration TEST_ARGS="-k your_intrinsic --update-expected-output" | ||
# Run test again and update the state with working implementation | ||
make test-integration TEST_ARGS="-k 'exec_smir and your_intrinsic' --update-expected-output" | ||
|
||
# Compare results | ||
diff tests/expected/integration/test_exec_smir/intrinsic_your_intrinsic.state.backup \ | ||
tests/expected/integration/test_exec_smir/intrinsic_your_intrinsic.state | ||
# Compare to see the progress | ||
diff your_intrinsic.state.initial \ | ||
kmir/src/tests/integration/data/exec-smir/intrinsic/your_intrinsic.state | ||
``` | ||
|
||
The diff should show progress past the intrinsic call if implementation is correct. | ||
### Step 7: Verify Both Backends | ||
|
||
```bash | ||
# Test with both LLVM and Haskell backends | ||
make test-integration TEST_ARGS="-k 'exec_smir and your_intrinsic'" | ||
|
||
# Both should pass with consistent results | ||
# If not, you may need backend-specific state files | ||
``` | ||
|
||
### Step 6: Verify Results | ||
Ensure the test completes successfully and the intrinsic behaves as expected. | ||
## Examples | ||
|
||
## Example: black_box | ||
### Example 1: `black_box` (Simple Identity) | ||
|
||
Initial state (before rule): | ||
```k | ||
// Takes one value, returns it unchanged | ||
rule <k> ListItem(ARG:Value) ~> #execIntrinsic(symbol("black_box"), DEST) | ||
=> #setLocalValue(DEST, ARG) | ||
... </k> | ||
``` | ||
#setUpCalleeData ( IntrinsicFunction ( mirString ( "black_box" ) ) , ...) | ||
|
||
### Example 2: `raw_eq` (Reference Comparison) | ||
|
||
```k | ||
// Takes two references, compares dereferenced values | ||
rule <k> ListItem(Reference(_OFF1, place(L1, P1), _M1, _META1):Value) | ||
ListItem(Reference(_OFF2, place(L2, P2), _M2, _META2):Value) | ||
~> #execIntrinsic(symbol("raw_eq"), DEST) | ||
=> #readOperands( | ||
operandCopy(place(L1, projectionElemDeref P1)) | ||
operandCopy(place(L2, projectionElemDeref P2)) | ||
.Operands | ||
) ~> #execRawEq(DEST) | ||
... </k> | ||
|
||
syntax KItem ::= #execRawEq(Place) | ||
rule <k> ListItem(VAL1:Value) ListItem(VAL2:Value) ~> #execRawEq(DEST) | ||
=> #setLocalValue(DEST, BoolVal(VAL1 ==K VAL2)) | ||
... </k> | ||
``` | ||
|
||
## Common Patterns | ||
|
||
### Pattern 1: Direct Value Processing | ||
Use when the intrinsic operates directly on values without indirection. | ||
|
||
### Pattern 2: Reference Dereferencing | ||
Use `projectionElemDeref` to access values behind references. | ||
|
||
### Pattern 3: Helper Functions | ||
Create dedicated functions like `#execYourIntrinsic` to: | ||
- Avoid recursion issues | ||
- Separate concerns | ||
- Make rules more readable | ||
|
||
### Pattern 4: Multiple Operands | ||
Pattern match multiple `ListItem` entries for multi-argument intrinsics. | ||
|
||
## Testing Best Practices | ||
|
||
1. **Start Simple**: Test with primitive types first | ||
2. **Save Initial State**: Keep the stuck state for comparison | ||
3. **Use Correct Test Filter**: Always use `exec_smir and your_intrinsic` to ensure correct test runs | ||
4. **Check Both Backends**: Ensure LLVM and Haskell produce same results | ||
5. **Document Limitations**: Note what cases aren't handled yet | ||
6. **Create Issue for Future Work**: Track enhancements needed (like #666 for `raw_eq`) | ||
|
||
## Debugging Tips | ||
|
||
### Check Execution State | ||
```bash | ||
# See where execution is stuck with verbose output | ||
make test-integration TEST_ARGS="-k 'exec_smir and your_intrinsic' -vv" | ||
|
||
# Look for the K cell content to see what values are present | ||
``` | ||
|
||
After implementing rule: | ||
### Understanding the State File | ||
The state file shows the complete execution state. Key sections to check: | ||
- `<k>`: Shows current execution point | ||
- `<locals>`: Shows local variable values | ||
- `<functions>`: Should contain `IntrinsicFunction(symbol("your_intrinsic"))` | ||
|
||
### Verify Intrinsic Recognition | ||
```bash | ||
# Check SMIR JSON to confirm intrinsic is recognized | ||
cat kmir/src/tests/integration/data/exec-smir/intrinsic/your_intrinsic.smir.json | grep -A5 your_intrinsic | ||
``` | ||
Program continues execution with value 11 passed through | ||
``` | ||
|
||
## Common Issues | ||
|
||
### Issue: Wrong test runs | ||
**Solution**: Use `-k 'exec_smir and your_intrinsic'` to ensure `test_exec_smir` runs, not other tests. | ||
|
||
### Issue: "Function not found" | ||
**Solution**: The intrinsic should be automatically recognized if it appears in SMIR. Check the SMIR JSON to confirm. | ||
|
||
### Issue: Execution stuck at `#execIntrinsic` | ||
**Solution**: Your rule pattern doesn't match. Check: | ||
- The exact intrinsic name in the symbol | ||
- Value types on K cell (use `ListItem(VAL:Value)` to match any value) | ||
- Number of arguments expected | ||
|
||
### Issue: Recursion/infinite loop | ||
**Solution**: Use helper functions to separate evaluation stages, avoid calling `#execIntrinsic` within itself. | ||
|
||
### Issue: Different backend results | ||
**Solution**: | ||
- Increase execution depth if needed | ||
- Check for backend-specific evaluation order issues | ||
- May need backend-specific expected files (`.llvm.state`, `.haskell.state`) | ||
|
||
### Issue: Test timeout | ||
**Solution**: | ||
- Start with smaller depth (e.g., 65 instead of 1000) | ||
- Optimize your rule to avoid unnecessary computation | ||
- Check for infinite loops in your implementation | ||
|
||
## Important Notes | ||
|
||
### The Freeze/Heat Pattern | ||
The current architecture ensures all operands are evaluated before the intrinsic executes: | ||
- **Freeze**: `#readOperands(ARGS)` evaluates operands to values | ||
- **Heat**: Your rule matches on these values | ||
- This prevents evaluation order issues and simplifies rules | ||
|
||
### When to Use Helper Functions | ||
Always use a helper function (like `#execRawEq`) when: | ||
- You need to call `#readOperands` again (to avoid recursion) | ||
- The logic is complex enough to benefit from separation | ||
- You need multiple evaluation steps | ||
|
||
### Testing Strategy | ||
1. Write the test with expected behavior first | ||
2. Generate initial state to see where it gets stuck | ||
3. Implement the minimal rule needed | ||
4. Update state to verify progress | ||
5. Iterate to handle edge cases | ||
6. Document limitations for future work | ||
|
||
## References | ||
|
||
- PR #665: `raw_eq` implementation with freeze/heat pattern refactoring | ||
- PR #659: `black_box` implementation | ||
- Issue #666: Enhancements for complex `raw_eq` cases | ||
- [Rust Intrinsics Documentation](https://doc.rust-lang.org/std/intrinsics/) |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Very comprehensive documentation! Maybe we can shorten it a bit? (but of course it needs to be enough information for the AI to follow the steps).
We should probably not mention PR numbers explicitly in the doc.s. Could we just point to a query URL that searches for all these PRs (for example, make sure the PR descriptions always mention "intrinsic", or a github label) ?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I just condensed this doc, and hope it look good to you. But I don't want to remove the
Patterns
andIssues
sections, because these are what I prompt to the Claude and they might help the AI to generate other intrinsics without comphrehensive prompt.