@@ -26,27 +26,27 @@ uv sync --dev
26
26
27
27
Run a simple smoke test (relu) with the default ATen backend:
28
28
``` bash
29
- uv run python scripts/main.py --suite smoke --backend aten
29
+ uv run python BackendBench/ scripts/main.py --suite smoke --backend aten
30
30
```
31
31
32
32
Run the smoke test with FlagGems:
33
33
``` bash
34
- uv run python scripts/main.py --suite smoke --backend flag_gems
34
+ uv run python BackendBench/ scripts/main.py --suite smoke --backend flag_gems
35
35
```
36
36
37
37
Run opinfo tests (correctness only) with ATen
38
38
``` bash
39
- uv run python scripts/main.py --suite opinfo --backend aten
39
+ uv run python BackendBench/ scripts/main.py --suite opinfo --backend aten
40
40
```
41
41
42
42
Run a filtered set of opinfo tests with FlagGems
43
43
``` bash
44
- uv run python scripts/main.py --suite opinfo --backend flag_gems --ops " add,sub"
44
+ uv run python BackendBench/ scripts/main.py --suite opinfo --backend flag_gems --ops " add,sub"
45
45
```
46
46
47
47
Run all the opinfo tests with FlagGems (takes a few minutes)
48
48
``` bash
49
- uv run python scripts/main.py --suite opinfo --backend flag_gems
49
+ uv run python BackendBench/ scripts/main.py --suite opinfo --backend flag_gems
50
50
```
51
51
52
52
## LLM-Based Kernel Generation and Evaluation
@@ -56,7 +56,7 @@ Generate and evaluate PyTorch kernels using Claude API:
56
56
Run LLM evaluation on smoke test (relu operation):
57
57
``` bash
58
58
export ANTHROPIC_API_KEY=your_api_key_here
59
- uv run python scripts/main.py --suite smoke --backend llm
59
+ uv run python BackendBench/ scripts/main.py --suite smoke --backend llm
60
60
```
61
61
62
62
## KernelAgent-Based Triton Kernel Generation
@@ -71,19 +71,19 @@ git submodule update --init --recursive
71
71
Run KernelAgent evaluation on smoke test (relu operation):
72
72
``` bash
73
73
export OPENAI_API_KEY=your_api_key_here
74
- uv run python scripts/main.py --suite smoke --backend kernel_agent
74
+ uv run python BackendBench/ scripts/main.py --suite smoke --backend kernel_agent
75
75
```
76
76
77
77
Run KernelAgent with custom configuration:
78
78
``` bash
79
79
export OPENAI_API_KEY=your_api_key_here
80
- uv run python scripts/main.py --suite smoke --backend kernel_agent --kernel-agent-workers 6 --kernel-agent-max-rounds 15
80
+ uv run python BackendBench/ scripts/main.py --suite smoke --backend kernel_agent --kernel-agent-workers 6 --kernel-agent-max-rounds 15
81
81
```
82
82
83
83
Run KernelAgent on opinfo tests with a specific operation:
84
84
``` bash
85
- export OPENAI_API_KEY=your_api_key_here
86
- uv run python scripts/main.py --suite opinfo --backend kernel_agent --ops " add"
85
+ export OPENAI_API_KEY=< your_api_key_here>
86
+ uv run python BackendBench/ scripts/main.py --suite opinfo --backend kernel_agent --ops " add"
87
87
```
88
88
89
89
## Directory-Based Kernel Development
@@ -177,7 +177,7 @@ uv run python generated_kernels/relu/relu_implementation_1.py
177
177
178
178
Test with BackendBench:
179
179
``` bash
180
- uv run python scripts/main.py --suite smoke --backend directory
180
+ uv run python BackendBench/ scripts/main.py --suite smoke --backend directory
181
181
```
182
182
183
183
## License
0 commit comments