Skip to content

Commit f975cc9

Browse files
authored
Update README.md
1 parent 5c5c256 commit f975cc9

File tree

1 file changed

+2
-1
lines changed

1 file changed

+2
-1
lines changed

README.md

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -31,9 +31,10 @@ To install deps:
3131
pip install --pre torch==2.8.0.dev20250605+cu126 --index-url https://download.pytorch.org/whl/nightly/cu126
3232
pip install --pre torchao==0.12.0.dev20250609+cu126 --index-url https://download.pytorch.org/whl/nightly/cu126
3333
pip install diffusers==0.33.1
34-
MAX_JOBS=4 pip install flash-attn==3.0.0b1 --no-build-isolation
3534
```
3635

36+
To install flash attention v3, follow the instructions in https://github.com/Dao-AILab/flash-attention#flashattention-3-beta-release.
37+
3738
For hardware, we used a 96GB 700W H100 GPU. Some of the optimizations applied (BFloat16, torch.compile, Combining q,k,v projections, dynamic float8 quantization) are available on CPU as well.
3839

3940
## Benchmarking

0 commit comments

Comments
 (0)