Skip to content

Added Hyperattention #47

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 66 commits into
base: main
Choose a base branch
from
Open

Added Hyperattention #47

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
66 commits
Select commit Hold shift + click to select a range
927d277
add lsh strategy
minghui-liu Aug 29, 2024
93737ad
fix debug attn loss
minghui-liu Aug 29, 2024
4326b7f
fix naming bug in LSH
minghui-liu Aug 29, 2024
2433f2c
add GSM8K task
minghui-liu Aug 29, 2024
e6b67cf
fix lsh prefill update
minghui-liu Aug 29, 2024
3a70fd6
add einops as dependency
minghui-liu Aug 29, 2024
63af176
add ChatGPT as a judge and ChatGPT Rouge metric
minghui-liu Aug 30, 2024
ad698d9
make chatgpt judge scorecard parsing more robust
minghui-liu Aug 30, 2024
685d6f6
add submit script and ignore slurm output
minghui-liu Aug 31, 2024
3a919ab
make chargpt scoreboard parsing robust
minghui-liu Aug 31, 2024
7ea0d5d
ignore submit script
minghui-liu Aug 31, 2024
40d19d5
add GSM8K KQApro task and fix metric
minghui-liu Sep 10, 2024
1b86212
add google cloud platform setup script
minghui-liu Sep 12, 2024
91df736
add parallel job list for kqa
minghui-liu Sep 12, 2024
d2798b5
remove compile option from kqa_pro.txt as it does not work
minghui-liu Sep 12, 2024
e59e959
fix kqa prompt
minghui-liu Sep 13, 2024
1083bfb
add rulerqa.txt
minghui-liu Sep 13, 2024
e633830
add rulerniah experiment command file
minghui-liu Sep 13, 2024
2a90ec6
add rulerqa and rulernian command file
minghui-liu Sep 14, 2024
98389d5
Use a fixed seed for reproducibility
minghui-liu Sep 15, 2024
ee4fd4a
add med qa task
minghui-liu Sep 18, 2024
bd57997
add medqa command file
minghui-liu Sep 18, 2024
5e3e809
change medqa max tokens to 1000
minghui-liu Sep 20, 2024
2ab03d1
use niah 4k and modify task set up
minghui-liu Sep 26, 2024
344b0f9
add vt and cwe 4k tasks
minghui-liu Sep 27, 2024
12ced39
reorganize task command files
minghui-liu Sep 27, 2024
50a93c7
add gsm multiple choice
minghui-liu Sep 27, 2024
324e34b
add passkey retrieval task
minghui-liu Sep 27, 2024
50d7e1e
add command file for gsm8k_mc and passkey
minghui-liu Sep 27, 2024
d7f6f1b
fix gsm8k_mc hf args
minghui-liu Sep 27, 2024
bba0c1a
fix gsm8k_mc row key
minghui-liu Sep 27, 2024
ef0d773
reduce passkey to 200 samples
minghui-liu Sep 27, 2024
5eb05bf
use full gsm8k
minghui-liu Sep 27, 2024
8519cee
fix passkey answer
minghui-liu Sep 27, 2024
78cca43
allow 100 tokens for passkey
minghui-liu Sep 28, 2024
8484761
add lsh dim ablation
minghui-liu Sep 30, 2024
a113fd2
use 500 samples for ablation
minghui-liu Sep 30, 2024
375c614
update all command files
minghui-liu Oct 1, 2024
7add29e
add readme for lsh experiments
minghui-liu Oct 1, 2024
13056b6
Add visualizations notebook
minghui-liu Oct 1, 2024
66c2d8b
ignore slurm submit files
minghui-liu Sep 17, 2024
9158eec
use .env file for HF and OPENAI API keys
minghui-liu Nov 4, 2024
03ed4a9
add convert hf to pth file
minghui-liu Nov 4, 2024
0b62cb9
add gsm experiments for fastgen and h2o
minghui-liu Nov 9, 2024
7c28d71
fix duplicate hf_args key
minghui-liu Nov 9, 2024
40f293f
merge fastgen and h2o gsm8k command file
minghui-liu Nov 9, 2024
8aa6419
fix medqa tasks use medqa.json
minghui-liu Nov 9, 2024
86630d0
add passkey command file
minghui-liu Nov 9, 2024
0b2a1e9
add medqa data files
minghui-liu Nov 9, 2024
07d91c4
fix gcp setup script and reqquirements.txt
minghui-liu Nov 14, 2024
000104d
fix fastgen bug where profile_attn_heads should not use self.requires…
minghui-liu Nov 14, 2024
0491dbc
add passage retrieval task from long bench
minghui-liu Nov 14, 2024
657c1a3
reorganize experiment command files
minghui-liu Nov 14, 2024
de3827f
reorganize experiments
minghui-liu Nov 14, 2024
076e5e3
fix passage retrieval answer
minghui-liu Nov 14, 2024
2339f68
fix fastgen and scissorhands command files
minghui-liu Nov 14, 2024
a699060
scissorhands need to use feed long prompt to avoid snapKV prompt comp…
minghui-liu Nov 14, 2024
2b5dbf8
add h2o all
minghui-liu Nov 14, 2024
a48a1d0
fix rulercwe spelling
minghui-liu Nov 15, 2024
4b9e757
fix scissorhands
minghui-liu Nov 15, 2024
ca3fee0
fix heavy hitter class bug of type mismatch when attn_thresholding is…
minghui-liu Nov 15, 2024
976daf2
fix h2o command file
minghui-liu Nov 15, 2024
fd6bf1d
add some missing iclr exp nov15
minghui-liu Nov 16, 2024
d57fa05
update nov15 command file
minghui-liu Nov 16, 2024
bd92539
fix illegal scorecard case
minghui-liu Nov 16, 2024
747a2ad
hyperattention untested
AnanthSankaralingam Nov 17, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 9 additions & 2 deletions .gitignore
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,9 +3,11 @@ __pycache__
.DS_Store
*.egg-info
build
.venv/
slurm-*.out
slurm_logs/

# data
data
checkpoints
out
!data/shakespeare/prepare.py
Expand All @@ -18,4 +20,9 @@ original_adapter.py
.vscode

torch_compile_debug
results
results
submit.sh
submit*.sh

.env
.cache/
1 change: 1 addition & 0 deletions .python-version
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
3.11.9
701 changes: 701 additions & 0 deletions COLD_COMPRESS_README.md
View file
Open in desktop

Large diffs are not rendered by default.

679 changes: 38 additions & 641 deletions README.md
View file
Open in desktop

Large diffs are not rendered by default.

Loading