Skip to content

Set EXPERIMENTAL_1b_CHECKPOINT to True by default #840

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
jwilber merged 8 commits into from
May 15, 2025
Merged

Set EXPERIMENTAL_1b_CHECKPOINT to True by default #840

Changes from all commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
441e501
Set EXPERIMENTAL_1b_CHECKPOINT to True by default
jwilber Apr 21, 2025
7cb763c
Remove experimental flag and make ckpt default for 1b model; update b…
jwilber Apr 21, 2025
69be922
Merge branch 'main' into jwilber/1665-ev2-brca1_nb-1b-use-bf16-ckpt
jwilber Apr 24, 2025
fbb46ef
Merge branch 'main' into jwilber/1665-ev2-brca1_nb-1b-use-bf16-ckpt
jwilber Apr 25, 2025
26ea687
Merge branch 'main' into jwilber/1665-ev2-brca1_nb-1b-use-bf16-ckpt
jwilber May 8, 2025
fc03790
Merge branch 'main' into jwilber/1665-ev2-brca1_nb-1b-use-bf16-ckpt
jstjohn May 12, 2025
2d2f873
Merge branch 'main' into jwilber/1665-ev2-brca1_nb-1b-use-bf16-ckpt
jwilber May 14, 2025
3bfebac
fix precommit
jwilber May 15, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 5 additions & 7 deletions sub-packages/bionemo-evo2/examples/zeroshot_brca1.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -562,16 +562,14 @@
"outputs": [],
"source": [
"%%capture\n",
"EXPERIMENTAL_1b_CHECKPOINT: bool = False\n",
"MODEL_SIZE = \"1b\" # also try 7b if you have a GPU with more than 32GB of memory\n",
"\n",
"# Define checkpoint path\n",
"if EXPERIMENTAL_1b_CHECKPOINT and MODEL_SIZE == \"1b\":\n",
"if MODEL_SIZE == \"1b\":\n",
" from bionemo.core.data.load import load\n",
"\n",
" # This is a new 1b checkpoint that has been fine-tuned on BF16 hardware. It should be able to handle FP8 as well.\n",
" # this line will download the checkpoint from NGC to your $HOME/.cache/bionemo directory and return the path.\n",
" # alternatively you can use `CHECKPOINT_PATH=$(download_bionemo_data evo2/1b-8k-bf16:1.0)` to do the same on the\n",
" # command line.\n",
" # This line will download the checkpoint from NGC to your $HOME/.cache/bionemo directory and return the path.\n",
" # To do the same from the command line, use `CHECKPOINT_PATH=$(download_bionemo_data evo2/1b-8k-bf16:1.0)`\n",
" checkpoint_path = load(\"evo2/1b-8k-bf16:1.0\")\n",
"else:\n",
" checkpoint_path = Path(f\"nemo2_evo2_{MODEL_SIZE}_8k\")\n",
Expand Down Expand Up @@ -1186,7 +1184,7 @@
"\n",
"| Model Size | Dataset Sample Fraction | AUROC |\n",
"|------------|------------------------|-------|\n",
"| Evo 2 1B | 100% | 0.76 |\n",
"| Evo 2 1B | 100% | 0.74 |\n",
"| Evo 2 7B | 100% | 0.87 |\n"
]
}
Expand Down