Skip to content

Commit 243125f

Browse files
committed
Update on "Remove sharded ckpt from export_llama"
Sharded checkpoint isn't used anymore; removing it and simplifying export_llama. Differential Revision: [D87828518](https://our.internmc.facebook.com/intern/diff/D87828518/) [ghstack-poisoned]
1 parent 8a5525f commit 243125f

File tree

1 file changed

+7
-0
lines changed

1 file changed

+7
-0
lines changed

extension/llm/export/config/llm_config.py

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -86,6 +86,10 @@ class BaseConfig:
8686
e.g. '"{\"get_bos_id\":128000, \"get_eos_ids\":[128009, 128001]}"'
8787
use_lora: Only for use with QAT. Rank of the LoRA adapter, disabled
8888
if set to 0.
89+
fairseq2: For legacy internal use cases, this is safe to ignore.
90+
preq_mode: Legacy option to specify how prequantized weights are loaded.
91+
Going forward, ExecuTorch supports loading weights prequantized through
92+
TorchAo as-is, without any special handling.
8993
preq_group_size: Legacy option to specify the group size of prequantized weights.
9094
preq_embedding_quantize: Legacy option to specify how prequantized embeddings
9195
are loaded.
@@ -99,6 +103,7 @@ class BaseConfig:
99103
tokenizer_path: Optional[str] = None
100104
metadata: Optional[str] = None
101105
use_lora: int = 0
106+
fairseq2: bool = False
102107
preq_mode: Optional[PreqMode] = None
103108
preq_group_size: int = 32
104109
preq_embedding_quantize: str = "8,0"
@@ -530,6 +535,8 @@ def from_args(cls, args: argparse.Namespace) -> "LlmConfig": # noqa: C901
530535
llm_config.base.metadata = args.metadata
531536
if hasattr(args, "use_lora"):
532537
llm_config.base.use_lora = args.use_lora
538+
if hasattr(args, "fairseq2"):
539+
llm_config.base.fairseq2 = args.fairseq2
533540

534541
# PreqMode settings
535542
if hasattr(args, "preq_mode") and args.preq_mode:

0 commit comments

Comments
 (0)