Skip to content

Commit dfc655d

Browse files
update recipe
Signed-off-by: Zhiyu Li <zhiyul@NVIDIA.com>
1 parent 0b5789f commit dfc655d

File tree

1 file changed

+7
-3
lines changed

1 file changed

+7
-3
lines changed

examples/llm_finetune/qwen/qwen3_8b_hellaswag_pp_peft.yaml

Lines changed: 7 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -25,6 +25,7 @@ step_scheduler:
2525
ckpt_every_steps: 200
2626
val_every_steps: 100 # will run every x number of gradient steps
2727
num_epochs: 2
28+
max_steps: 100
2829

2930
dist_env:
3031
backend: nccl
@@ -38,7 +39,6 @@ rng:
3839
model:
3940
_target_: nemo_automodel.NeMoAutoModelForCausalLM.from_pretrained
4041
pretrained_model_name_or_path: Qwen/Qwen3-8B
41-
cache_dir: /nemo-rl/hf_cache/
4242

4343
peft:
4444
_target_: nemo_automodel.components._peft.lora.PeftConfig
@@ -82,7 +82,9 @@ packed_sequence:
8282

8383
dataloader:
8484
_target_: torchdata.stateful_dataloader.StatefulDataLoader
85-
collate_fn: nemo_automodel.components.datasets.utils.default_collater
85+
collate_fn:
86+
_target_: nemo_automodel.components.datasets.utils.default_collater
87+
pad_seq_len_divisible: 320
8688
shuffle: true
8789

8890
validation_dataset:
@@ -93,7 +95,9 @@ validation_dataset:
9395

9496
validation_dataloader:
9597
_target_: torchdata.stateful_dataloader.StatefulDataLoader
96-
collate_fn: nemo_automodel.components.datasets.utils.default_collater
98+
collate_fn:
99+
_target_: nemo_automodel.components.datasets.utils.default_collater
100+
pad_seq_len_divisible: 320
97101

98102
optimizer:
99103
_target_: torch.optim.Adam

0 commit comments

Comments
 (0)