Skip to content

Commit 77e4f4c

Browse files
Georgedsikka
andauthored
[Dataset Performance] Add num workers on dataset processing - labels, tokenization (#1189)
SUMMARY: * Add `preprocessing_num_workers` to run dataset processing in parallel for 2:4 example. Before: Tokenizing: 371.12 examples/s, Adding labels: 1890.18 examples/s, Tokenizing: 333.39 examples/s ```bash Tokenizing: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 12802/12802 [00:34<00:00, 371.12 examples/s] Adding labels: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████| 12802/12802 [00:06<00:00, 1890.18 examples/s] Tokenizing: 9%|█████████▌ | 22077/256032 [00:59<11:41, 333.39 examples/s ``` After (num_proc=8): Tokenizing: 2703.93 examples/s, Adding labels: 5524.98 examples/s, Tokenizing: 2925.98 examples/s ```bash Tokenizing (num_proc=8): 100%|██████████████████████████████████████████████████████████████████████████████████████████████████| 12802/12802 [00:04<00:00, 2703.93 examples/s] Adding labels (num_proc=8): 100%|███████████████████████████████████████████████████████████████████████████████████████████████| 12802/12802 [00:02<00:00, 5524.98 examples/s] Tokenizing (num_proc=8): 100%|████████████████████████████████████████████████████████████████████████████████████████████████| 256032/256032 [01:27<00:00, 2925.98 examples/s] ``` TEST PLAN: * Pass existing tests Co-authored-by: Dipika Sikka <dipikasikka1@gmail.com>
1 parent d3d2d1d commit 77e4f4c

File tree

1 file changed

+2
-0
lines changed

1 file changed

+2
-0
lines changed

examples/quantization_2of4_sparse_w4a16/llama7b_sparse_w4a16.py

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -33,6 +33,7 @@
3333
bf16 = False # using full precision for training
3434
lr_scheduler_type = "cosine"
3535
warmup_ratio = 0.1
36+
preprocessing_num_workers = 8
3637

3738
# this will run the recipe stage by stage:
3839
# oneshot sparsification -> finetuning -> oneshot quantization
@@ -52,6 +53,7 @@
5253
learning_rate=learning_rate,
5354
lr_scheduler_type=lr_scheduler_type,
5455
warmup_ratio=warmup_ratio,
56+
preprocessing_num_workers=preprocessing_num_workers,
5557
)
5658
logger.info(
5759
"llmcompressor does not currently support running compressed models in the marlin24 format." # noqa

0 commit comments

Comments
 (0)