[Dataset Performance] Add num workers on dataset processing - labels, tokenization#1189

dsikka merged 3 commits intomainvllm-project/llm-compressor:mainfrom

num-proc-datasetvllm-project/llm-compressor:num-proc-datasetCopy head branch name to clipboard