Skip to content
This repository was archived by the owner on Jun 3, 2025. It is now read-only.

Commit eb15ca6

Browse files
committed
fix token classification example command in nlp use cases
1 parent fafc301 commit eb15ca6

File tree

1 file changed

+8
-7
lines changed

1 file changed

+8
-7
lines changed

src/content/use-cases/natural-language-processing/token-classification.mdx

Lines changed: 8 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -38,7 +38,7 @@ There are some additional tutorials for this functionality on GitHub.
3838

3939
### Sparsifying Popular Transformer Models
4040

41-
In the example below, a dense BERT model is sparsified and fine-tuned on the CoNLL-2003 dataset.
41+
In the example below, a dense BERT model is sparsified and fine-tuned on the CoNLL-2003 dataset.
4242

4343
```bash
4444
sparseml.transformers.token_classification \
@@ -52,14 +52,14 @@ sparseml.transformers.token_classification \
5252
--recipe zoo:nlp/token_classification/bert-base/pytorch/huggingface/conll2003/12layer_pruned80_quant-none-vnni
5353
```
5454

55-
The SparseML train script is a wrapper around a [HuggingFace script](https://huggingface.co/docs/transformers/run_scripts),
55+
The SparseML train script is a wrapper around a [HuggingFace script](https://huggingface.co/docs/transformers/run_scripts),
5656
and usage for most arguments follows the HuggingFace. The most important arguments for SparseML are:
5757

5858
- `--model_name_or_path` indicates which model to start the pruning process from. It can be a SparseZoo stub, HF model identifier, or a path to a local model.
5959
- `--recipe` points to recipe file containing the sparsification hyperparamters. It can be a SparseZoo stub or a local file. For more on creating a recipe see [here](/user-guide/recipes/creating).
6060
- `--dataset_name` indicates that we should fine tune on the CoNLL-2003 dataset.
6161

62-
To utilize a custom dataset, use the `--train_file` and `--validation_file` arguments. To use a dataset from the HuggingFace hub, use `--dataset_name`.
62+
To utilize a custom dataset, use the `--train_file` and `--validation_file` arguments. To use a dataset from the HuggingFace hub, use `--dataset_name`.
6363
See the [HF Docs](https://huggingface.co/docs/transformers/run_scripts#run-a-script) for more details.
6464

6565
Run the following to see the full list of options:
@@ -84,10 +84,10 @@ sparseml.transformers.token_classification \
8484
--recipe zoo:nlp/masked_language_modeling/bert-base/pytorch/huggingface/wikipedia_bookcorpus/12layer_pruned80_quant-none-vnni?recipe_type=transfer-token_classification
8585
```
8686

87-
This usage of the script is the same as the above.
87+
This usage of the script is the same as the above.
8888

8989
In this example, however, the starting model is a pruned-quantized version of BERT from SparseZoo (rather than a dense BERT)
90-
and the recipe is a transfer learning recipe, which instructs Transformers to maintain sparsity of the base model (rather than
90+
and the recipe is a transfer learning recipe, which instructs Transformers to maintain sparsity of the base model (rather than
9191
a recipe that sparsifies a model from scratch).
9292

9393
#### Knowledge Distillation
@@ -108,10 +108,11 @@ sparseml.transformers.token_classification \
108108
--do_train \
109109
--do_eval \
110110
--output_dir models/teacher \
111-
--recipe zoo:nlp/masked_language_modeling/bert-base/pytorch/huggingface/wikipedia_bookcorpus/12layer_pruned80_quant-none-vnni?recipe_type=transfer-token_classification
111+
--recipe zoo:nlp/masked_language_modeling/bert-base/pytorch/huggingface/wikipedia_bookcorpus/12layer_pruned80_quant-none-vnni?recipe_type=transfer-token_classification \
112+
--distill_teacher zoo:nlp/token_classification/bert-base/pytorch/huggingface/conll2003/base-none
112113
```
113114

114-
Once the dense teacher is trained we may reuse it for KD in Sparsification or Sparse Transfer learning.
115+
Once the dense teacher is trained we may reuse it for KD in Sparsification or Sparse Transfer learning.
115116
Simply pass the path to the directory with the teacher model to the `--distill_teacher` argument. For example:
116117

117118
```bash

0 commit comments

Comments
 (0)