Skip to content

Commit f56e429

Browse files
authored
Merge pull request #1483 from PaddlePaddle/update-faster-tokenizer-map
Support ppminilm in FasterTokenizer
2 parents 374d31b + 10fc6a6 commit f56e429

File tree

1 file changed

+1
-0
lines changed

1 file changed

+1
-0
lines changed

paddlenlp/experimental/faster_tokenizer.py

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -71,6 +71,7 @@ class FasterTokenizer(nn.Layer):
7171
"ernie-1.0": ErnieTokenizer,
7272
"ernie-2.0-en": ErnieTokenizer,
7373
"ernie-2.0-large-en": ErnieTokenizer,
74+
"ppminilm-6l-768h": ErnieTokenizer,
7475
"roberta-wwm-ext": RobertaTokenizer,
7576
"roberta-wwm-ext-large": RobertaTokenizer,
7677
"rbt3": RobertaTokenizer,

0 commit comments

Comments
 (0)