Commit b8bedb2
Added information about the use of slow tokenizers (#2517)
Added information about the use of slow tokenizers to generate vocab
files in ML.
---------
Co-authored-by: David Kyle <[email protected]>
Co-authored-by: Visha Angelova <[email protected]>
Co-authored-by: shainaraskas <[email protected]>
Co-authored-by: Brandon Morelli <[email protected]>1 parent 1affd66 commit b8bedb2
1 file changed
+1
-3
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
15 | 15 | | |
16 | 16 | | |
17 | 17 | | |
18 | | - | |
19 | | - | |
20 | | - | |
| 18 | + | |
21 | 19 | | |
22 | 20 | | |
23 | 21 | | |
| |||
0 commit comments