Skip to content

Commit 537d30b

Browse files
authored
Bump tokenizers to 4ed91cc (#13888)
### Summary Update the tokenizers dependency to 4ed91cc to pick up the Windows changes. Note that this also involves updating the submodule URL, as it's been recently migrated from pytorch-labs to meta-pytorch. ### Test plan CI
1 parent c393d17 commit 537d30b

File tree

2 files changed

+2
-2
lines changed

2 files changed

+2
-2
lines changed

.gitmodules

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -27,7 +27,7 @@
2727
url = https://github.com/google/pthreadpool.git
2828
[submodule "extension/llm/tokenizers"]
2929
path = extension/llm/tokenizers
30-
url = https://github.com/pytorch-labs/tokenizers.git
30+
url = https://github.com/meta-pytorch/tokenizers.git
3131
[submodule "kernels/optimized/third-party/eigen"]
3232
path = kernels/optimized/third-party/eigen
3333
url = https://gitlab.com/libeigen/eigen.git

0 commit comments

Comments
 (0)