From b75c42f460d194498a0e0736bb792917fd0aedf0 Mon Sep 17 00:00:00 2001
From: Jacob Szwejbka <jakeszwe@meta.com>
Date: Thu, 16 Oct 2025 13:55:04 -0700
Subject: [PATCH] Change tokenizer pin to get msvc change

Summary: pull in https://github.com/meta-pytorch/tokenizers/commit/15066901652d9bf5d371ab93d4a266d1506ee8df

Differential Revision: D84854271
---
 extension/llm/tokenizers | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/extension/llm/tokenizers b/extension/llm/tokenizers
index d710a0cf10c..15066901652 160000
--- a/extension/llm/tokenizers
+++ b/extension/llm/tokenizers
@@ -1 +1 @@
-Subproject commit d710a0cf10cfa8cb7ffda33c4e61af63119bc95f
+Subproject commit 15066901652d9bf5d371ab93d4a266d1506ee8df