Skip to content

Commit 50c43a3

Browse files
talumbaucopybara-github
authored andcommitted
Enable DUS-based KV Cache update for OSS
PiperOrigin-RevId: 705986742
1 parent 7074ae7 commit 50c43a3

File tree

1 file changed

+1
-2
lines changed

1 file changed

+1
-2
lines changed

ai_edge_torch/generative/layers/kv_cache.py

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -20,6 +20,7 @@
2020

2121
from ai_edge_torch import hlfb
2222
from ai_edge_torch.generative.layers import model_config
23+
from ai_edge_torch.generative.utilities.dynamic_update_slice import dynamic_update_slice
2324
import torch
2425
import torch.utils._pytree as pytree
2526

@@ -159,8 +160,6 @@ def update(
159160
Returns:
160161
KVCacheEntry: The updated KVCache entry based on the passed inputs.
161162
"""
162-
# Turn dynamic_update_slice updates off for now.
163-
use_dus=False
164163
update_kv_cache = _update_kv_impl if use_dus else _update_kv_base_impl
165164
return update_kv_cache(cache, input_pos, k_slice, v_slice)
166165

0 commit comments

Comments
 (0)