Skip to content
This repository was archived by the owner on Sep 10, 2025. It is now read-only.

Commit 978baa3

Browse files
committed
Add setup_cache for aoti_package_path
1 parent a31ab39 commit 978baa3

File tree

1 file changed

+5
-0
lines changed

1 file changed

+5
-0
lines changed

torchchat/cli/builder.py

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -632,6 +632,11 @@ def do_nothing(max_batch_size, max_seq_length):
632632
aoti_compiled_model = load_package(
633633
str(builder_args.aoti_package_path.absolute())
634634
)
635+
636+
def do_nothing(max_batch_size, max_seq_length):
637+
pass
638+
model.setup_caches = do_nothing
639+
635640
model.forward = aoti_compiled_model
636641
metadata = aoti_compiled_model.get_metadata()
637642
builder_args.device = metadata["AOTI_DEVICE_KEY"]

0 commit comments

Comments
 (0)