Skip to content

Commit 7e88212

Browse files
[None][bug] fix export for microsoft/Phi-3-medium-128k-instruct (#10455)
Signed-off-by: Tal Cherckez <[email protected]>
1 parent 8722104 commit 7e88212

File tree

1 file changed

+4
-1
lines changed
  • tensorrt_llm/_torch/auto_deploy/models/patches

1 file changed

+4
-1
lines changed

tensorrt_llm/_torch/auto_deploy/models/patches/phi.py

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -70,9 +70,12 @@ def _patched_phi3_long_emb_init(
7070
self,
7171
):
7272
_patched_phi3_emb_init(self)
73-
self.ext_factors = torch.tensor(
73+
ext_factors_tensor = torch.tensor(
7474
self.short_factor, dtype=torch.float32, device=torch.device("cpu")
7575
)
76+
if hasattr(self, "ext_factors"):
77+
delattr(self, "ext_factors")
78+
self.register_buffer("ext_factors", ext_factors_tensor, persistent=False)
7679

7780

7881
# Copied from https://huggingface.co/microsoft/Phi-3-mini-4k-instruct/blob/main/modeling_phi3.py#L151

0 commit comments

Comments
 (0)