Skip to content

Commit 4655850

Browse files
jackzhxngfacebook-github-bot
authored andcommitted
Enable kwarg inputs for pt2e quantize (#7436)
Summary: For quantizing models that have kwarg forward() inputs, such as TorchTune Llama models Test Plan: N/A Reviewed By: tarun292 Differential Revision: D70206003 Pulled By: jackzhxng
1 parent 4b85ee2 commit 4655850

File tree

1 file changed

+4
-1
lines changed

1 file changed

+4
-1
lines changed

extension/llm/export/builder.py

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -360,7 +360,10 @@ def pt2e_quantize(self, quantizers: Optional[List[Quantizer]]) -> "LLMEdgeManage
360360
logging.info(
361361
"No calibration provided, using dummy input to calibrate..."
362362
)
363-
m(*self.example_inputs)
363+
if self.example_kwarg_inputs:
364+
m(*self.example_inputs, **self.example_kwarg_inputs)
365+
else:
366+
m(*self.example_inputs)
364367
m = convert_pt2e(m)
365368
DuplicateDynamicQuantChainPass()(m)
366369
self.pre_autograd_graph_module = m

0 commit comments

Comments
 (0)