Skip to content

Commit 221a879

Browse files
committed
Adding the support of CCL to the Prefilling of Disaggregated Serving
Signed-off-by: Vahid Janfaza <vjanfaza@qti.qualcomm.com>
1 parent a5943b3 commit 221a879

File tree

1 file changed

+0
-2
lines changed

1 file changed

+0
-2
lines changed

QEfficient/transformers/models/modeling_auto.py

Lines changed: 0 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -3333,8 +3333,6 @@ def compile(
33333333
ccl_lengths = self.comp_ctx_lengths_decode if prefill_seq_len == 1 else self.comp_ctx_lengths_prefill
33343334
# Adding elements from self.comp_ctx_lengths_prefill to prefill_specialization
33353335
for i in range(0, len(ccl_lengths)):
3336-
# if prefill_only or enable_chunking:
3337-
# raise NotImplementedError("prefill_only or enable_chunking is not supported with CCL")
33383336
specializations.append(
33393337
self.build_prefill_specialization(
33403338
prefill_seq_len=prefill_seq_len,

0 commit comments

Comments
 (0)