Skip to content

Commit 1baf849

Browse files
[ci] convert lcnc test to use tp 2, till the issue is fixed in neuron… (#2696)
1 parent 7b93555 commit 1baf849

File tree

1 file changed

+2
-0
lines changed

1 file changed

+2
-0
lines changed

tests/integration/llm/prepare.py

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -239,6 +239,8 @@
239239
"option.model_id": "s3://djl-llm/tinyllama-1.1b-chat/",
240240
"option.rolling_batch": "auto",
241241
"option.model_loading_timeout": 3600,
242+
# TODO: Remove this once neuron 2.21 issue if fixed.
243+
"option.tensor_parallel_degree": 2,
242244
},
243245
"tiny-llama-rb-aot": {
244246
"option.model_id": "s3://djl-llm/tinyllama-1.1b-chat/",

0 commit comments

Comments
 (0)