Running on DGX Spark

[JayShukla8]

Hello there,

trying to run the experimental branch on a DGX Spark.

When trying to run in the intent-based router mode, all containers except the qwen-router get started successfully but the qwen-router container keeps on waiting.

Here are the logs for qwen-router container where they get stuck

Question:

Have you guys faced any issue when running on Spark? How did you resolve it? Is ARM64 architecture supported?

Thanks and Regards,
Jay

[slopp]

We have been able to successfully load the routing Qwen model on the Spark, though it sometimes can take time (10+ minutes in some cases). Can you share the output of nvidia-smi ? What other models are you running at the same time?