Add Kimi-K2-Instruct multinode inference example#30

Draft

jvmncs wants to merge 6 commits intomainfrom

kimi-k2-inference

Commits on Jul 18, 2025

Add Kimi-K2-Instruct multinode inference example
jvmncs
committed
Add DeepGEMM + DeepEP to image; break code out into several parallelism configs
jvmncs
committed
Fix DeepEP installation; use overlay0 interface to get container rank; add Ray dashboard deps for data parallel; remove OOM parallel configs
jvmncs
committed

Commits on Jul 22, 2025

Use get_cluster_info(); add another tested parallelism config
jvmncs
committed

Commits on Aug 11, 2025

Add load test + alt deployment, and reorganize
jvmncs
committed

Commits on Aug 29, 2025

Extra K2 work
jvmncs
committed