How to deploy GPT-OSS 120B on KubeAI with multi-node GPU setup?

[SupakritCRO]

Hi everyone 👋

I have a Kubernetes cluster with 2 control plane nodes and 2 worker nodes, each worker has 1× NVIDIA RTX 4090 GPU.

I want to use KubeAI to deploy a GPT-OSS 120B open-weight model as an API or chat GUI for internal use — ideally using distributed inference across the two GPU nodes.

Has anyone tried this setup?

• How does KubeAI handle large models split across nodes?
• What kind of performance or latency difference should I expect compared to running on a single node with 2 GPUs?

Any tips or real-world experience would be appreciated 🙏