How to add Runai support?

[igor-krawczuk]

Hi, is there any documentation on how to add new providers? This PR seems to no longer give a useful link?

Specifically, we use https://www.run.ai/ in my org, which uses a modified kubernetes-yaml file to launch "jobs" which abstract away volume and pods. The "setup" would be either creating a new dockerfile or installing some dependencies, the "run" would be simply a kubernetes container + cmd specification.

What would be the best way to implement this? I guess generating that yaml automatically based on the sky yaml?

[concretevitamin]

Hello @igor-krawczuk -- do you have a ~/.kube/config file that grants access to the RunAI k8s cluster? If so, check out https://skypilot.readthedocs.io/en/latest/reference/kubernetes/index.html which may just work without adding a new provider. cc @romilbhardwaj

[romilbhardwaj]

+1 - if you can run kubectl commands from your local machine, our Kubernetes support should work for you. The current active context in the kubeconfig file is used as the target cluster to run tasks on.

[igor-krawczuk]

Yes, but the way it works is that I have to specify a special kind (RunaiJob) with some specific metadata...normally I launch this via kubectl create -f run-ai-spec.yaml.

Given that this is is backend specific, I would expect this to not be compatible with the sky task.yaml?

[romilbhardwaj]

I see. Our k8s implementation is designed around the core Kubernetes API, but can be extended to use CRDs like RunaiJob. If you want to want to add a new provider for RunAI, here's some resources:

• The Kubernetes cloud class gets the resource spec and other fields and writes it to a ray autoscaler YAML file. The base template for this ray autoscaler YAML can be found here, which contains the Kubernetes pod spec.
• SkyPilot then invokes the ray autoscaler with the YAML above. The control flow is handed to our KubernetesNodeProvider which implements the logic to create, manage and delete relevant Kubernetes resources (pods, svcs etc).
• Once the pods are created and running and ssh access is working, control flow goes back to SkyPilot which proceeds with setup and other tasks.

Note that this is based on our older Ray based provisioning code, also explained in the new-cloud-skeleton repo. Compared to our new provisioner (example GCP implementation), you might find this easier to start with since you can re-use a lot of our k8s implementation.

[romilbhardwaj]

BTW, can you share a bit more about why you have to use RunAI? SkyPilot pods should be able to co-exist with pods from other RunaiJobs.

If there are any missing features, we would be very happy to support them and make our Kubernetes support work for you.

[igor-krawczuk]

BTW, can you share a bit more about why you have to use RunAI? SkyPilot pods should be able to co-exist with pods from other RunaiJobs.

It is simply what the shared internal cloud provider with subsidized GPUs uses to orchestrate, so if I want to use cheap, GPUs I can use grant money for, it's what I have to deal with.

If there are any missing features, we would be very happy to support them and make our Kubernetes support work for you.

Thanks a lot, I'll try with the guidance you have ans might report back here/open a pull requey