Integrating kubectl-ai with Custom Chatbot Interface

[Ketan0493]

Hi,

I'm relatively new to the AI space, but I’ve been really impressed with this tool—it offers a great way for teams to run Kubernetes commands with minimal prior knowledge.

I'm currently working on integrating it into our Kubernetes cluster with a custom chatbot interface layered on top. I’d appreciate some guidance on the following points:

1. I’m running kubectl-ai inside a pod and exposing it as a service. However, when making API calls, it appears to operate in --quiet mode and doesn't respond interactively. Additionally it doesn't stores the context.

2. I’ve also explored the built-in UI mode, but found it less intuitive for our use case. Hence, I’m aiming to build a more user-friendly chatbot frontend.

If anyone has implemented a similar setup or has suggestions on how to best approach this integration, I’d be grateful for your insights.

[kamleshjoshi8102]

@Ketan0493 did you found any workaround? I'm also facing similar issue, we have deployed this in k8s and using it to troubleshoot the issues in the cluster but it's not maintaining it's context and it's only for single user only. please let me know if you found anything.

[kamleshjoshi8102]

• @droot

[droot]

but it's not maintaining it's context and it's only for single user only.

Can you say more about this ?

I will share more details about the plans for running it in the cluster in a multi-tenant mode.

[kamleshjoshi8102]

@droot thanks for looking into this, we have deployed this as webui and it's ingress is behind an IAP. so whenever a user is asking any questions it's being shown to another user who's opening the webui ingress.

regarding context, when pods are being deleted it's history is lost, maybe some volumeMounts would do the trick here. but I'm mostly interested in multi-tenant session handling in the shared web UI mode.

[gilad-tayeb]

Hi @Ketan0493
I'm not sure if it's exactly what you meant, but i've implemented it to (one of) my clusters as a microservice, in order - as you said - diagnose and have some visibility on my cluster.
I'm running on EKS (if it matters), and i'm using AzureOpenAI as llm provider.
My method (which works currently) is:

• Created an IAM role, no policy attached, simply the trust policy which is configured with the cluster oidc (aud/sub with relevant principle).
• Created the k8s objects: deployment, service, ingress (i'm running with kong, and configured accordingly), serviceAccount (which is configured to the IAM role stated above), clusterRole and clusterRoleBinding (currently for testing purposes - only configured to read-only mode, meaning it can observe all resources but can't delete anything), secret (for my api-key and endpoint), configMap (for any relevant configurations, such as llm-provider, model, api-key which is referencing the secret etc.) and some more objects.
  I've exposes my service using a not-so-random-port (act surprised: 8080 XD) and it works. I'm facing other issues, such as the ability to see other users sessions (although i've used the flag --new-session by default, FYI @droot ) but i'll figure it out soon I guess.

If you have the same configurations as I have - feel free to ask any question, i'd be happy to help

[kamleshjoshi8102]

great! again thanks for sharing all the details.
Regarding the design partner - yes please let me know when we can connect.

[kamleshjoshi8102]

Hi @droot ,
Reaching out again to check if we have planned in this year or early next year, to rollout of the multiple sessions.

[droot]

Good timing :) @ShubyM spent sometime on this effort last few weeks to get something we can get early feedback on. Here is the PR that's under review:

#620

Here is a quick summary of how this works:

1. When running in web mode, the server supports multiple sessions. Each session is isolated using sandbox pod. What that means is that kubectl commands for a corresponding sessions run in a kubernetes pods.
2. We haven't added any user authn/authz yet. That is something we want to add soon but need some feedback on it especially it has an interplay with the permissions with the sandbox pods.

The PR also has a guide for deploying to GKE. The PR is low on documentation, but @ShubyM will be more than happy to jump on a call and help clarify things.

We are thinking of having it in a branch to bake it and then merge it in the main. If you can try it out and share the early feedback, it will be great.

One of other things this enables is that the backend has an API that other programs can call to trigger an agentic session, so this can help in integrating with other systems.

Is there an email we can connect on to discuss this further ?

/cc @gilad-tayeb @kamleshjoshi8102

[droot]

Update: we merged the PR. Treat the multi-session UI to be an alpha feature for now so we plan to evolve this rapidly.

[kamleshjoshi8102]

Thank you @droot ! that was very quick. I'll check if I can play around this early next week and will share feedback as well.
personal mail: (j.kamlesh8102@gmail.com)