Unable to connect to Proxmox cluster

[calebsmithdev]

My Proxmox setup: A single node with 3 controlplanes and 1 worker VM. Currently in a Proxmox cluster with my single node.

This is almost certainly a "me" problem, but I am stumped on what I am doing wrong. At the moment, I have everything configured and working from what I can visually see in ArgoCD. The only thing not working is my PersistentVolumeClaim status is "Released", and I do not see the drive attached to my one worker VM. After running kubectl -n csi-proxmox logs -f proxmox-csi-plugin-controller-** I can see the error message GetCapacity: failed to get proxmox cluster, which I assume is the ultimate issue.

The logs prior to this point:
"GetCapacity: called" args="{\"accessible_topology\":{\"segments\":{\"topology.kubernetes.io/region\":\"smith-lab\",\"topology.kubernetes.io/zone\":\"the-smith\"}},\"parameters\":{\"cache\":\"writethrough\",\"csi.storage.k8s.io/fstype\":\"ext4\",\"ssd\":\"true\",\"storage\":\"local-lvm\"},\"volume_capabilities\":[{\"AccessType\":{\"Mount\":{}},\"access_mode\":{}}]}"

"GetCapacity" region="smith-lab" zone="the-smith" storageID="local-lvm"

"GetCapacity: failed to get proxmox cluster" err="proxmox cluster smith-lab not found" cluster="smith-lab"

GRPC error: rpc error: code = Internal desc = proxmox cluster smith-lab not found

When I run a status check on my Proxmox (/api2/json/cluster/status), I see the following:

{
  "data": [
    {
      "id": "cluster",
      "version": 1,
      "type": "cluster",
      "name": "smith-lab",
      "nodes": 1,
      "quorate": 1
    },
    {
      "level": "",
      "nodeid": 1,
      "online": 1,
      "local": 1,
      "name": "the-smith",
      "ip": "192.168.1.98",
      "type": "node",
      "id": "node/the-smith"
    }
  ]
}

I think I am spelling everything correctly, and I have my roles configured correctly. Does anyone have any insight on ideas for what I could test to see what I am missing from a config standpoint?

[sergelogvinov]

Hello, proxmox cluster smith-lab not found

https://github.com/sergelogvinov/proxmox-csi-plugin/blob/main/docs/install.md#install-the-plugin-by-using-kubectl

Cluster name is region in config file, You're right, it's not very clear at a quick glance...

clusters:
  - url: https://cluster-api-1.exmple.com:8006/api2/json
    insecure: false
    token_id: "kubernetes-csi@pve!csi"
    token_secret: "secret"
    region: Region-1 <--- cluster name

[calebsmithdev]

I checked that! It seems to be okay. I am cheating and using the Terraform script of another homelabber who posted here a while back, but my cluster config ends up looking like this (redacted secret) from a Secret config file:

clusters:
- url: "https://192.168.1.98:8006/api2/json"
  insecure: true
  token_id: "kubernetes-csi@pve!csi"
  token_secret: "secret"
  region: smith-lab

Which seems to be correct. But if nothing looks immediately off to you, then I'll just assume It's still some random config thing that I forgot somewhere. It seemed so close to getting everything connected that some one thing must obviously be off. I'll try a different role/secret and see what that does. Maybe it's just failing at a different connection point that I am missing.

EDIT: Recreating my proxmox-csi namespace as a whole fixed it, so it might have been something cached or something and I didn't think to recreate it beforehand 🤦 I still have one other issue going on, but at least I'm past that annoying issue I couldn't figure out 😓

[calebsmithdev]

Finally got it! Ultimately, I just needed to recreate the proxmox-csi-plugin-controller, and then it was firing off just fine. It was using one of my test credentials still, which I am guessing was the original issue I ran into. Once I erased it and recreated the secret, then that all fired off correctly. Next, I needed to give it more RAM to actually deal with the pod that I was experimenting with. I was just chasing the wrong issues all day yesterday.

The VM drives all moved to the worker as expected and things are working today! Yay! Thanks so much for building this. Very helpful.