Unable to add the host: Cannot find the server resources

[KobesM]

ISSUE TYPE

• Other

COMPONENT NAME

cloudstack-agent

CLOUDSTACK VERSION

4.19

OS / ENVIRONMENT

• single cloudstack management 4.19 server fresh install on Almalinux 9.3
• cloudstack agent 4.19 fresh install on Almalinux 9.3

SUMMARY

This is my first time setting up apache cloudstack. After installing my management server and having it up and running I am trying to add my first KVM host. When adding the host from the UI I receive the following error:

Error 530
Unable to add the host: Cannot find the server resources at http://xxxxxxxx

I found that the cloudstack-agent is not running on the host:

]# systemctl status cloudstack-agent
● cloudstack-agent.service - CloudStack Agent
     Loaded: loaded (/usr/lib/systemd/system/cloudstack-agent.service; enabled; preset: disabled)
     Active: activating (auto-restart) (Result: exit-code) since Sat 2024-03-30 11:42:23 CET; 5s ago
       Docs: http://www.cloudstack.org/
    Process: 163327 ExecStart=/usr/bin/java $JAVA_OPTS $JAVA_DEBUG -cp $CLASSPATH $JAVA_CLASS (code=exited, status=67)
   Main PID: 163327 (code=exited, status=67)
        CPU: 2.548s

The agent log on the host has the following:

2024-03-30 11:43:16,759 INFO  [cloud.agent.AgentShell] (main:null) (logid:) Agent started
2024-03-30 11:43:16,763 INFO  [cloud.agent.AgentShell] (main:null) (logid:) Implementation Version is 4.19.0.0
2024-03-30 11:43:16,764 INFO  [cloud.agent.AgentShell] (main:null) (logid:) agent.properties found at /etc/cloudstack/agent/agent.properties
2024-03-30 11:43:16,809 INFO  [cloud.agent.AgentShell] (main:null) (logid:) Defaulting to using properties file for storage
2024-03-30 11:43:16,809 INFO  [cloud.agent.AgentShell] (main:null) (logid:) Defaulting to the constant time backoff algorithm
2024-03-30 11:43:16,850 INFO  [cloud.utils.LogUtils] (main:null) (logid:) log4j configuration found at /etc/cloudstack/agent/log4j-cloud.xml
2024-03-30 11:43:16,850 INFO  [cloud.agent.AgentShell] (main:null) (logid:) Using default Java settings for IPv6 preference for agent connection
2024-03-30 11:43:16,925 INFO  [cloud.agent.Agent] (main:null) (logid:) id is
2024-03-30 11:43:16,936 ERROR [kvm.resource.LibvirtComputingResource] (main:null) (logid:) uefi properties file not found due to: Unable to find file uefi.properties.
2024-03-30 11:43:16,982 INFO  [kvm.resource.LibvirtConnection] (main:null) (logid:) No existing libvirtd connection found. Opening a new one
2024-03-30 11:43:17,075 ERROR [cloud.agent.AgentShell] (main:null) (logid:) Unable to start agent:
com.cloud.utils.exception.CloudRuntimeException: Failed to connect socket to '/var/run/libvirt/virtqemud-sock': Connection refused
        at com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.configure(LibvirtComputingResource.java:1131)
        at com.cloud.agent.Agent.<init>(Agent.java:193)
        at com.cloud.agent.AgentShell.launchNewAgent(AgentShell.java:452)
        at com.cloud.agent.AgentShell.launchAgentFromClassInfo(AgentShell.java:431)
        at com.cloud.agent.AgentShell.launchAgent(AgentShell.java:415)
        at com.cloud.agent.AgentShell.start(AgentShell.java:511)
        at com.cloud.agent.AgentShell.main(AgentShell.java:541)

STEPS TO REPRODUCE

EXPECTED RESULTS

ACTUAL RESULTS

[boring-cyborg]

Thanks for opening your first issue here! Be sure to follow the issue template!

[KobesM]

I solved the issue by adding the user I used to connect to the KVM host to the wheel group. I do not know if this is actually required since it is not documented as such.

[weizhouapache]

I solved the issue by adding the user I used to connect to the KVM host to the wheel group. I do not know if this is actually required since it is not documented as such.

@KobesM
thanks for sharing
I think it is required

[KobesM]

I did some further investigation and it seems I do not have to add the user to the wheel group. If I add the host for the first time I receive the error:

Error 530
Unable to add the host: Cannot find the server resources at http://<fqdn>

Trying to add it again does not change the state and I receive this error every time.

However when I reboot the server (after trying to add it for the first time) I get the following error:

Error 530
Could not add host at [http://<fqdn>] with zone [1], pod [1] and cluster [1] due to: [ can't setup agent, due to com.cloud.utils.exception.CloudRuntimeException: Skipping host<ip> because 37d27800-15cc-341d-8fed-4ae730f54b58 is already in the database for resource 37d27800-15cc-341d-8fed-4ae730f54b58-LibvirtComputingResource with ID 235783ee-0606-4212-928b-7c3947d46db2 - Skipping host <ip> because 37d27800-15cc-341d-8fed-4ae730f54b58 is already in the database for resource 37d27800-15cc-341d-8fed-4ae730f54b58-LibvirtComputingResource with ID 235783ee-0606-4212-928b-7c3947d46db2].

And after I close the wizard and refresh the page the host is listed and works as expected. So something seems to "trigger" by rebooting the server after trying to add the host for the first time.

[weizhouapache]

I did some further investigation and it seems I do not have to add the user to the wheel group. If I add the host for the first time I receive the error:

Error 530
Unable to add the host: Cannot find the server resources at http://<fqdn>

Trying to add it again does not change the state and I receive this error every time.

However when I reboot the server (after trying to add it for the first time) I get the following error:

Error 530
Could not add host at [http://<fqdn>] with zone [1], pod [1] and cluster [1] due to: [ can't setup agent, due to com.cloud.utils.exception.CloudRuntimeException: Skipping host<ip> because 37d27800-15cc-341d-8fed-4ae730f54b58 is already in the database for resource 37d27800-15cc-341d-8fed-4ae730f54b58-LibvirtComputingResource with ID 235783ee-0606-4212-928b-7c3947d46db2 - Skipping host <ip> because 37d27800-15cc-341d-8fed-4ae730f54b58 is already in the database for resource 37d27800-15cc-341d-8fed-4ae730f54b58-LibvirtComputingResource with ID 235783ee-0606-4212-928b-7c3947d46db2].

And after I close the wizard and refresh the page the host is listed and works as expected. So something seems to "trigger" by rebooting the server after trying to add the host for the first time.

so it looks same as #8597 ? @KobesM

[printfdebugging]

I was getting the same error. Turns out that reading the log files helps, as a log file is cloudstack's way of communicating back what's the issue. I searched for "server resources" in the log file, and found that it's related to libvirtd. Then I checked for if libvirtd is running is fine or not using # systemctl status libvirtd, and I found that it wasn't. Then I ran journalctl -xe as asked in the libvirtd status info, and found that there was some syntax error in libvirtd.conf.

I hope it helps somehow.

[yadvr]

Same as #8824 - sounds like setup/config issue of kvm/libvirtd