The consoleproxy is not starting and console of the instance not starting, too

[ghost]

ISSUE TYPE

• Bug Report

COMPONENT NAME

API

CLOUDSTACK VERSION

4.19.0.1 

CONFIGURATION

OS / ENVIRONMENT

Ubuntu Server 24.04

SUMMARY

Although it is possible to create the instance, it is not possible to access the instance console.

STEPS TO REPRODUCE

When starting the console of the instance an error occurs

In consoleproxy it does not start either

Filter by date

grep -a '2024-06-22' management-server.log.copy

The file is very long, so I share the complete log by gist

Last error with console proxy failure
2024-06-22 18:17:20,047 INFO [o.a.c.f.j.i.AsyncJobMonitor] (Work-Job-Executor-106:ctx-f07acb36 job-1655/job-1761) (logid:79ff0ac2) Remove job-1761 from job monitoring
2024-06-22 18:17:20,066 DEBUG [c.c.c.CapacityManagerImpl] (consoleproxy-1:ctx-ecf20f94) (logid:6b8fc04e) VM instance {"id":486,"instanceName":"v-486-VM","type":"ConsoleProxy","uuid":"1d39808f-517f-461c-b0c8-7dcfe259185d"} state transited from [Stopped] to [Expunging] with event [ExpungeOperation]. VM's original host: null, new host: null, host before state transition: null
2024-06-22 18:17:20,072 DEBUG [c.c.v.VirtualMachineManagerImpl] (consoleproxy-1:ctx-ecf20f94) (logid:6b8fc04e) Expunging vm VM instance {"id":486,"instanceName":"v-486-VM","type":"ConsoleProxy","uuid":"1d39808f-517f-461c-b0c8-7dcfe259185d"}
2024-06-22 18:17:20,072 DEBUG [c.c.v.VirtualMachineManagerImpl] (consoleproxy-1:ctx-ecf20f94) (logid:6b8fc04e) Cleaning up NICS [] of VM instance {"id":486,"instanceName":"v-486-VM","type":"ConsoleProxy","uuid":"1d39808f-517f-461c-b0c8-7dcfe259185d"}.
2024-06-22 18:17:20,072 DEBUG [o.a.c.e.o.NetworkOrchestrator] (consoleproxy-1:ctx-ecf20f94) (logid:6b8fc04e) Cleaning network for vm: 486
2024-06-22 18:17:20,080 DEBUG [c.c.n.NetworkModelImpl] (consoleproxy-1:ctx-ecf20f94) (logid:6b8fc04e) Service SecurityGroup is not supported in the network id=208
2024-06-22 18:17:20,083 DEBUG [c.c.n.g.DirectNetworkGuru] (consoleproxy-1:ctx-ecf20f94) (logid:6b8fc04e) Deallocate network: networkId: 208, ip: null
2024-06-22 18:17:20,088 DEBUG [o.a.c.e.o.NetworkOrchestrator] (consoleproxy-1:ctx-ecf20f94) (logid:6b8fc04e) Removed nic id=1395
2024-06-22 18:17:20,090 DEBUG [o.a.c.e.o.NetworkOrchestrator] (consoleproxy-1:ctx-ecf20f94) (logid:6b8fc04e) Revoving nic secondary ip entry ...
2024-06-22 18:17:20,101 DEBUG [o.a.c.e.o.NetworkOrchestrator] (consoleproxy-1:ctx-ecf20f94) (logid:6b8fc04e) Removed nic id=1396
2024-06-22 18:17:20,104 DEBUG [o.a.c.e.o.NetworkOrchestrator] (consoleproxy-1:ctx-ecf20f94) (logid:6b8fc04e) Revoving nic secondary ip entry ...
2024-06-22 18:17:20,120 DEBUG [o.a.c.e.o.NetworkOrchestrator] (consoleproxy-1:ctx-ecf20f94) (logid:6b8fc04e) Removed nic id=1397
2024-06-22 18:17:20,122 DEBUG [o.a.c.e.o.NetworkOrchestrator] (consoleproxy-1:ctx-ecf20f94) (logid:6b8fc04e) Revoving nic secondary ip entry ...
2024-06-22 18:17:20,123 DEBUG [c.c.v.VirtualMachineManagerImpl] (consoleproxy-1:ctx-ecf20f94) (logid:6b8fc04e) Cleaning up hypervisor data structures (ex. SRs in XenServer) for managed storage. Data from VM instance {"id":486,"instanceName":"v-486-VM","type":"ConsoleProxy","uuid":"1d39808f-517f-461c-b0c8-7dcfe259185d"}.
2024-06-22 18:17:20,126 DEBUG [o.a.c.e.o.VolumeOrchestrator] (consoleproxy-1:ctx-ecf20f94) (logid:6b8fc04e) Cleaning storage for VM [null].
2024-06-22 18:17:20,163 INFO [o.a.c.s.v.VolumeServiceImpl] (consoleproxy-1:ctx-ecf20f94) (logid:6b8fc04e) Expunge volume with no data store specified
2024-06-22 18:17:20,167 INFO [o.a.c.s.v.VolumeServiceImpl] (consoleproxy-1:ctx-ecf20f94) (logid:6b8fc04e) Volume 489 is not referred anywhere, remove it from volumes table
2024-06-22 18:17:20,167 DEBUG [c.c.s.d.VolumeDaoImpl] (consoleproxy-1:ctx-ecf20f94) (logid:6b8fc04e) Removing volume 489 from DB
2024-06-22 18:17:20,180 DEBUG [c.c.v.VirtualMachineManagerImpl] (consoleproxy-1:ctx-ecf20f94) (logid:6b8fc04e) Expunged VM instance {"id":486,"instanceName":"v-486-VM","type":"ConsoleProxy","uuid":"1d39808f-517f-461c-b0c8-7dcfe259185d"}
2024-06-22 18:17:20,190 DEBUG [c.c.a.ConsoleProxyAlertAdapter] (consoleproxy-1:ctx-ecf20f94) (logid:6b8fc04e) received console proxy alert
2024-06-22 18:17:20,191 DEBUG [c.c.a.ConsoleProxyAlertAdapter] (consoleproxy-1:ctx-ecf20f94) (logid:6b8fc04e) Console proxy creation failure. Zone [cloud-zone].
2024-06-22 18:17:20,193 WARN [c.c.a.AlertManagerImpl] (consoleproxy-1:ctx-ecf20f94) (logid:6b8fc04e) alertType=[10] dataCenterId=[2] podId=[null] clusterId=[null] message=[Console proxy creation failure. Zone [cloud-zone]. Error details: null].
2024-06-22 18:17:20,199 WARN [c.c.a.AlertManagerImpl] (consoleproxy-1:ctx-ecf20f94) (logid:6b8fc04e) No recipients set in global setting 'alert.email.addresses', skipping sending alert with subject [Console proxy creation failure. Zone [cloud-zone]. Error details: null] and content [Console proxy creation failure (zone cloud-zone)].
2024-06-22 18:17:23,187 DEBUG [o.a.c.h.HAManagerImpl] (BackgroundTaskPollManager-6:ctx-40668147) (logid:591f5e3e) HA health check task is running...
2024-06-22 18:17:24,094 DEBUG [c.c.a.m.AgentManagerImpl] (AgentManager-Handler-5:null) (logid:) Ping from 21(s-461-VM)
2024-06-22 18:17:27,190 DEBUG [o.a.c.h.HAManagerImpl] (BackgroundTaskPollManager-1:ctx-3df156e4) (logid:35a16dad) HA health check task is running...

In the libvirtd service

EXPECTED RESULTS

To start the virtual machine console

ACTUAL RESULTS

Unable to start virtual machine console

[boring-cyborg[bot]]

Thanks for opening your first issue here! Be sure to follow the issue template!

[sureshanaparti]

@saengraciaEspe I'm able to see the console proxy running and console of system VMs & instances. Can you share the management server log with console proxy start failures?

[ghost]

@saengraciaEspe I'm able to see the console proxy running and console of system VMs & instances. Can you share the management server log with console proxy start failures?

Done

[rajujith]

@saengraciaEspe Based on the logs cluster 1 does not have free MEMORY capacity to deploy a console proxy VM that needs 1 GB RAM. Could you check your host/cluster free capacity? Is there a cluster or host in disabled state?

2024-06-22 18:17:18,063 WARN  [c.c.d.FirstFitPlanner] (Work-Job-Executor-105:ctx-268db4c2 job-1655/job-1760 ctx-41b3df84) (logid:79ff0ac2) Cannot allocate cluster list [1] for VM creation since their allocated percentage crosses the disable capacity threshold defined at each cluster at Global Settings Configuration [name: cluster.memory.allocated.capacity.disablethreshold, value: 0.85] for capacity Type : MEMORY, skipping these clusters
2024-06-22 18:17:18,064 DEBUG [c.c.d.FirstFitPlanner] (Work-Job-Executor-105:ctx-268db4c2 job-1655/job-1760 ctx-41b3df84) (logid:79ff0ac2) No clusters found after removing disabled clusters and clusters in avoid list, returning.

[weizhouapache]

the log shows there is no capacity to start the vm