Nginx or Apache as a reverse proxy in front of CloudStack (including VNC console support)

[dR3b]

Hello,

I’m looking for a working configuration for Nginx or Apache as a reverse proxy in front of CloudStack (including VNC console support).

I’m currently setting up our first CloudStack cluster and have 3 servers available for this. At the moment, I have a single-node running, and so far, everything is working as expected. VMs can be created, and storage is connected via NFS without issues.

The CloudStack Management server is still running on port 8080, and the VNC console is working as well.

Now I would like to place Nginx or Apache in front of it. Forwarding port 443 to 8080 is no problem, but I can’t get the VNC console to start when accessed through the reverse proxy. The error message I receive is always: “Failed to connect to server / access token has expired” or “404 not found".

Could please someone share their Nginx configuration with me?

Thanks.

[patrick-jpg-668]

Not sure about Nginx or Apache but i'm using HAProxy on pfSense and work pretty well for the console.

[daviftorres]

Not sure about Nginx or Apache but i'm using HAProxy on pfSense and work pretty well for the console.

Hey @patrick-jpg-668 , you are correct. It does works. But there are cases where it needs polishing. See this issue:

#11020

[dR3b]

@patrick-jpg-668 Would you be able to show us your configuration, please?

[TadiosAbebe]

@patrick-jpg-668 I would also be glad to see your haproxy/pfsense config for the console proxy vms. I remember trying once and giving up.

[weizhouapache]

@dR3b
have you configured reverse proxy for websocket ?

Do you use ssl offloading ?

[dR3b]

Yes i think so:

server {
        listen 80;
        listen [::]:80;

        server_name x.y.z;

        access_log      /var/log/nginx/cloudstack-http.access.log;
        error_log       /var/log/nginx/cloudstack-http.error.log;

        return 301 https://$host$request_uri;
}

server {
        listen  443 ssl;
        listen  [::]:443 ssl;

        server_name x.y.z;

        access_log      /var/log/nginx/cloudstack-https.access.log;
        error_log       /var/log/nginx/cloudstack-https.error.log;

        ssl_certificate         /etc/nginx/ssl/nginx.crt;
        ssl_certificate_key     /etc/nginx/ssl/nginx.key;

        location / {
                proxy_pass            http://localhost:8080;
                proxy_read_timeout    90s;
                proxy_connect_timeout 90s;
                proxy_send_timeout    90s;
                proxy_set_header      Host $host;
                proxy_set_header      X-Real-IP $remote_addr;
                proxy_set_header      X-Forwarded-For $proxy_add_x_forwarded_for;
                proxy_set_header      Proxy "";
        }
        location /websockify {
                proxy_pass              http://localhost:8080/websockify;
                proxy_http_version      1.1;
                proxy_set_header        Upgrade $http_upgrade;
                proxy_set_header        Connection "Upgrade";
                proxy_set_header        Host $host;
                proxy_read_timeout      600s;
                proxy_buffering         off;
        }
}

[weizhouapache]

@dR3b
is the nginx configured in the console proxy VM ?
if not, you should use http://<public IP of CPVM>:8080/ instead of localhost

can you upload ssl certificate in cloudstack ? please refer to https://www.shapeblue.com/securing-cloudstack-4-11-with-https-tls/

[bhouse-nexthop]

The biggest issue with a proxy would be the fact that the consolevm and ssvm can spawn multiple instances dynamically so you'd have to do something like a regex match on the server name to extract the ip address of the real backend console or ssvm node.

I haven't found a premade example for cloudstack, but openstack has a similar problem and this is their solution:
https://platform9.com/docs/openstack-docs/openstack/accessing-vms-access-vm-kvm

Of course we'd need to parse server_name instead of location, which could be done something like this:
https://stackoverflow.com/a/37578602

[dR3b]

I’ve created a diagram here to show what I actually want:

Port 8080, works fine:

With Nginx:

[daviftorres]

Here’s my NGINX setup (very similar to what @dR3b and @bhouse-nexthop shared).

I’m assuming SSL (wildcard cert) is terminated on NGINX for all services:

ssl_certificate /etc/nginx/fullchain.crt;
ssl_certificate_key /etc/nginx/privatekey.key;
ssl_protocols TLSv1.2 TLSv1.3;
ssl_prefer_server_ciphers on;
ssl_ciphers <your_preferred_ciphers>;
server {
    listen 80 default_server;
    listen [::]:80 default_server;
    server_name _;
    return 301 https://$host$request_uri;
}

For small environments (around 10~30 concurrent SysVMs), you can create a host-to-IP mapping that includes the environment name (dev, test, stage, poc, hotfix, etc.)

map $host $backend_ip {
    10-10-1-200.<environment>.example.com 10.10.1.200;
    10-10-1-201.<environment>.example.com 10.10.1.201;
    10-10-1-202.<environment>.example.com 10.10.1.202;
    # add more as needed
}

This setup covers Console Proxy VMs and Secondary Storage VMs:

server {
    listen 443 ssl;
    server_name ~^(10-10-1-\d+\.<environment>\.example\.com)$;
    client_max_body_size 0;
    proxy_http_version 1.1;
    proxy_request_buffering off;
    location / {
        proxy_pass https://$backend_ip:443;
        proxy_set_header Host $http_host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
    }
}

For WebSocket Secure (WSS) traffic:

server {
    listen 8443 ssl;
    server_name _;
    client_max_body_size 0;
    proxy_http_version 1.1;
    proxy_request_buffering off;
    location /websockify {
        proxy_pass https://$backend_ip:8443;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "Upgrade";
        proxy_set_header Host $host;
        proxy_read_timeout 600s;
        proxy_buffering off;
    }
}

Both use the same host-to-IP mapping, without a tailored regex.

For larger environments, consider HAProxy, Varnish, or Envoy because they scale better and handle complex routing more efficiently.

Warning: Using a loose regex for server_name can expose other network assets (via fuzzing/guessing IPs). Keep the regex strict to SysVMs only.