VM does not see virtual router after migration to another host

[miloserdoff]

Good day. I have 2 hosts, a virtual router and a VM.

The virtual router and VM are on host1

When the virtual machine and the virtual router are on the same host host1, I can send ping 10.1.1.1 (the IP address of the virtual router) and I will receive a response.

But when I migrate the virtual machine to another host host2, and the virtual router remains on host host1, I cannot send ping 10.1.1.1

What could be the problem?

[miloserdoff]

@weizhouapache @shwstppr can u help me?

[miloserdoff]

@DaanHoogland can u help me?

[pcfriek1987]

are you using any vlans?

[miloserdoff]

@pcfriek1987 On any of the available hosts I can create a VM, it will have access to the Internet.

[DaanHoogland]

@pcfriek1987 On any of the available hosts I can create a VM, it will have access to the Internet.

@miloserdoff , can you create two VMs in the same network on the two different hosts and check if these can both ping each other and the VR?

[miloserdoff]

@DaanHoogland
Ok.
Virtual machine-1 is located on host host1 (172.17.8.6/24) and has ip 10.1.1.2/24.

Virtual machine-2 is located on host maas (172.17.8.5/24) and has ip 10.1.1.3/24.

Both of these virtual machines have network test_network (10.1.1.0/24) and a common virtual router with IP 10.1.1.1/24

VM-1 (host1 172.17.8.6/24, ip 10.1.1.2/24)

VM-2 (maas 172.17.8.5/24, ip 10.1.1.3/24)

[DaanHoogland]

ok, so clearly the bridge does not allow from the vm-2 on host-2 to the vr or vm-1 on host-1. That would mean a configuration error on the network interfaces of you hosts.
If you migrate the VR to host-2 are the results also reversed?

[miloserdoff]

@DaanHoogland Yes, now when the virtual machine is on the same host as the virtual router, it can ping the virtual router and virtual machine-1

[DaanHoogland]

several trouble shooting steps:

Can the hosts ping each other?
Can the VM ping the other host?
Can the MS and the SSVM ping host2?

[miloserdoff]

@DaanHoogland

1. Yes, two hosts can ping each other

2. Yes, a virtual machine that is on a host with IP 172.17.8.6/24 can ping another host with IP 172.17.8.7/24

[miloserdoff]

@DaanHoogland
SSVM (Secondary Storage VM) can ping host2 (172.17.8.7/24)

MS (Management server) can ping host2 (172.17.8.7/24)

[DaanHoogland]

@miloserdoff , i see you found what I meant with SSVM. By MS i meant the Management Server.

I am assuming your network has all the right egress and ingress rules set. Please see my response in the other thread (#11236 (reply in thread))

[DaanHoogland]

@miloserdoff , it seems the interfaces cannot route the vlan to each other. That is either a config of the interfaces or an upstream router/switch.

[pcfriek1987]

what do your bridges look like?

[miloserdoff]

@pcfriek1987
host maas:
4: cloudbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
link/ether 56:aa:00:e8:3d:f6 brd ff:ff:ff:ff:ff:ff
inet 172.17.8.5/24 brd 172.17.8.255 scope global cloudbr0
valid_lft forever preferred_lft forever
inet6 fe80::54aa:ff:fee8:3df6/64 scope link
valid_lft forever preferred_lft forever

host host1:
4: cloudbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
link/ether 7a:8e:8a:db:18:19 brd ff:ff:ff:ff:ff:ff
inet 172.17.8.6/24 brd 172.17.8.255 scope global cloudbr0
valid_lft forever preferred_lft forever
inet6 fe80::788e:8aff:fedb:1819/64 scope link
valid_lft forever preferred_lft forever

host host2:
4: cloudbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
link/ether 8e:f2:b1:89:bb:b8 brd ff:ff:ff:ff:ff:ff
inet 172.17.8.7/24 brd 172.17.8.255 scope global cloudbr0
valid_lft forever preferred_lft forever
inet6 fe80::8cf2:b1ff:fe89:bbb8/64 scope link
valid_lft forever preferred_lft forever

[miloserdoff]

@pcfriek1987 @weizhouapache

UPDATE:
Here is the output of the brctl show command from each host:

host maas:

bridge name     bridge id               STP enabled     interfaces
brenp1s0f0-11           8000.0ee3479059e8       no              enp1s0f0.11
                                                        vnet0
                                                        vnet5
brenp1s0f0-719          8000.3216d7c9a0ed       no              enp1s0f0.719
brenp1s0f0-740          8000.0e25784bc03a       no              enp1s0f0.740
                                                        vnet36
cloud0          8000.668bee3adc2a       no              vnet1
                                                        vnet10
                                                        vnet2
cloudbr0                8000.56aa00e83df6       no              enp1s0f0
                                                        vnet11
                                                        vnet12
                                                        vnet3
                                                        vnet4
lxdbr0          8000.00163e433215       no              veth3c3ae630

host host1:

bridge name     bridge id               STP enabled     interfaces
br-6254953832f3         8000.828908d784de       no              veth3c95051
brenp1s0f0-1104         8000.324c20e552a6       no              enp1s0f0.1104
brenp1s0f0-740          8000.1e98f64bb2a4       no              enp1s0f0.740
                                                        vnet63
                                                        vnet74
brenp1s0f0-851          8000.1a7a75a82012       no              enp1s0f0.851
cloud0          8000.c6ccf0674a22       no              vnet75
cloudbr0                8000.7a8e8adb1819       no              enp1s0f0
                                                        vnet76
docker0         8000.5a5cdf6f27f7       no

host host2:

bridge name     bridge id               STP enabled     interfaces
brenp1s0f0-1104         8000.3eb4399ab879       no              enp1s0f0.1104
                                                        vnet64
                                                        vnet66
                                                        vnet67
brenp1s0f0-819          8000.461baa4842ab       no              enp1s0f0.819
brenp1s0f0-895          8000.06d3c019846a       no              enp1s0f0.895
cloud0          8000.9a434294d4f2       no              vnet65
cloudbr0                8000.8ef2b189bbb8       no              enp1s0f0

[DaanHoogland]

@miloserdoff , what is the vlan used for the network we are talking about?

[miloserdoff]

@DaanHoogland VLAN 740. Host maas and host host1

[miloserdoff]

@DaanHoogland @pcfriek1987 @weizhouapache

UPDATE: I tried to do the same thing, but in another cluster:

My architecture looks like this:

And now when I migrate to another host:

[weizhouapache]

@weizhouapache Hi, my problem is still not solved. Are there any other ways?

@miloserdoff
it is a very typical issue of virtualization platforms, which is caused by network configurations of infrastructure (not ACS issue).

you can do a quick testing

• find a bridge which exist in at least two hosts
• assign IPs in same subnet to the bridge (for example brenp1s0f0-740)
• ping the IPs from each other, it should work

[miloserdoff]

@weizhouapache Over time, my ACS has changed a bit. The problem remains. My two hosts have the same bridges (breno1np0-852):

host1:

host2:

I assigned IP addresses to these bridges on these hosts (10.1.1.3/24 and 10.1.1.4/24)
host1:

host2:

But when I ping 10.1.1.1 and ping 10.1.1.4, I see "Destination Host Unreachable" in the response.

[weizhouapache]

@miloserdoff
ok, it approves it is not a ACS issue.

if you have already set switch ports to trunk mode, it might be caused by other switch or host settings.

[miloserdoff]

@weizhouapache What settings on the host should I check?

[weizhouapache]

@miloserdoff
it requires investigation. there are several possible root causes.
you can capture the packets to see where it is stuck