vr_balancing

Load Balancing

Introduction

There are two different VR load-balancer implementations:

1. Keepalived + LVS (IPVS).
2. HAProxy.

There are three modes of VR operation:

1. Deployed as a proper Virtual Router instance.
2. Deployed as a VM, but inside an OneFlow service instance.
3. Deployed as a standalone VM.

There are four possible VR load-balancing scenarios:

1. Statically in the Virtual Router mode.
2. Statically and dynamically in the Virtual Router mode.
3. Statically in the OneFlow Service mode.
4. Statically and dynamically in the OneFlow Service mode.

Static LBs

Scenarios 1. and 3. (static-only) are exactly the same for all three operation modes.

To create a static LVS-based TCP LB, the following context variables can be provided (example):

CONTEXT = [
  ...
  ONEAPP_VNF_LB_ENABLED = "YES",

  ONEAPP_VNF_LB0_IP        = "<ETH0_EP0>", # Interpolate the first "endpoint".
  ONEAPP_VNF_LB0_PORT      = "80",
  ONEAPP_VNF_LB0_PROTOCOL  = "TCP",
  ONEAPP_VNF_LB0_METHOD    = "NAT",
  ONEAPP_VNF_LB0_SCHEDULER = "rr", # "Round-robin".

  ONEAPP_VNF_LB0_SERVER0_HOST = "172.20.0.104",
  ONEAPP_VNF_LB0_SERVER0_PORT = "8080",
  ONEAPP_VNF_LB0_SERVER1_HOST = "172.20.0.105",
  ONEAPP_VNF_LB0_SERVER1_PORT = "8080",
  ...
]

You can see in the example above, that we defined the LB0 by providing the IP / PORT pair with some extra settings. Then two (static) backends are defined for the LB0.

Similarly to create a HAProxy-based TCP LB, the following context variables can be provided (example):

CONTEXT = [
  ...
  ONEAPP_VNF_HAPROXY_ENABLED = "YES",

  ONEAPP_VNF_HAPROXY_LB0_IP   = "<ETH0_EP0>", # Interpolate the first "endpoint".
  ONEAPP_VNF_HAPROXY_LB0_PORT = "80",

  ONEAPP_VNF_HAPROXY_LB0_SERVER0_HOST = "172.20.0.104",
  ONEAPP_VNF_HAPROXY_LB0_SERVER0_PORT = "8080",
  ONEAPP_VNF_HAPROXY_LB0_SERVER1_HOST = "172.20.0.105",
  ONEAPP_VNF_HAPROXY_LB0_SERVER1_PORT = "8080",
  ...
]

IP / VIP / EP Placeholders

VR understands notations:

• <ETHx_IPy> means "interpolate (in-place) the y-th IP of the x-th NIC"
• <ETHx_VIPy> means "interpolate (in-place) the y-th Virtual IP of the x-th NIC"
• <ETHx_EPy> means "interpolate (in-place) the y-th EndPoint of the x-th NIC"

Please take a look at the table below:

CONTEXT VARIABLES		PLACEHOLDERS
ETH0_IP	ONEAPP_VROUTER_ETH0_VIP0	ETH0_IP0	ETH0_VIP0	ETH0_EP0
10.11.12.13/24	10.11.12.254/24	10.11.12.13/24	10.11.12.254/24	10.11.12.254/24
10.11.12.13/24	undefined	10.11.12.13/24	undefined	10.11.12.13/24
undefined	10.11.12.254/24	undefined	10.11.12.254/24	10.11.12.254/24
undefined	undefined	undefined	undefined	undefined

Important

Values of ETHx_IPy and ETHx_VIPy are always merged together to produce ETHx_EPy. When ETHx_VIPy is undefined, then ETHx_EPy is always set to ETHx_IPy as a fallback, that way you can always have some valid "endpoints" even in non-HA scenarios (no VIPs defined).

Dynamic LBs

Scenarios 2. and 4. (dynamic) are similar for first two operation modes, the third mode (standalone VM) does not support dynamic LBs.

Define a LB with a single backend (static):

CONTEXT = [
  ...
  ONEAPP_VNF_LB_ENABLED         = "YES",
  ONEAPP_VNF_LB_ONEGATE_ENABLED = "YES", # Enable "dynamic" load-balancing.

  ONEAPP_VNF_LB0_IP        = "<ETH0_EP0>", # Interpolate the first "endpoint".
  ONEAPP_VNF_LB0_PORT      = "80",
  ONEAPP_VNF_LB0_PROTOCOL  = "TCP",
  ONEAPP_VNF_LB0_METHOD    = "NAT",
  ONEAPP_VNF_LB0_SCHEDULER = "rr", # "Round-robin".

  ONEAPP_VNF_LB0_SERVER0_HOST = "172.20.0.104",
  ONEAPP_VNF_LB0_SERVER0_PORT = "8080",
  ...
]

This time add the second backend dynamically by executing on the backend VM 172.20.0.105:

$ onegate vm update --data "ONEGATE_LB0_IP=<ETH0_EP0>" # Interpolate the first "endpoint".
$ onegate vm update --data "ONEGATE_LB0_PORT=80"
$ onegate vm update --data "ONEGATE_LB0_SERVER_HOST=172.20.0.105"
$ onegate vm update --data "ONEGATE_LB0_SERVER_PORT=8080"

Note

You can call onegate vm update on the backend VM or just modify VM's user template by other means (API, CLI or Sunstone).

Note

Dynamic variable ONEGATE_LB0_SERVER_HOST does not contain the server index, this is different from the static definition ONEAPP_VNF_LB0_SERVER0_HOST.

Important

The LB's LBx_IP / LBx_PORT pair in the onegate vm update call must match a statically defined LB, that means the LB must be defined in the context. If this is not the case, then no change will be produced.

Important

When running in the mode 1. (Virtual Router instance) you must define the CONTEXT = [..., BACKEND = "YES", ...] variable for each LB backend. If this is not the case, then no change will be produced.

Again, you can similarly define a dynamic HAProxy LB:

CONTEXT = [
  ...
  ONEAPP_VNF_HAPROXY_ENABLED         = "YES",
  ONEAPP_VNF_HAPROXY_ONEGATE_ENABLED = "YES", # Enable "dynamic" load-balancing.

  ONEAPP_VNF_HAPROXY_LB0_IP   = "<ETH0_EP0>", # Interpolate the first "endpoint".
  ONEAPP_VNF_HAPROXY_LB0_PORT = "80",

  ONEAPP_VNF_HAPROXY_LB0_SERVER0_HOST = "172.20.0.104",
  ONEAPP_VNF_HAPROXY_LB0_SERVER0_PORT = "8080",
  ...
]

This time add the second backend dynamically by executing on the backend VM 172.20.0.105:

$ onegate vm update --data "ONEGATE_HAPROXY_LB0_IP=<ETH0_EP0>" # Interpolate the first "endpoint".
$ onegate vm update --data "ONEGATE_HAPROXY_LB0_PORT=80"
$ onegate vm update --data "ONEGATE_HAPROXY_LB0_SERVER_HOST=172.20.0.105"
$ onegate vm update --data "ONEGATE_HAPROXY_LB0_SERVER_PORT=8080"

Virtual Router vs OneFlow

When running both types of LBs in modes 1. and 2. the context / OneGate interface is exactly the same (with the important caveat, that the mode 1. requires backends to have the CONTEXT = [..., BACKEND = "YES", ...] flag defined).

Under the hood the source of updates is different, it both cases OneGate is used, but in the mode 1. all VNETs attached to the VR are scanned (recursively), in the mode 2. OneFlow provides API responses.

In that sense mode 1. is superior to mode 2., because a "proper" VR can reverse-proxy to any VMs in its attached VNETs (also to VMs running in OneFlow instances), at the same time mode 2. allows for backends from within its OneFlow instance only.