Consolidate Node Peer Annotations

docs/bgp.md

@@ -10,7 +10,7 @@ pod IPs, service IPs, etc.).
 
 This is the default mode. All nodes in the clusters form iBGP peering relationships with rest of the nodes forming a
 full node-to-node mesh. Each node advertise the pod CIDR allocated to the nodes with its peers (the rest of the nodes in
-the cluster).  There is no configuration required in this mode. All the nodes in the cluster are associated with the
+the cluster). There is no configuration required in this mode. All the nodes in the cluster are associated with the
 private ASN 64512 implicitly (which can be configured with `--cluster-asn` flag) and users are transparent to use of
 iBGP. This mode is suitable in public cloud environments or small cluster deployments.
 
@@ -30,7 +30,7 @@ kubectl annotate node <kube-node> "kube-router.io/node.asn=64512"
 
 Only nodes within same ASN form full mesh. Two nodes with different ASNs never get peered.
 
-### Route-Reflector setup  Without Full Mesh
+### Route-Reflector setup Without Full Mesh
 
 This model supports the common scheme of using a Route Reflector Server node to concentrate peering from client peers.
 This has the big advantage of not needing full mesh, and will scale better. In this mode kube-router expects each node
@@ -75,11 +75,45 @@ For example:
 
 ### Node Specific External BGP Peers
 
-Alternatively, each node can be configured with one or more node specific BGP peers. Information regarding node specific
-BGP peer is read from node API object annotations:
+Each node can be configured with one or more node specific BGP peers using the `kube-router.io/peers` node annotation.
+Previously, these settings were configured using individual `kube-router.io/peer.*` annotations.
+While these individual annotations are still supported, they're now deprecated and
+will be removed in a future release.
+
+#### Using Consolidated Annotation
+
+The `kube-router.io/peers` annotation accepts peer configurations in YAML format with the following fields:
+
+- `remoteip` (required): The IP address of the peer
+- `remoteasn` (required): The ASN of the peer
+- `localip` (optional): Local IP address to use for this peer connection
+- `password` (optional): Base64 encoded password for BGP authentication
+- `port` (optional): BGP port (defaults to 179 if not specified)
+
+```shell
+kubectl annotate node <kube-node> \
+kube-router.io/peers="$(cat <<'EOF'
+- remoteip: 192.168.1.99
+  remoteasn: 65000
+  password: U2VjdXJlUGFzc3dvcmQK,
+- remoteip: 192.168.1.100
+  remoteasn: 65000'
+  password: U2VjdXJlUGFzc3dvcmQK,
+EOF
+)"
+```
+
+#### Using Individual Annotations (Deprecated)
+
+> **NOTE:** The individual peer annotations listed below are deprecated in favor of the consolidated `kube-router.io/peers`
+> annotation. They are maintained for backward compatibility but will be removed in a future release.
+
+Node-specific BGP peer configs can also be set via individual node API object annotations:
 
 - `kube-router.io/peer.ips`
 - `kube-router.io/peer.asns`
+- `kube-router.io/peer.passwords`
+- `kube-router.io/peer.localips`
 
 For example, users can annotate node object with below commands:
 
@@ -106,26 +140,23 @@ kubectl annotate node <kube-node> "kube-router.io/path-prepend.repeat-n=5"
 
 ### BGP Peer Local IP configuration
 
-In some setups it might be desirable to set a local IP address used for connecting external BGP peers. This can be
-accomplished on nodes with annotations:
+In some setups it might be desirable to set a local IP address used for connecting external BGP peers.
 
-- `kube-router.io/peer.localips`
-
-If set, this must be a list with a local IP address for each peer, or left empty to use nodeIP.
+When using the `kube-router.io/peers` annotation, specify the `localip` field for each peer as shown in the
+[Node Specific External BGP Peers](#node-specific-external-bgp-peers) section above.
 
-Example:
+When using individual annotations, you can specify the local IP address using `kube-router.io/peer.localips`:
 
 ```shell
 kubectl annotate node <kube-node> "kube-router.io/peer.localips=10.1.1.1,10.1.1.2"
 ```
 
-This will instruct kube-router to use IP `10.1.1.1` for first BGP peer as a local address, and use `10.1.1.2`for the
-second.
+If set, this must be a list with a local IP address for each peer, or left empty to use nodeIP.
 
 ### BGP Peer Password Authentication
 
-The examples above have assumed there is no password authentication with BGP peer routers. If you need to use a password
-for peering, you can use the `--peer-router-passwords` command-line option, the `kube-router.io/peer.passwords` node
+If you need to use a password for peering with BGP peer routers, you can configure it using the `kube-router.io/peers`
+annotation, the `--peer-router-passwords` command-line option, the deprecated `kube-router.io/peer.passwords` node
 annotation, or the `--peer-router-passwords-file` command-line option.
 
 #### Base64 Encoding Passwords
@@ -142,7 +173,15 @@ U2VjdXJlUGFzc3dvcmQ=
 
 #### Password Configuration Examples
 
-In this CLI flag example the first router (192.168.1.99) uses a password, while the second (192.168.1.100) does not.
+**Using the consolidated annotation (recommended):**
+
+When using the `kube-router.io/peers` annotation, specify the `password` field with a base64 encoded password for each
+peer that requires authentication. See the
+[Node Specific External BGP Peers](#node-specific-external-bgp-peers) section for an example.
+
+**Using CLI flags:**
+
+In this example the first router (192.168.1.99) uses a password, while the second (192.168.1.100) does not:
 
 ```sh
 --peer-router-ips="192.168.1.99,192.168.1.100"
@@ -152,14 +191,18 @@ In this CLI flag example the first router (192.168.1.99) uses a password, while
 
 Note the comma indicating the end of the first password.
 
-Here's the same example but configured as node annotations:
+**Using individual annotations (deprecated):**
+
+Here's the same example but configured with individual node annotations:
 
 ```shell
 kubectl annotate node <kube-node> "kube-router.io/peer.ips=192.168.1.99,192.168.1.100"
 kubectl annotate node <kube-node> "kube-router.io/peer.asns=65000,65000"
 kubectl annotate node <kube-node> "kube-router.io/peer.passwords=U2VjdXJlUGFzc3dvcmQK,"
 ```
 
+**Using a password file:**
+
 Finally, to include peer passwords as a file you would run kube-router with the following option:
 
 ```shell
@@ -168,8 +211,8 @@ Finally, to include peer passwords as a file you would run kube-router with the
 --peer-router-passwords-file="/etc/kube-router/bgp-passwords.conf"
 ```
 
-The password file, closely follows the syntax of the command-line and node annotation options.
-Here, the first peer IP (192.168.1.99) would be configured with a password, while the second would not.
+The password file closely follows the syntax of the command-line and node annotation options.
+Here, the first peer IP (192.168.1.99) would be configured with a password, while the second would not:
 
 ```sh
 U2VjdXJlUGFzc3dvcmQK,

go.mod

@@ -9,6 +9,8 @@ require (
 	github.com/ccoveille/go-safecast v1.6.1
 	github.com/coreos/go-iptables v0.8.0
 	github.com/docker/docker v28.4.0+incompatible
+	github.com/goccy/go-yaml v1.18.0
+	github.com/google/go-cmp v0.7.0
 	github.com/hashicorp/go-version v1.7.0
 	github.com/moby/ipvs v1.1.0
 	github.com/onsi/ginkgo v1.16.5
@@ -65,7 +67,6 @@ require (
 	github.com/go-openapi/swag v0.23.0 // indirect
 	github.com/gogo/protobuf v1.3.2 // indirect
 	github.com/google/gnostic-models v0.7.0 // indirect
-	github.com/google/go-cmp v0.7.0 // indirect
 	github.com/google/uuid v1.6.0 // indirect
 	github.com/hashicorp/hcl v1.0.0 // indirect
 	github.com/josharian/intern v1.0.0 // indirect

go.sum

@@ -95,6 +95,8 @@ github.com/go-task/slim-sprig/v3 v3.0.0 h1:sUs3vkvUymDpBKi3qH1YSqBQk9+9D/8M2mN1v
 github.com/go-task/slim-sprig/v3 v3.0.0/go.mod h1:W848ghGpv3Qj3dhTPRyJypKRiqCdHZiAzKg9hl15HA8=
 github.com/go-test/deep v1.1.0 h1:WOcxcdHcvdgThNXjw0t76K42FXTU7HpNQWHpA2HHNlg=
 github.com/go-test/deep v1.1.0/go.mod h1:5C2ZWiW0ErCdrYzpqxLbTX7MG14M9iiw8DgHncVwcsE=
+github.com/goccy/go-yaml v1.18.0 h1:8W7wMFS12Pcas7KU+VVkaiCng+kG8QiFeFwzFb+rwuw=
+github.com/goccy/go-yaml v1.18.0/go.mod h1:XBurs7gK8ATbW4ZPGKgcbrY1Br56PdM69F7LkFRi1kA=
 github.com/gogo/protobuf v1.3.2 h1:Ov1cvc58UF3b5XjBnZv7+opcTcQFZebYjWzi34vdm4Q=
 github.com/gogo/protobuf v1.3.2/go.mod h1:P1XiOD3dCwIKUDQYPy72D8LYyHL2YPYrpS2s69NZV8Q=
 github.com/golang/protobuf v1.2.0/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=

pkg/bgp/peer_config.go

@@ -0,0 +1,205 @@
+package bgp
+
+import (
+	"errors"
+	"fmt"
+	"net"
+	"strconv"
+
+	"github.com/cloudnativelabs/kube-router/v2/pkg/options"
+	"github.com/cloudnativelabs/kube-router/v2/pkg/utils"
+	"github.com/goccy/go-yaml"
+)
+
+type PeerConfig struct {
+	LocalIP   *string             `yaml:"localip"`
+	Password  *utils.Base64String `yaml:"password"`
+	Port      *uint32             `yaml:"port"`
+	RemoteASN *uint32             `yaml:"remoteasn"`
+	RemoteIP  *net.IP             `yaml:"remoteip"`
+}
+
+func (p *PeerConfig) UnmarshalYAML(raw []byte) error {
+	tmp := struct {
+		LocalIP   *string             `yaml:"localip"`
+		Password  *utils.Base64String `yaml:"password"`
+		Port      *uint32             `yaml:"port"`
+		RemoteASN *uint32             `yaml:"remoteasn"`
+		RemoteIP  string              `yaml:"remoteip"`
+	}{}
+
+	if err := yaml.Unmarshal(raw, &tmp); err != nil {
+		return fmt.Errorf("failed to unmarshal peer config: %w", err)
+	}
+
+	p.LocalIP = tmp.LocalIP
+	p.Password = tmp.Password
+	p.Port = tmp.Port
+	p.RemoteASN = tmp.RemoteASN
+
+	if tmp.RemoteIP != "" {
+		ip := net.ParseIP(tmp.RemoteIP)
+		if ip == nil {
+			return fmt.Errorf("%s is not a valid IP address", tmp.RemoteIP)
+		}
+		p.RemoteIP = &ip
+	}
+	return nil
+}
+
+type PeerConfigs []PeerConfig
+
+func (p PeerConfigs) LocalIPs() []string {
+	localIPs := make([]string, 0)
+	for _, cfg := range p {
+		if cfg.LocalIP != nil {
+			localIPs = append(localIPs, *cfg.LocalIP)
+		}
+	}
+	return localIPs
+}
+
+// Returns b64 decoded passwords
+func (p PeerConfigs) Passwords() []string {
+	passwords := make([]string, 0)
+	for _, cfg := range p {
+		if cfg.Password != nil {
+			passwords = append(passwords, string(*cfg.Password))
+		}
+	}
+	return passwords
+}
+
+func (p PeerConfigs) Ports() []uint32 {
+	ports := make([]uint32, 0)
+	for _, cfg := range p {
+		if cfg.Port != nil {
+			ports = append(ports, *cfg.Port)
+		}
+	}
+	return ports
+}
+
+func (p PeerConfigs) RemoteASNs() []uint32 {
+	asns := make([]uint32, 0)
+	for _, cfg := range p {
+		if cfg.RemoteASN != nil {
+			asns = append(asns, *cfg.RemoteASN)
+		}
+	}
+	return asns
+}
+
+func (p PeerConfigs) RemoteIPs() []net.IP {
+	remoteIPs := make([]net.IP, 0)
+	for _, cfg := range p {
+		if cfg.RemoteIP != nil {
+			remoteIPs = append(remoteIPs, *cfg.RemoteIP)
+		}
+	}
+	return remoteIPs
+}
+
+func (p PeerConfigs) RemoteIPStrings() []string {
+	remoteIPs := make([]string, 0)
+	for _, cfg := range p {
+		if cfg.RemoteIP != nil {
+			remoteIPs = append(remoteIPs, cfg.RemoteIP.String())
+		}
+	}
+	return remoteIPs
+}
+
+func (p *PeerConfigs) UnmarshalYAML(raw []byte) error {
+	type tmpPeerConfigs PeerConfigs
+	tmp := (*tmpPeerConfigs)(p)
+
+	if err := yaml.Unmarshal(raw, tmp); err != nil {
+		return err
+	}
+
+	return p.Validate()
+}
+
+func (p PeerConfigs) Validate() error {
+	return validatePeerConfigs(p.RemoteIPStrings(), p.RemoteASNs(), p.Ports(), p.Passwords(), p.LocalIPs(), "")
+}
+
+func NewPeerConfigs(
+	remoteIPs []string,
+	remoteASNs []uint32,
+	ports []uint32,
+	b64EncodedPasswords []string,
+	localIPs []string,
+	localAddress string,
+) (PeerConfigs, error) {
+	if err := validatePeerConfigs(remoteIPs, remoteASNs, ports, b64EncodedPasswords, localIPs, localAddress); err != nil {
+		return nil, err
+	}
+
+	peerCfgs := make(PeerConfigs, len(remoteIPs))
+	for i, remoteIP := range remoteIPs {
+		ip := net.ParseIP(remoteIP)
+		if ip == nil {
+			return nil, fmt.Errorf("invalid IP address: %s", remoteIP)
+		}
+		peerCfgs[i].RemoteIP = &ip
+		peerCfgs[i].RemoteASN = &remoteASNs[i]
+
+		if len(ports) != 0 {
+			peerCfgs[i].Port = &ports[i]
+		}
+
+		if len(b64EncodedPasswords) != 0 {
+			pw := utils.Base64String(b64EncodedPasswords[i])
+			peerCfgs[i].Password = &pw
+		}
+
+		if len(localIPs) != 0 && localIPs[i] != "" {
+			peerCfgs[i].LocalIP = &localIPs[i]
+		}
+	}
+
+	return peerCfgs, nil
+}
+
+func validatePeerConfigs(
+	remoteIPs []string,
+	remoteASNs []uint32,
+	ports []uint32,
+	b64EncodedPasswords []string,
+	localIPs []string,
+	localAddress string,
+) error {
+	if len(remoteIPs) != len(remoteASNs) {
+		return errors.New("invalid peer router config, the number of IPs and ASN numbers must be equal")
+	}
+	if len(remoteIPs) != len(b64EncodedPasswords) && len(b64EncodedPasswords) != 0 {
+		return errors.New("invalid peer router config. The number of passwords should either be zero, or " +
+			"one per peer router. Use blank items if a router doesn't expect a password. Example: \"pass,,pass\" " +
+			"OR [\"pass\",\"\",\"pass\"]")
+	}
+	if len(remoteIPs) != len(ports) && len(ports) != 0 {
+		return fmt.Errorf("invalid peer router config. The number of ports should either be zero, or "+
+			"one per peer router. If blank items are used, it will default to standard BGP port, %s. "+
+			"Example: \"port,,port\" OR [\"port\",\"\",\"port\"]", strconv.Itoa(options.DefaultBgpPort))
+	}
+	if len(remoteIPs) != len(localIPs) && len(localIPs) != 0 {
+		return fmt.Errorf("invalid peer router config. The number of localIPs should either be zero, or "+
+			"one per peer router. If blank items are used, it will default to nodeIP, %s. "+
+			"Example: \"10.1.1.1,,10.1.1.2\" OR [\"10.1.1.1\",\"\",\"10.1.1.2\"]", localAddress)
+	}
+
+	for _, asn := range remoteASNs {
+		if (asn < 1 || asn > 23455) &&
+			(asn < 23457 || asn > 63999) &&
+			(asn < 64512 || asn > 65534) &&
+			(asn < 131072 || asn > 4199999999) &&
+			(asn < 4200000000 || asn > 4294967294) {
+			return fmt.Errorf("reserved ASN number \"%d\" for global BGP peer",
+				asn)
+		}
+	}
+
+	return nil
+}