Skip to content

Commit e19f2a6

Browse files
bzubbzub
authored
BGP peer password auth, consistent configurations (#164)
* Add --peer-router-password option Also: - Consolodated NRC peer fields into a []config.NeighborConfig to store address, asn, and password for each peer. - BREAKING: --peer-router and --peer-asn flags now take slices rather than strings. * Add password auth node annotation for external peer * Update documentation New CLI flags and annotations Renamed ones as well * Consistent CLI flags, annotations, and peer config BGP configs now all accept multiple values and are treated consistently. Other refactoring was done as well. * Stop bgpserver on peering errors to avoid listener leak * Clarify BGP doc sections Fix some typos
1 parent d6ea740 commit e19f2a6

File tree

6 files changed

+321
-164
lines changed

6 files changed

+321
-164
lines changed

‎Documentation/README.md‎

Lines changed: 26 additions & 24 deletions
Original file line numberDiff line numberDiff line change
@@ -97,28 +97,30 @@ Also you can choose to run kube-router as agent running on each cluster node. Al
9797

9898
```
9999
Usage of ./kube-router:
100-
--advertise-cluster-ip Add Cluster IP to the RIB and advertise to peers.
101-
--cleanup-config Cleanup iptables rules, ipvs, ipset configuration and exit.
102-
--cluster-asn string ASN number under which cluster nodes will run iBGP.
103-
--config-sync-period duration The delay between apiserver configuration synchronizations (e.g. '5s', '1m'). Must be greater than 0. (default 1m0s)
104-
--enable-pod-egress SNAT traffic from Pods to destinations outside the cluster. (default true)
105-
--enable-overlay When enable-overlay set to true, IP-in-IP tunneling is used for pod-to-pod networking across nodes in different subnets. When set to false no tunneling is used and routing infrastrcture is expected to route traffic for pod-to-pod networking across nodes in different subnets (default true)
106-
--hairpin-mode Add iptable rules for every Service Endpoint to support hairpin traffic.
107-
-h, --help Print usage information.
108-
--hostname-override string Overrides the NodeName of the node. Set this if kube-router is unable to determine your NodeName automatically.
109-
--iptables-sync-period duration The delay between iptables rule synchronizations (e.g. '5s', '1m'). Must be greater than 0. (default 1m0s)
110-
--ipvs-sync-period duration The delay between ipvs config synchronizations (e.g. '5s', '1m', '2h22m'). Must be greater than 0. (default 1m0s)
111-
--kubeconfig string Path to kubeconfig file with authorization information (the master location is set by the master flag).
112-
--masquerade-all SNAT all traffic to cluster IP/node port.
113-
--master string The address of the Kubernetes API server (overrides any value in kubeconfig).
114-
--nodes-full-mesh Each node in the cluster will setup BGP peering with rest of the nodes. (default true)
115-
--peer-asn string ASN number of the BGP peer to which cluster nodes will advertise cluster ip and node's pod cidr.
116-
--peer-router string Comma sepereated list of ip address of the external routers to which all nodes will peer and advertise the cluster ip and pod cidr's.
117-
--routes-sync-period duration The delay between route updates and advertisements (e.g. '5s', '1m', '2h22m'). Must be greater than 0. (default 1m0s)
118-
--run-firewall Enables Network Policy -- sets up iptables to provide ingress firewall for pods. (default true)
119-
--run-router Enables Pod Networking -- Advertises and learns the routes to Pods via iBGP. (default true)
120-
--run-service-proxy Enables Service Proxy -- sets up IPVS for Kubernetes Services. (default true)
121-
--nodeport-bindon-all-ip For service of NodePort type create IPVS service that listens on all IP's of the node. (default false)
100+
--advertise-cluster-ip Add Cluster IP to the RIB and advertise to peers.
101+
--cleanup-config Cleanup iptables rules, ipvs, ipset configuration and exit.
102+
--cluster-asn uint ASN number under which cluster nodes will run iBGP.
103+
--cluster-cidr string CIDR range of pods in the cluster. It is used to identify traffic originating from and destinated to pods.
104+
--config-sync-period duration The delay between apiserver configuration synchronizations (e.g. '5s', '1m'). Must be greater than 0. (default 1m0s)
105+
--enable-overlay When enable-overlay set to true, IP-in-IP tunneling is used for pod-to-pod networking across nodes in different subnets. When set to false no tunneling is used and routing infrastrcture is expected to route traffic for pod-to-pod networking across nodes in different subnets (default true)
106+
--enable-pod-egress SNAT traffic from Pods to destinations outside the cluster. (default true)
107+
--hairpin-mode Add iptable rules for every Service Endpoint to support hairpin traffic.
108+
-h, --help Print usage information.
109+
--hostname-override string Overrides the NodeName of the node. Set this if kube-router is unable to determine your NodeName automatically.
110+
--iptables-sync-period duration The delay between iptables rule synchronizations (e.g. '5s', '1m'). Must be greater than 0. (default 1m0s)
111+
--ipvs-sync-period duration The delay between ipvs config synchronizations (e.g. '5s', '1m', '2h22m'). Must be greater than 0. (default 1m0s)
112+
--kubeconfig string Path to kubeconfig file with authorization information (the master location is set by the master flag).
113+
--masquerade-all SNAT all traffic to cluster IP/node port.
114+
--master string The address of the Kubernetes API server (overrides any value in kubeconfig).
115+
--nodeport-bindon-all-ip For service of NodePort type create IPVS service that listens on all IP's of the node.
116+
--nodes-full-mesh Each node in the cluster will setup BGP peering with rest of the nodes. (default true)
117+
--peer-router-asns uintSlice ASN numbers of the BGP peer to which cluster nodes will advertise cluster ip and node's pod cidr. (default [])
118+
--peer-router-ips ipSlice The ip address of the external router to which all nodes will peer and advertise the cluster ip and pod cidr's. (default [])
119+
--peer-router-passwords stringSlice Password for authenticating against the BGP peer defined with "--peer-router-ips".
120+
--routes-sync-period duration The delay between route updates and advertisements (e.g. '5s', '1m', '2h22m'). Must be greater than 0. (default 1m0s)
121+
--run-firewall Enables Network Policy -- sets up iptables to provide ingress firewall for pods. (default true)
122+
--run-router Enables Pod Networking -- Advertises and learns the routes to Pods via iBGP. (default true)
123+
--run-service-proxy Enables Service Proxy -- sets up IPVS for Kubernetes Services. (default true)```
122124
```
123125

124126
### requirements
@@ -178,7 +180,7 @@ and run kube-proxy with the configuration you have.
178180

179181
Communication from a Pod that is behind a Service to its own ClusterIP:Port is
180182
not supported by default. However, It can be enabled per-service by adding the
181-
`kube-router.io/hairpin-mode=` annotation, or for all Services in a cluster by
183+
`io.kube-router.net.service.hairpin=` annotation, or for all Services in a cluster by
182184
passing the flag `--hairpin-mode=true` to kube-router.
183185

184186
Additionally, the `hairpin_mode` sysctl option must be set to `1` for all veth
@@ -207,7 +209,7 @@ Service ClusterIP if it is logging the source IP.
207209

208210
To enable hairpin traffic for Service `my-service`:
209211
```
210-
kubectl annotate service my-service 'kube-router.io/hairpin-mode='
212+
kubectl annotate service my-service "io.kube-router.net.service.hairpin="
211213
```
212214

213215

‎Documentation/bgp.md‎

Lines changed: 83 additions & 28 deletions
Original file line numberDiff line numberDiff line change
@@ -1,48 +1,103 @@
1-
## Configuring BGP Peers
1+
# Configuring BGP Peers
22

33
When kube-router is used to provide pod-to-pod networking, BGP is used to exchange routes across the nodes. Kube-router
4-
provides flexible networking models to support different deployment (public vs private cloud, routable vs non-routable
5-
pod IP's, service ip's etc)
4+
provides flexible networking models to support different deployments (public vs private cloud, routable vs non-routable
5+
pod IP's, service ip's etc).
66

7-
### Full node-to-node mesh
7+
## Peering Within The Cluster
8+
### Full Node-To-Node Mesh
89

9-
This is the default mode. All nodes in the clusters form iBGP peering relationship with rest of the nodes forming full
10-
node-to-node mesh. Each node advertise the pod CIDR allocated to the nodes with peers (rest of the nodes in the cluster).
11-
There is no configuration required in this mode. All the nodes in the cluster are associated with private ASN 64512
12-
implicitly (which can be configured with `--cluster-asn` flag). Users are transparent to use of iBGP. This mode is
13-
suitable in public cloud environments or small cluster deployments. In this mode all the nodes are expected to be L2 adjacent.
10+
This is the default mode. All nodes in the clusters form iBGP peering
11+
relationship with rest of the nodes forming full node-to-node mesh. Each node
12+
advertise the pod CIDR allocated to the nodes with peers (rest of the nodes in
13+
the cluster). There is no configuration required in this mode. All the nodes in
14+
the cluster are associated with private ASN 64512 implicitly (which can be
15+
configured with `--cluster-asn` flag). Users are transparent to use of iBGP.
16+
This mode is suitable in public cloud environments or small cluster deployments.
17+
In this mode all the nodes are expected to be L2 adjacent.
1418

15-
### Node specific BGP peers
19+
### Node-To-Node Peering Without Full Mesh
1620

17-
This model support more than a single AS per cluster to allow AS per rack or AS per node models. Nodes in the cluster
18-
does not form full node-to-node mesh. Users has to explicitly select this mode by specifying `--nodes-full-mesh=false`
19-
when launching kube-router. In this mode kube-router expects each node is configured with ASN number to be used for the
20-
node from the nodes API object annoations. Kube-router will use the configured value for the key `net.kuberouter.nodeasn`
21-
in the node object as the ASN number for the node.
21+
This model support more than a single AS per cluster to allow AS per rack or AS
22+
per node models. Nodes in the cluster does not form full node-to-node mesh.
23+
Users has to explicitly select this mode by specifying `--nodes-full-mesh=false`
24+
when launching kube-router. In this mode kube-router expects each node is
25+
configured with an ASN number from the node's API object annoations. Kube-router
26+
will use the node's `io.kube-router.net.node.asn` annotation value as the ASN
27+
number for the node.
2228

23-
Users can annotate node object with below command
29+
Users can annotate node objects with the following command:
2430

2531
```
26-
kubectl annotate node <kube-node> "net.kuberouter.nodeasn=64512”"
32+
kubectl annotate node <kube-node> "io.kube-router.net.node.asn=64512"
2733
```
2834

29-
Only nodes with in same ASN form full mesh. Two nodes with different configured ASN never gets peered.
35+
Only nodes with in same ASN form full mesh. Two nodes with different ASNs never
36+
get peered.
3037

31-
### Global BGP Peer
38+
## Peering Outside The Cluster
39+
### Global External BGP Peers
3240

33-
An optional global BGP peer can be configured by specifying `--peer-asn` and `--peer-router` parameters. When configured
34-
each node in the cluster forms a peer relationship with specified global peer. Pod cidr, cluster IP's get advertised to
35-
the global BGP peer. For redundancy you can also configure more than one peer router by specifying comma seperated list
36-
of BGP peers for `--peer-router` flag, like `--peer-router=192.168.1.99,192.168.1.100`
41+
An optional global BGP peer can be configured by specifying `--peer-router-asns`
42+
and `--peer-router-ips` parameters. When configured each node in the cluster
43+
forms a peer relationship with specified global peer. Pod CIDR and Cluster IP's
44+
get advertised to the global BGP peer. For redundancy you can also configure
45+
more than one peer router by specifying a slice of BGP peers.
3746

38-
### Node specific BGP peer
47+
For example:
48+
```
49+
--peer-router-ips="192.168.1.99,192.168.1.100"
50+
--peer-router-asns="65000,65000"
51+
```
52+
53+
### Node Specific External BGP Peers
3954

40-
Alternativley, each node can be configured with one or mode node specific BGP peer. Information regarding node specific BGP peer is
41-
read from node API object annotations `net.kuberouter.node.bgppeer.address` and `net.kuberouter.node.bgppeer.asn`.
55+
Alternativley, each node can be configured with one or more node specific BGP
56+
peers. Information regarding node specific BGP peer is read from node API object
57+
annotations:
58+
- `io.kube-router.net.peer.ips`
59+
- `io.kube-router.net.peer.asns`
4260

4361

4462
For e.g users can annotate node object with below commands
4563
```
46-
kubectl annotate node <kube-node> “net.kuberouter.node.bgppeer.address=192.168.1.98,192.168.1.99”
47-
kubectl annotate node <kube-node> "net.kuberouter.node.bgppeer.asn=64513”"
64+
kubectl annotate node <kube-node> "io.kube-router.net.peer.ips=192.168.1.98,192.168.1.99"
65+
kubectl annotate node <kube-node> "io.kube-router.net.peer.asns=64513"
66+
```
67+
68+
### BGP Peer Password Authentication
69+
70+
The examples above have assumed there is no password authentication with BGP
71+
peer routers. If you need to use a password for peering, you can use the
72+
`--peer-router-passwords` CLI flag or the `io.kube-router.net.peer.passwords` node
73+
annotation.
74+
75+
#### Base64 Encoding Passwords
76+
77+
To ensure passwords are easily parsed, but not easily read by human eyes,
78+
kube-router requires that they are encoded as base64.
79+
80+
On a Linux or MacOS system you can encode your passwords on the command line:
81+
```
82+
$ echo "SecurePassword" | base64
83+
U2VjdXJlUGFzc3dvcmQK
84+
```
85+
86+
#### Password Configuration Examples
87+
88+
In this CLI flag example the first router (192.168.1.99) uses a password, while
89+
the second (192.168.1.100) does not.
90+
```
91+
--peer-router-ips="192.168.1.99,192.168.1.100"
92+
--peer-router-asns="65000,65000"
93+
--peer-router-passwords="U2VjdXJlUGFzc3dvcmQK,"
94+
```
95+
96+
Note the comma indicating the end of the first password.
97+
98+
Now here's the same example but configured as node annotations:
99+
```
100+
kubectl annotate node <kube-node> "io.kube-router.net.peer.ips=192.168.1.99,192.168.1.100"
101+
kubectl annotate node <kube-node> "io.kube-router.net.peer.asns=65000,65000"
102+
kubectl annotate node <kube-node> "io.kube-router.net.peer.passwords=U2VjdXJlUGFzc3dvcmQK,"
48103
```

0 commit comments

Comments
 (0)