|
| 1 | +// Applies to 4.6 and newer. |
| 2 | + |
| 3 | +:ign-config-version: 3.1.0 |
| 4 | +ifeval::[{product-version} > 4.6] |
| 5 | +:ign-config-version: 3.2.0 |
| 6 | +endif::[] |
| 7 | + |
| 8 | +[id="nw-how-nw-iface-selected_{context}"] |
| 9 | += How the network interface is selected |
| 10 | + |
| 11 | +For installations on bare metal or with virtual machines that have more than one network interface controller (NIC), the NIC that {product-title} uses for communication with the Kubernetes API server is determined by the `nodeip-configuration.service` service unit that is run by systemd when the node boots. |
| 12 | +The service iterates through the network interfaces on the node and the first network interface that is configured with a subnet than can host the IP address for the API server is selected for {product-title} communication. |
| 13 | + |
| 14 | +After the `nodeip-configuration.service` service determines the correct NIC, the service creates the `/etc/systemd/system/kubelet.service.d/20-nodenet.conf` file. |
| 15 | +The `20-nodenet.conf` file sets the `KUBELET_NODE_IP` environment variable to the IP address that the service selected. |
| 16 | + |
| 17 | +When the kubelet service starts, it reads the value of the environment variable from the `20-nodenet.conf` file and sets the IP address as the value to the `--node-ip` kubelet command-line argument. |
| 18 | +As a result, the kubelet service uses the selected IP address as the node IP address. |
| 19 | + |
| 20 | +If hardware or networking is reconfigured after installation, it is possible that the `nodeip-configuration.service` service can select a different NIC after a reboot. |
| 21 | +In some cases, you might be able to detect that a different NIC is selected by reviewing the `INTERNAL-IP` column in the output from the `oc get nodes -o wide` command. |
| 22 | + |
| 23 | +If network communication is disrupted or misconfigured because a different NIC is selected, one strategy for overriding the selection process is to set the correct IP address explicitly. |
| 24 | +The following list identifies the high-level steps and considerations: |
| 25 | + |
| 26 | +* Create a shell script that determines the IP address to use for {product-title} communication. Have the script create a custom unit file such as `/etc/systemd/system/kubelet.service.d/98-nodenet-override.conf`. Use the custom unit file, `98-nodenet-override.conf`, to set the `KUBELET_NODE_IP` environment variable to the IP address. |
| 27 | + |
| 28 | +* Do not overwrite the `/etc/systemd/system/kubelet.service.d/20-nodenet.conf` file. Specify a file name with a numerically higher value such as `98-nodenet-override.conf` in the same directory path. The goal is to have the custom unit file run after `20-nodenet.conf` and override the value of the environment variable. |
| 29 | + |
| 30 | +* Create a machine config object with the shell script as a base64-encoded string and use the Machine Config Operator to deploy the script to the nodes at a file system path such as `/usr/local/bin/override-node-ip.sh`. |
| 31 | + |
| 32 | +* Ensure that `systemctl daemon-reload` runs after the shell script runs. The simplest method is to specify `ExecStart=systemctl daemon-reload` in the machine config, as shown in the following sample. |
| 33 | + |
| 34 | +.Sample machine config to override the network interface for kubelet |
| 35 | +[source,yaml,subs="attributes+"] |
| 36 | +---- |
| 37 | +apiVersion: machineconfiguration.openshift.io/v1 |
| 38 | +kind: MachineConfig |
| 39 | +metadata: |
| 40 | + labels: |
| 41 | + machineconfiguration.openshift.io/role: worker |
| 42 | + name: 98-nodenet-override |
| 43 | +spec: |
| 44 | + config: |
| 45 | + ignition: |
| 46 | + version: {ign-config-version} |
| 47 | + storage: |
| 48 | + files: |
| 49 | + - contents: |
| 50 | + source: data:text/plain;charset=utf-8;base64,<encoded_script> |
| 51 | + mode: 0755 |
| 52 | + overwrite: true |
| 53 | + path: /usr/local/bin/override-node-ip.sh |
| 54 | + systemd: |
| 55 | + units: |
| 56 | + - contents: | |
| 57 | + [Unit] |
| 58 | + Description=Override node IP detection |
| 59 | + Wants=network-online.target |
| 60 | + Before=kubelet.service |
| 61 | + After=network-online.target |
| 62 | + [Service] |
| 63 | + Type=oneshot |
| 64 | + ExecStart=/usr/local/bin/override-node-ip.sh |
| 65 | + ExecStart=systemctl daemon-reload |
| 66 | + [Install] |
| 67 | + WantedBy=multi-user.target |
| 68 | + enabled: true |
| 69 | + name: nodenet-override.service |
| 70 | +
|
| 71 | +---- |
| 72 | + |
| 73 | +// Link to info for creating a machine config. |
| 74 | + |
| 75 | +// Clear temporary attributes |
| 76 | +ifdef::ign-config-version[] |
| 77 | +:!ign-config-version: |
| 78 | +endif::[] |
0 commit comments