Skip to content

Latest commit

 

History

History
175 lines (132 loc) · 5.65 KB

cni-cilium-oss.mdx

File metadata and controls

175 lines (132 loc) · 5.65 KB
sidebar_label title description hide_table_of_contents type category sidebar_class_name logoUrl tags
Cilium
Cilium
Cilium network pack for Spectro Cloud Palette
true
integration
network
amd64
hide-from-sidebar
packs
cilium
network

Versions Supported

:::warning

If you are using Bring Your Own Operating System (BYOOS), then HWE (Hardware Enabled) Kernel or a Kernel that supports eBPF modules needs to be provisioned.

:::

Configure Cilium for Agent Mode Edge Clusters

For agent mode Edge clusters, add the following stages to the OS pack YAML file to enable Cilium.

stages:
  boot.before:
    - name: "Ensure CNI directory permissions on restart"
      if: "[ -d /opt/cni/bin ]"
      commands:
        - chown --recursive root:root /opt/cni/bin
  boot:
    - name: "Ensure CNI directory permissions on restart"
      if: "[ -d /opt/cni/bin ]"
      commands:
        - chown --recursive root:root /opt/cni/bin
  boot.after:
    - name: "Ensure CNI directory permissions on restart"
      if: "[ -d /opt/cni/bin ]"
      commands:
        - chown --recursive root:root /opt/cni/bin

Disable UDP Segmentation on VMXNET3 Edge Hosts for Clusters with Overlay Networks

Due to a known issue with VMware's VMXNET3 adapter, which is widely used in different virtual machine management services, including VMware vSphere and Hyper-V, Cilium Pods may face network connectivity issues.

If deploying an Edge host in a virtual machine environment using a VMXNET3 adapter and enabling an for your cluster, add the following commands in the user-data file at the boot stage. Replace <interface-name> with the name of the network interface on your Edge host.

stages:
  initramfs:
    - name: "Disable UDP segmentation"
      commands:
        - ethtool --offload <interface-name> tx-udp_tnl-segmentation off
        - ethtool --offload <interface-name> tx-udp_tnl-csum-segmentation off

Troubleshooting

Scenario - I/O Timeout Error on VMware

If you are deploying a cluster to a VMware environment using the VXLAN tunnel protocol, you may encounter I/O timeout errors. This is due to a known bug in the VXMNET3 adapter that results in VXLAN traffic being dropped. You can learn more about this issue in Cilium's GitHub issue #21801.

You can work around the issue by using one of the two following methods:

  • Option 1: Set a different tunnel protocol in the Cilium configuration. You can set the tunnel protocol to geneve.

    charts:
  cilium:
    tunnelProtocol: "geneve"

  • Option 2: Modify the Operating System (OS) layer of your cluster profile to automatically disable UDP Segmentation Offloading (USO).

    kubeadmconfig:
  preKubeadmCommands:
    # Disable hardware segmentation offloading due to VMXNET3 issue
    - |
      install -m 0755 /dev/null /usr/lib/networkd-dispatcher/routable.d/10-disable-offloading
      cat <<EOF > /usr/lib/networkd-dispatcher/routable.d/10-disable-offloading
      #!/bin/sh
      ethtool -K eth0 tx-udp_tnl-segmentation off
      ethtool -K eth0 tx-udp_tnl-csum-segmentation off
      ethtool --offload eth0 rx off tx off
      EOF
      systemctl restart systemd-networkd

Prerequisite

:::warning

If you are using Bring Your Own Operating System (BYOOS), then HWE (Hardware Enabled) Kernel or a Kernel that supports eBPF modules needs to be provisioned.

:::

Troubleshooting

Scenario - I/O Timeout Error on VMware

If you are deploying a cluster to a VMware environment using the VXLAN tunnel protocol, you may encounter I/O timeout errors. This is due to a known bug in the VXMNET3 adapter that results in VXLAN traffic being dropped. You can learn more about this issue in Cilium's GitHub issue #21801.

You can work around the issue by using one of the two following methods:

  • Option 1: Set a different tunnel protocol in the Cilium configuration. You can set the tunnel protocol to geneve.

    charts:
  cilium:
    tunnelProtocol: "geneve"

  • Option 2: Modify the Operating System (OS) layer of your cluster profile to automatically disable UDP Segmentation Offloading (USO).

    kubeadmconfig:
  preKubeadmCommands:
    # Disable hardware segmentation offloading due to VMXNET3 issue
    - |
      install -m 0755 /dev/null /usr/lib/networkd-dispatcher/routable.d/10-disable-offloading
      cat <<EOF > /usr/lib/networkd-dispatcher/routable.d/10-disable-offloading
      #!/bin/sh
      ethtool -K eth0 tx-udp_tnl-segmentation off
      ethtool -K eth0 tx-udp_tnl-csum-segmentation off
      ethtool --offload eth0 rx off tx off
      EOF
      systemctl restart systemd-networkd

Terraform

You can reference the Cilium pack in Terraform with the following data resource.

data "spectrocloud_registry" "public_registry" {
  name = "Public Repo"
}

data "spectrocloud_pack" "cilium" {
  name    = "cni-cilium-oss"
  version = "1.15.3"
  registry_uid = data.spectrocloud_registry.public_registry.id
}