Skip to content

feat: add Kubernetes 1.33 support and upgrade Calico to 3.30.0 #162

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 6 commits into
base: main
Choose a base branch
from
Draft

feat: add Kubernetes 1.33 support and upgrade Calico to 3.30.0 #162

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
8443e14
feat: add Kubernetes 1.33 support and upgrade Calico to 3.30.0
abhikumar2204 Aug 13, 2025
ef090ce
feat: add Kubernetes 1.33 support and upgrade Calico to 3.30.0
abhikumar2204 Aug 13, 2025
4a9d248
feat: add Kubernetes 1.33 support and upgrade Calico to 3.30.0
abhikumar2204 Aug 13, 2025
02d5148
feat: add Kubernetes 1.33 support and upgrade Calico to 3.30.0
abhikumar2204 Aug 13, 2025
1d8ecd0
fix(ansible/docker): improve unattended-upgrades handling and Dockerf…
Sep 23, 2025
eb7042e
fix(ansible/docker): improve unattended-upgrades handling and Dockerf…
Sep 23, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
56 changes: 41 additions & 15 deletions Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,39 +1,65 @@
# Base layer
FROM ubuntu:22.04 AS baseline

RUN apt-get update && apt-get upgrade -y --no-install-recommends \
&& apt-get install -y python3 python3-dev python3-pip curl unzip gnupg --no-install-recommends \
&& apt-get install -y \
python3 python3-dev python3-pip \
curl unzip gnupg lsb-release ca-certificates software-properties-common \
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I understand the extra lines, but wondering about lsb-release, ca-certificates, and software-properties-common

--no-install-recommends \
&& update-alternatives --install /usr/bin/python python /usr/bin/python3 1 \
&& update-alternatives --install /usr/bin/pip pip /usr/bin/pip3 1 \
&& apt-get clean && rm -rf /var/lib/apt/lists/*

# Layers used for building/downloading/installing tools

# Tool building layer
FROM baseline AS tool_builder

ARG HELM_VERSION=3.17.1
ARG KUBECTL_VERSION=1.30.10
ARG TERRAFORM_VERSION=1.10.5-*
ARG KUBECTL_VERSION=1.32.7

WORKDIR /build
ARG TERRAFORM_VERSION=1.10.5
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this compatible with the older syntax? Just making sure ;)


WORKDIR /build
SHELL ["/bin/bash", "-o", "pipefail", "-c"]
RUN curl -fsSL https://apt.releases.hashicorp.com/gpg | apt-key add - \
&& echo "deb [arch=amd64] https://apt.releases.hashicorp.com focal main" > /etc/apt/sources.list.d/tf.list \
&& apt-get update \
&& curl -sLO https://dl.k8s.io/release/v$KUBECTL_VERSION/bin/linux/amd64/kubectl && chmod 755 ./kubectl \
&& curl -ksLO https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3 && chmod 755 get-helm-3 \
&& ./get-helm-3 --version v$HELM_VERSION --no-sudo \
&& apt-get install -y terraform=$TERRAFORM_VERSION --no-install-recommends \

# Install kubectl
RUN curl -sLO https://dl.k8s.io/release/v${KUBECTL_VERSION}/bin/linux/amd64/kubectl \
&& chmod 755 ./kubectl

# Install helm
RUN curl -fsSL https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3 -o get-helm-3 \
&& chmod 755 get-helm-3 \
&& ./get-helm-3 --version v${HELM_VERSION} --no-sudo

# Install terraform (APT + fallback to binary)
RUN set -e \
&& curl -fsSL https://apt.releases.hashicorp.com/gpg | gpg --dearmor -o /usr/share/keyrings/hashicorp.gpg \
&& echo "deb [signed-by=/usr/share/keyrings/hashicorp.gpg] https://apt.releases.hashicorp.com $(lsb_release -cs) main" > /etc/apt/sources.list.d/hashicorp.list \
&& apt-get update || true \
&& (apt-get install -y terraform=${TERRAFORM_VERSION} --no-install-recommends || \
(echo "APT install failed. Falling back to direct download..." && \
curl -fsSL https://releases.hashicorp.com/terraform/${TERRAFORM_VERSION}/terraform_${TERRAFORM_VERSION}_linux_amd64.zip -o terraform.zip \
&& unzip terraform.zip \
&& mv terraform /usr/bin/terraform \
&& chmod +x /usr/bin/terraform \
&& rm terraform.zip)) \
Comment on lines +24 to +45
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A single RUN command creates a single layer in docker. Not sure why you have broken these items out. Especially since kubectl is one of the lower or base layer. Any change there causes all other RUN commands and build option to rebuild. If that is the intent, then fine, but I am sure you are increasing the size of the docker container.

Comment on lines +35 to +45
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Was this a problem? Never saw where apt install failed? Guessing you all have seen this or this is part of this PR? The title says "Add kubernetes 1.33 support and upgrade Calico to 3.30.0"

&& apt-get clean && rm -rf /var/lib/apt/lists/*

# Installation steps

# Final image with tools and dependencies
FROM baseline

RUN apt-get update && apt-get -y install git sshpass jq \
# Install additional packages
RUN apt-get update && apt-get install -y \
git sshpass jq \
&& apt-get clean && rm -rf /var/lib/apt/lists/*

# Copy tools from builder stage
COPY --from=tool_builder /usr/local/bin/helm /usr/local/bin/helm
COPY --from=tool_builder /build/kubectl /usr/local/bin/kubectl
COPY --from=tool_builder /usr/bin/terraform /usr/bin/terraform

# Copy your source
WORKDIR /viya4-iac-k8s
COPY . /viya4-iac-k8s/

Expand All @@ -53,4 +79,4 @@ ENV TF_VAR_ansible_vars=/workspace/ansible-vars.yaml
ENV ANSIBLE_CONFIG=/viya4-iac-k8s/ansible.cfg

VOLUME ["/workspace"]
ENTRYPOINT ["/viya4-iac-k8s/docker-entrypoint.sh"]
ENTRYPOINT ["/viya4-iac-k8s/docker-entrypoint.sh"]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

File needs a correct line ending. This can be done in VS Code automatically. This usually happens when switching between windows/linux or your editor, i.e. nano or something strips the last line without adding the correct file ending.

4 changes: 2 additions & 2 deletions docs/CONFIG-VARS.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -69,9 +69,9 @@ Terraform input variables can be set in the following ways:

| Name | Description | Type | Default | Notes |
| :--- | :--- | :--- | :--- | :--- |
| cluster_version | Kubernetes version | string | "1.30.10" | Valid values are listed here: [SAS Viya platform Supported Kubernetes Versions](https://documentation.sas.com/?cdcId=itopscdc&cdcVersion=default&docsetId=itopssr&docsetTarget=n1ika6zxghgsoqn1mq4bck9dx695.htm#p03v0o4maa8oidn1awe0w4xlxcf6). |
| cluster_version | Kubernetes version | string | "1.32.7" | Valid values are listed here: [SAS Viya platform Supported Kubernetes Versions](https://documentation.sas.com/?cdcId=itopscdc&cdcVersion=default&docsetId=itopssr&docsetTarget=n1ika6zxghgsoqn1mq4bck9dx695.htm#p03v0o4maa8oidn1awe0w4xlxcf6). |
| cluster_cni | Kubernetes container network interface (CNI) | string | "calico" | |
| cluster_cni_version | Kubernetes Container Network Interface (CNI) Version | string | "3.29.0" | |
| cluster_cni_version | Kubernetes Container Network Interface (CNI) Version | string | "3.30.0" | |
| cluster_cri | Kubernetes container runtime interface (CRI) | string | "containerd" | |
| cluster_cri_version | Version of the CRI specifed by `cluster_cri` to be installed | string | "1.7.24" | Set as an empty string to use the latest upstream version from the Docker APT repository. Currently only containerd is supported, see the [releases page](https://github.com/containerd/containerd/releases) for available versions |
| cluster_service_subnet | Kubernetes service subnet | string | "10.43.0.0/16" | |
Expand Down
4 changes: 2 additions & 2 deletions docs/REQUIREMENTS.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -215,9 +215,9 @@ vsphere_network = "" # Name of the network to to use for the VMs
system_ssh_keys_dir = "~/.ssh" # Directory holding public keys to be used on each machine

# Kubernetes - Cluster
cluster_version = "1.30.10" # Kubernetes version
cluster_version = "1.32.7" # Kubernetes version
cluster_cni = "calico" # Kubernetes Container Network Interface (CNI)
cluster_cni_version = "3.29.0" # Kubernetes Container Network Interface (CNI) Version
cluster_cni_version = "3.30.0" # Kubernetes Container Network Interface (CNI) Version
cluster_cri = "containerd" # Kubernetes Container Runtime Interface (CRI)
cluster_cri_version = "1.7.24" # Kubernetes Container Runtime Interface (CRI) Version
cluster_service_subnet = "10.35.0.0/16" # Kubernetes service subnet
Expand Down
2 changes: 1 addition & 1 deletion docs/user/Dependencies.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,7 @@ Example of using build arguments to control specific versions of dependencies in
```bash
# Override kubectl version
docker build \
--build-arg KUBECTL_VERSION=1.30.10 \
--build-arg KUBECTL_VERSION=1.32.7 \
-t viya4-iac-k8s .
```

Expand Down
4 changes: 2 additions & 2 deletions examples/vsphere/sample-terraform-dhcp.tfvars
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,9 +18,9 @@ vsphere_network = "" # Name of the network to to use for the VMs
system_ssh_keys_dir = "~/.ssh/oss" # Directory holding public keys to be used on each system

# Kubernetes - Cluster
cluster_version = "1.30.10" # Kubernetes Version
cluster_version = "1.32.7" # Kubernetes Version
cluster_cni = "calico" # Kubernetes Container Network Interface (CNI)
cluster_cni_version = "3.29.0" # Kubernetes Container Network Interface (CNI) Version
cluster_cni_version = "3.30.0" # Kubernetes Container Network Interface (CNI) Version
cluster_cri = "containerd" # Kubernetes Container Runtime Interface (CRI)
cluster_cri_version = "1.7.24" # Kubernetes Container Runtime Interface (CRI) Version
cluster_service_subnet = "10.43.0.0/16" # Kubernetes Service Subnet
Expand Down
4 changes: 2 additions & 2 deletions examples/vsphere/sample-terraform-minimal.tfvars
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,9 +18,9 @@ vsphere_network = "" # Name of the network to to use for the VMs
system_ssh_keys_dir = "~/.ssh/oss" # Directory holding public keys to be used on each system

# Kubernetes - Cluster
cluster_version = "1.30.10" # Kubernetes Version
cluster_version = "1.32.7" # Kubernetes Version
cluster_cni = "calico" # Kubernetes Container Network Interface (CNI)
cluster_cni_version = "3.29.0" # Kubernetes Container Network Interface (CNI) Version
cluster_cni_version = "3.30.0" # Kubernetes Container Network Interface (CNI) Version
cluster_cri = "containerd" # Kubernetes Container Runtime Interface (CRI)
cluster_cri_version = "1.7.24" # Kubernetes Container Runtime Interface (CRI) Version
cluster_service_subnet = "10.43.0.0/16" # Kubernetes Service Subnet
Expand Down
4 changes: 2 additions & 2 deletions examples/vsphere/sample-terraform-static-ips.tfvars
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,9 +18,9 @@ vsphere_network = "" # Name of the network to to use for the VMs
system_ssh_keys_dir = "~/.ssh/oss" # Directory holding public keys to be used on each system

# Kubernetes - Cluster
cluster_version = "1.30.10" # Kubernetes Version
cluster_version = "1.32.7" # Kubernetes Version
cluster_cni = "calico" # Kubernetes Container Network Interface (CNI)
cluster_cni_version = "3.29.0" # Kubernetes Container Network Interface (CNI) Version
cluster_cni_version = "3.30.0" # Kubernetes Container Network Interface (CNI) Version
cluster_cri = "containerd" # Kubernetes Container Runtime Interface (CRI)
cluster_cri_version = "1.7.24" # Kubernetes Container Runtime Interface (CRI) Version
cluster_service_subnet = "10.43.0.0/16" # Kubernetes Service Subnet
Expand Down
4 changes: 2 additions & 2 deletions examples/vsphere/sample-terraform-static-singlestore.tfvars
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,9 +18,9 @@ vsphere_network = "" # Name of the network to to use for the VMs
system_ssh_keys_dir = "~/.ssh/oss" # Directory holding public keys to be used on each system

# Kubernetes - Cluster
cluster_version = "1.30.10" # Kubernetes Version
cluster_version = "1.32.7" # Kubernetes Version
cluster_cni = "calico" # Kubernetes Container Network Interface (CNI)
cluster_cni_version = "3.29.0" # Kubernetes Container Network Interface (CNI) Version
cluster_cni_version = "3.30.0" # Kubernetes Container Network Interface (CNI) Version
cluster_cri = "containerd" # Kubernetes Container Runtime Interface (CRI)
cluster_cri_version = "1.7.24" # Kubernetes Container Runtime Interface (CRI) Version
cluster_service_subnet = "10.43.0.0/16" # Kubernetes Service Subnet
Expand Down
4 changes: 2 additions & 2 deletions examples/vsphere/sample-terraform-vi.tfvars
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,9 +18,9 @@ vsphere_network = "" # Name of the network to to use for the VMs
system_ssh_keys_dir = "~/.ssh/oss" # Directory holding public keys to be used on each system

# Kubernetes - Cluster
cluster_version = "1.30.10" # Kubernetes Version
cluster_version = "1.32.7" # Kubernetes Version
cluster_cni = "calico" # Kubernetes Container Network Interface (CNI)
cluster_cni_version = "3.29.0" # Kubernetes Container Network Interface (CNI) Version
cluster_cni_version = "3.30.0" # Kubernetes Container Network Interface (CNI) Version
cluster_cri = "containerd" # Kubernetes Container Runtime Interface (CRI)
cluster_cri_version = "1.7.24" # Kubernetes Container Runtime Interface (CRI) Version
cluster_service_subnet = "10.43.0.0/16" # Kubernetes Service Subnet
Expand Down
23 changes: 19 additions & 4 deletions roles/kubernetes/control_plane/init/primary/templates/kubeadm-config.j2
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,14 +7,21 @@
#
# kubeadm Configuration : https://kubernetes.io/docs/reference/config-api/kubeadm-config.v1beta2/
#
{% if kubernetes_version is version('1.26.0', 'lt', version_type='semver') %}
apiVersion: kubeadm.k8s.io/v1beta2
{% else %}
{% if kubernetes_version is version('1.31.0', 'lt', version_type='semver') %}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this a breaking change? Asking as you've removed the previous logic. If the code is no longer compatible with any previous version, then its a breaking change. This is a process and logistical call on your teams part. Make sure this one creates a new primary version of the tooling.

apiVersion: kubeadm.k8s.io/v1beta3
{% elif kubernetes_version is version('1.33.0', 'lt', version_type='semver') %}
apiVersion: kubeadm.k8s.io/v1beta4
{% else %}
apiVersion: kubeadm.k8s.io/v1
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

kubeadm only has v1beta3 and v1beta4 listed. There is no v1 listed at this time. As long as you're sure it will not break things if someone has entered the wrong k8s version this may be fine. I would only add these once available. Link to reference site for this repo - https://kubernetes.io/docs/reference/config-api/ Latest version of kubeadm config api - https://kubernetes.io/docs/reference/config-api/kubeadm-config.v1beta4/

{% endif %}
kind: ClusterConfiguration
certificatesDir: /etc/kubernetes/pki
kubernetesVersion: v{{ kubernetes_version }}
clusterName: "{{ kubernetes_cluster_name }}"
controlPlaneEndpoint: "{{ kubernetes_vip_fqdn }}:6443"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is the port static? I have seen where this can change, but not sure if that's a configurable value here.


imageRepository: registry.k8s.io

apiServer:
certSANs:
- "{{ kubernetes_vip_fqdn }}"
Expand All @@ -37,7 +44,11 @@ clusterName: "{{ kubernetes_cluster_name }}"
#
# Kubelet Configuration : https://kubernetes.io/docs/reference/config-api/kubelet-config.v1beta1/
#
{% if kubernetes_version is version('1.31.0', 'lt', version_type='semver') %}
apiVersion: kubelet.config.k8s.io/v1beta1
{% else %}
apiVersion: kubelet.config.k8s.io/v1beta2
{% endif %}
Comment on lines +47 to +51
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't see v1beta2 for this config. Only v1beta1 and v1alpha1 along with v1. This needs to change. Link - https://kubernetes.io/docs/reference/config-api/kubelet-config.v1/

kind: KubeletConfiguration
authentication:
anonymous:
Expand All @@ -51,11 +62,15 @@ cgroupDriver: systemd
#
# kube-proxy Configuration : https://kubernetes.io/docs/reference/config-api/kube-proxy-config.v1alpha1/
#
{% if kubernetes_version is version('1.31.0', 'lt', version_type='semver') %}
apiVersion: kubeproxy.config.k8s.io/v1alpha1
{% else %}
apiVersion: kubeproxy.config.k8s.io/v1beta1
{% endif %}
Comment on lines +68 to +69
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There seem to be lots of additions here that are not in the current docs. Please provide info as to where you're finding these apis vs the official doc here - https://kubernetes.io/docs/reference/config-api/kube-proxy-config.v1alpha1/ I believe I am using the latest docs from kubernetes.

kind: KubeProxyConfiguration
mode: "ipvs"
ipvs:
strictARP: true
metricsBindAddress: "0.0.0.0:10249"
enableProfiling: true
clusterCIDR: "{{ kubernetes_pod_subnet }}"
clusterCIDR: "{{ kubernetes_pod_subnet }}"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fix line ending

10 changes: 8 additions & 2 deletions roles/systems/common/tasks/main.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -47,8 +47,9 @@
#
- name: Kill the unattended-upgrade task if it's running
ansible.builtin.shell: |
killall -q -9 unattended-upgrade 2>&1 /dev/null
ignore_errors: true
if pgrep -x unattended-upgrade >/dev/null; then
killall -q -9 unattended-upgrade
fi
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This now fails if the kill does not work? Again, logic change here. Please explain

tags:
- install
- update
Expand All @@ -57,7 +58,12 @@
ansible.builtin.apt:
name: unattended-upgrades
state: absent

purge: true
register: apt_remove_result
until: apt_remove_result is succeeded
retries: 5 # retry up to 5 times
delay: 30 # wait 30s between retries
tags:
- install
- update
Expand Down
10 changes: 5 additions & 5 deletions tests/variable_defaults.tftest.hcl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@
#
# The expected values are:
# - cluster_cni: "calico"
# - cluster_cni_version: "3.29.0"
# - cluster_cni_version: "3.30.0"
#
# In order to run this test, the following environment variables must be exported to the shell where the test is run:
# - vsphere_server
Expand Down Expand Up @@ -54,10 +54,10 @@ vsphere_network = "IACdhcp" # Name of the network to to use for the VMs
system_ssh_keys_dir = "/workspace/.ssh" # Directory holding public keys to be used on each system, TODO REPLACE ME your path may differ

# Kubernetes - Cluster
cluster_version = "1.30.4" # Kubernetes Version
cluster_version = "1.32.7" # Kubernetes Version
# The next two lines are intentionally commented out to test the assigned default values
#cluster_cni = "calico" # Kubernetes Container Network Interface (CNI)
#cluster_cni_version = "3.29.0" # Kubernetes Container Network Interface (CNI) Version
#cluster_cni_version = "3.30.0" # Kubernetes Container Network Interface (CNI) Version
cluster_cri = "containerd" # Kubernetes Container Runtime Interface (CRI)
cluster_cri_version = "1.7.24" # Kubernetes Container Runtime Interface (CRI) Version
cluster_service_subnet = "10.43.0.0/16" # Kubernetes Service Subnet
Expand Down Expand Up @@ -243,15 +243,15 @@ run "cluster_cni_should_default_to_calico" {
}
}

run "cluster_cni_version_should_default_to_3_29_0" {
run "cluster_cni_version_should_default_to_3_30_0" {

command = plan

variables {
}

assert {
condition = var.cluster_cni_version == "3.29.0"
condition = var.cluster_cni_version == "3.30.0"
error_message = "A default value of \"${var.cluster_cni_version}\" for cluster_cni_version was not expected."
}
}
2 changes: 1 addition & 1 deletion variables.tf
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -307,7 +307,7 @@ variable "cluster_cni" {

variable "cluster_cni_version" {
type = string
default = "3.29.0"
default = "3.30.0"
}

variable "cluster_cri" {
Expand Down
Loading