Skip to content

Create karpenter custom resources in workload clusters #268

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 35 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
35 commits
Select commit Hold shift + click to select a range
0cd84be
wip
fiunchinho Jun 24, 2025
ef2c8dc
wip2
fiunchinho Jun 25, 2025
ec6e643
wip3
fiunchinho Jun 30, 2025
d8163fe
Use envtest k8s client
fiunchinho Jul 9, 2025
7599b88
Only use security groups and subnets if defined
fiunchinho Jul 9, 2025
ff48046
Use map for subnet and security group selectors
fiunchinho Jul 9, 2025
810a7b3
Take types from karpenter project
fiunchinho Jul 10, 2025
e87d502
Add managed-by label to resources created by controller
fiunchinho Jul 10, 2025
bdc953f
Check if NodePool field is nil
fiunchinho Jul 10, 2025
60cd70e
Avoid setting defaults
fiunchinho Jul 10, 2025
aa07033
Run goimports in api files (including generated files)
fiunchinho Jul 10, 2025
3ae94b5
Refactor tests
fiunchinho Jul 13, 2025
9fe6d2f
Fix user data s3 key
fiunchinho Jul 15, 2025
fc99872
Ignore 'no matches for kind' errors when deleting karpenter CRs
fiunchinho Jul 16, 2025
0482904
Mimic upstream ec2nodeclass
fiunchinho Jul 17, 2025
cc1af99
Add CAPA additional tags to karpenter resources
fiunchinho Jul 21, 2025
22a8cab
Refactor
fiunchinho Jul 22, 2025
2d227a4
Use conditions properly
fiunchinho Jul 24, 2025
e866ede
Fix condition message formatting
fiunchinho Jul 24, 2025
fe3096b
Merge branch 'main' into manage-karpenter-wc-crs
fiunchinho Jul 24, 2025
1f8fb8d
Go mod tidy
fiunchinho Jul 24, 2025
bd9def1
Use different copies for the different patch operations
fiunchinho Jul 24, 2025
6527931
Use update instead of patch for conditions
fiunchinho Jul 24, 2025
38c008c
Rename policy skew condition
fiunchinho Jul 24, 2025
92d3ae4
Fix condition message formatting again
fiunchinho Jul 24, 2025
d06eece
Update conditions immediately
fiunchinho Jul 24, 2025
b8f2c7b
Use patchHelper
fiunchinho Jul 27, 2025
b40b8d2
Improve condition handling
fiunchinho Jul 27, 2025
99941c1
Reconcile AWS metadata endpoint config
fiunchinho Jul 28, 2025
daf0289
Bring back status.ready field
fiunchinho Jul 28, 2025
23ead49
Reconcile taints and startuptaints
fiunchinho Jul 28, 2025
810440b
Extract version skew to its own package
fiunchinho Jul 29, 2025
da0c949
Requeue instead of returning error when version skew policy does not …
fiunchinho Jul 29, 2025
df1bd01
Add comment explaining goimports on generated files
fiunchinho Jul 29, 2025
3ed9a49
Merge branch 'main' into manage-karpenter-wc-crs
fiunchinho Jul 29, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,10 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

## [Unreleased]

### Changed

- Create karpenter custom resources in workload clusters.

## [0.20.0] - 2025-06-23

### Changed
Expand Down
12 changes: 3 additions & 9 deletions Makefile.custom.mk
Original file line number Diff line number Diff line change
Expand Up @@ -26,14 +26,8 @@ crds: controller-gen ## Generate CustomResourceDefinition.
generate: controller-gen crds ## Generate code containing DeepCopy, DeepCopyInto, and DeepCopyObject method implementations.
go generate ./...
$(CONTROLLER_GEN) object:headerFile="hack/boilerplate.go.txt" paths="./..."

.PHONY: fmt
fmt: ## Run go fmt against code.
go fmt ./...

.PHONY: vet
vet: ## Run go vet against code.
go vet ./...
# We need to run goimports after controller-gen to avoid CI complains about goimports in the generated files
@go run golang.org/x/tools/cmd/goimports -w ./api/v1alpha1

.PHONY: create-acceptance-cluster
create-acceptance-cluster: kind
Expand Down Expand Up @@ -127,7 +121,7 @@ coverage-html: test-unit
CONTROLLER_GEN = $(shell pwd)/bin/controller-gen
.PHONY: controller-gen
controller-gen: ## Download controller-gen locally if necessary.
$(call go-get-tool,$(CONTROLLER_GEN),sigs.k8s.io/controller-tools/cmd/controller-gen@v0.16.5)
$(call go-get-tool,$(CONTROLLER_GEN),sigs.k8s.io/controller-tools/cmd/controller-gen@v0.18.0)

ENVTEST = $(shell pwd)/bin/setup-envtest
.PHONY: envtest
Expand Down
1 change: 0 additions & 1 deletion Makefile.gen.app.mk
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,6 @@ lint-chart: check-env ## Runs ct against the default chart.
rm -rf /tmp/$(APPLICATION)-test
mkdir -p /tmp/$(APPLICATION)-test/helm
cp -a ./helm/$(APPLICATION) /tmp/$(APPLICATION)-test/helm/
architect helm template --dir /tmp/$(APPLICATION)-test/helm/$(APPLICATION)
docker run -it --rm -v /tmp/$(APPLICATION)-test:/wd --workdir=/wd --name ct $(IMAGE) ct lint --validate-maintainers=false --charts="helm/$(APPLICATION)"
rm -rf /tmp/$(APPLICATION)-test

Expand Down
2 changes: 1 addition & 1 deletion Makefile.gen.go.mk
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ GITSHA1 := $(shell git rev-parse --verify HEAD)
MODULE := $(shell go list -m)
OS := $(shell go env GOOS)
SOURCES := $(shell find . -name '*.go')
VERSION := $(shell architect project version)

ifeq ($(OS), linux)
EXTLDFLAGS := -static
endif
Expand Down
80 changes: 80 additions & 0 deletions api/v1alpha1/duration.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,80 @@
package v1alpha1

import (
"encoding/json"
"fmt"
"slices"
"time"

"github.com/samber/lo"
)

const Never = "Never"

// NillableDuration is a wrapper around time.Duration which supports correct
// marshaling to YAML and JSON. It uses the value "Never" to signify
// that the duration is disabled and sets the inner duration as nil
type NillableDuration struct {
*time.Duration

// Raw is used to ensure we remarshal the NillableDuration in the same format it was specified.
// This ensures tools like Flux and ArgoCD don't mistakenly detect drift due to our conversion webhooks.
Raw []byte `hash:"ignore"`
}

func MustParseNillableDuration(val string) NillableDuration {
nd := NillableDuration{}
// Use %q instead of %s to ensure that we unmarshal the value as a string and not an int
lo.Must0(json.Unmarshal([]byte(fmt.Sprintf("%q", val)), &nd))
return nd
}

// UnmarshalJSON implements the json.Unmarshaller interface.
func (d *NillableDuration) UnmarshalJSON(b []byte) error {
var str string
err := json.Unmarshal(b, &str)
if err != nil {
return err
}
if str == Never {
return nil
}
pd, err := time.ParseDuration(str)
if err != nil {
return err
}
d.Raw = slices.Clone(b)
d.Duration = &pd
return nil
}

// MarshalJSON implements the json.Marshaler interface.
func (d NillableDuration) MarshalJSON() ([]byte, error) {
if d.Raw != nil {
return d.Raw, nil
}
if d.Duration != nil {
return json.Marshal(d.Duration.String())
}
return json.Marshal(Never)
}

// ToUnstructured implements the value.UnstructuredConverter interface.
func (d NillableDuration) ToUnstructured() interface{} {
if d.Raw != nil {
// Decode the JSON bytes to get the actual string value
var str string
if err := json.Unmarshal(d.Raw, &str); err == nil {
return str
}
// Fallback to string conversion if unmarshal fails
if d.Duration != nil {
return d.Duration.String()
}
return Never
}
if d.Duration != nil {
return d.Duration.String()
}
return Never
}
451 changes: 451 additions & 0 deletions api/v1alpha1/ec2nodeclass.go

Large diffs are not rendered by default.

27 changes: 22 additions & 5 deletions api/v1alpha1/karpentermachinepool_types.go
Original file line number Diff line number Diff line change
Expand Up @@ -18,14 +18,19 @@ package v1alpha1

import (
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
capi "sigs.k8s.io/cluster-api/api/v1beta1"
)

// KarpenterMachinePoolSpec defines the desired state of KarpenterMachinePool.
type KarpenterMachinePoolSpec struct {
// The name or the Amazon Resource Name (ARN) of the instance profile associated
// with the IAM role for the instance. The instance profile contains the IAM
// role.
IamInstanceProfile string `json:"iamInstanceProfile,omitempty"`
// NodePool specifies the configuration for the Karpenter NodePool
// +optional
NodePool *NodePoolSpec `json:"nodePool,omitempty"`

// EC2NodeClass specifies the configuration for the Karpenter EC2NodeClass
// +optional
EC2NodeClass *EC2NodeClassSpec `json:"ec2NodeClass,omitempty"`

// ProviderIDList are the identification IDs of machine instances provided by the provider.
// This field must match the provider IDs as seen on the node objects corresponding to a machine pool's machine instances.
// +optional
Expand All @@ -34,13 +39,17 @@ type KarpenterMachinePoolSpec struct {

// KarpenterMachinePoolStatus defines the observed state of KarpenterMachinePool.
type KarpenterMachinePoolStatus struct {
// Ready is true when the provider resource is ready.
// Ready denotes that the KarpenterMachinePool is ready and fulfilling the infrastructure contract.
// +optional
Ready bool `json:"ready"`

// Replicas is the most recently observed number of replicas
// +optional
Replicas int32 `json:"replicas"`

// Conditions defines current service state of the KarpenterMachinePool.
// +optional
Conditions capi.Conditions `json:"conditions,omitempty"`
}

// +kubebuilder:object:root=true
Expand Down Expand Up @@ -72,3 +81,11 @@ type KarpenterMachinePoolList struct {
func init() {
SchemeBuilder.Register(&KarpenterMachinePool{}, &KarpenterMachinePoolList{})
}

func (in *KarpenterMachinePool) GetConditions() capi.Conditions {
return in.Status.Conditions
}

func (in *KarpenterMachinePool) SetConditions(conditions capi.Conditions) {
in.Status.Conditions = conditions
}
Loading
Loading