Skip to content

Commit 8b2e21d

Browse files
authored
✨implement separate hub and spoke resources (#59)
* feat: wip - fill out v1beta1 structs Signed-off-by: Artur Shad Nik <[email protected]> * wip: working hub init Signed-off-by: Artur Shad Nik <[email protected]> * chore: make reviewable Signed-off-by: Artur Shad Nik <[email protected]> * refactor: conditions Signed-off-by: Artur Shad Nik <[email protected]> * fix: implement spoke cleanup Signed-off-by: Artur Shad Nik <[email protected]> * wip: implement addons Signed-off-by: Artur Shad Nik <[email protected]> * chore: make reviewable Signed-off-by: Artur Shad Nik <[email protected]> * chore: make reviewable Signed-off-by: Artur Shad Nik <[email protected]> * chore: fix test data Signed-off-by: Artur Shad Nik <[email protected]> * feat: background delete spokes during hub deletion Signed-off-by: Artur Shad Nik <[email protected]> * fix: allow parallel spoke reconciles; actually do hub cleanup Signed-off-by: Artur Shad Nik <[email protected]> * refactor: simplify addon conds; disable hubAddons during cleanup Signed-off-by: Artur Shad Nik <[email protected]> * refactor: dont fail if hub not found Signed-off-by: Artur Shad Nik <[email protected]> * feat: allow global and per-spoke klusterlet values Signed-off-by: Artur Shad Nik <[email protected]> * refactor: use 3rd party merge lib; set managed spoke fields Signed-off-by: Artur Shad Nik <[email protected]> * chore: make reviewable Signed-off-by: Artur Shad Nik <[email protected]> * refactor: get addon state from cluster not status Signed-off-by: Artur Shad Nik <[email protected]> * fix: tweak hub addon reconcile Signed-off-by: Artur Shad Nik <[email protected]> * chore: prevent flooding stdout Signed-off-by: Artur Shad Nik <[email protected]> * feat: watch resource updates to speed up syncs Signed-off-by: Artur Shad Nik <[email protected]> * chore: make reviewable Signed-off-by: Artur Shad Nik <[email protected]> * fix: spelling Signed-off-by: Artur Shad Nik <[email protected]> * feat: webhooks Signed-off-by: Artur Shad Nik <[email protected]> * chore: some rabbit comments Signed-off-by: Artur Shad Nik <[email protected]> * fix: dont append empty annotations Signed-off-by: Artur Shad Nik <[email protected]> * chore: update validation, skip redundant hub cleanup checks Signed-off-by: Artur Shad Nik <[email protected]> * chore: tighten spoke reconcile predicate Signed-off-by: Artur Shad Nik <[email protected]> * chore: make reviewable Signed-off-by: Artur Shad Nik <[email protected]> * chore: add kubeconfig validation on update Signed-off-by: Artur Shad Nik <[email protected]> * chore: add kubeconfig validation on update Signed-off-by: Artur Shad Nik <[email protected]> * refactor: many things Signed-off-by: Artur Shad Nik <[email protected]> * chore: error handling Signed-off-by: Artur Shad Nik <[email protected]> * fix: update event map for enqueue Signed-off-by: Artur Shad Nik <[email protected]> * chore: tweak enqueue mapping Signed-off-by: Artur Shad Nik <[email protected]> * chore: allow apiServer updates Signed-off-by: Artur Shad Nik <[email protected]> * fix: guard against nil hub in spoke upgrade; only delete spokes managed by the hub being deleted Signed-off-by: Artur Shad Nik <[email protected]> * chore: nil check Signed-off-by: Artur Shad Nik <[email protected]> * feat: namespaced hub Signed-off-by: Artur Shad Nik <[email protected]> * feat: v1beta1 secret namespace Signed-off-by: Artur Shad Nik <[email protected]> * refactor: fix resource interface Signed-off-by: Artur Shad Nik <[email protected]> * test: fix int test scaffolds Signed-off-by: Artur Shad Nik <[email protected]> * test: add controller integration tests Signed-off-by: Artur Shad Nik <[email protected]> * test: add webhook int tests Signed-off-by: Artur Shad Nik <[email protected]> * chore: delete unused hub defaulting webhook Signed-off-by: Artur Shad Nik <[email protected]> * test: add some unit tests Signed-off-by: Artur Shad Nik <[email protected]> * feat: update helm webhook configs Signed-off-by: Artur Shad Nik <[email protected]> * chore: dont requeue spoke if hub timeout/verbosity updated Signed-off-by: Artur Shad Nik <[email protected]> * fix: remove defaulter, set overrides in controller Signed-off-by: Artur Shad Nik <[email protected]> * chore: helm chart Signed-off-by: Artur Shad Nik <[email protected]> * fix: helm template, values Signed-off-by: Artur Shad Nik <[email protected]> * fix: tweak helm chart Signed-off-by: Artur Shad Nik <[email protected]> * fix: guard against nil configmap ref Signed-off-by: Artur Shad Nik <[email protected]> * test: WIP - add e2e tests Signed-off-by: Artur Shad Nik <[email protected]> * test: reintroduce v1alpha1 suite Signed-off-by: Artur Shad Nik <[email protected]> * chore: make reviewable Signed-off-by: Artur Shad Nik <[email protected]> * test: fix v1alpha1 tests; update chart Signed-off-by: Artur Shad Nik <[email protected]> * ci: label filters Signed-off-by: Artur Shad Nik <[email protected]> * ci: change label filter collector logic Signed-off-by: Artur Shad Nik <[email protected]> * fix: var name Signed-off-by: Artur Shad Nik <[email protected]> * test: update test helper Signed-off-by: Artur Shad Nik <[email protected]> * chore: validation wording Signed-off-by: Artur Shad Nik <[email protected]> * ci: only run planner on pull_request_target Signed-off-by: Artur Shad Nik <[email protected]> * chore: rabbit Signed-off-by: Artur Shad Nik <[email protected]> * feat: long-lived addon clientset for webhooks Signed-off-by: Artur Shad Nik <[email protected]> * fix: add back pull_request event for now Signed-off-by: Artur Shad Nik <[email protected]> * test: update tests Signed-off-by: Artur Shad Nik <[email protected]> * chore: enforce hub one-of condition Signed-off-by: Artur Shad Nik <[email protected]> * chore: move code around Signed-off-by: Artur Shad Nik <[email protected]> * chore: address review comments Signed-off-by: Artur Shad Nik <[email protected]> * chore: default labels Signed-off-by: Artur Shad Nik <[email protected]> * ci: always apply labels Signed-off-by: Artur Shad Nik <[email protected]> * fix: dbl pointer Signed-off-by: Artur Shad Nik <[email protected]> --------- Signed-off-by: Artur Shad Nik <[email protected]>
1 parent fe2f31e commit 8b2e21d

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

51 files changed

+8871
-687
lines changed

.github/workflows/e2e.yml

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -9,6 +9,9 @@ on:
99
artifacts:
1010
required: false
1111
type: string
12+
ginkgoLabelFilter:
13+
required: false
14+
type: string
1215

1316
env:
1417
GO_REQUIRED_MIN_VERSION: ''
@@ -33,7 +36,8 @@ jobs:
3336

3437
- name: Test E2E
3538
run: |
36-
cd ${{ inputs.repo }} && make test-e2e
39+
cd ${{ inputs.repo }}
40+
LABEL_FILTER=${{ inputs.ginkgoLabelFilter }} make test-e2e
3741
3842
- name: Upload Artifacts
3943
if: |

.github/workflows/planner.yml

Lines changed: 49 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,7 @@ name: Planner
33
on:
44
pull_request:
55
pull_request_target:
6-
types: [unlabeled]
6+
types: [opened, labeled, unlabeled, reopened, synchronize, ready_for_review]
77
workflow_dispatch:
88

99
concurrency:
@@ -92,6 +92,50 @@ jobs:
9292
9393
echo "Matrix: $matrixJson"
9494
echo "Artifacts: $artifacts_json"
95+
96+
extract-label-filter:
97+
name: extract-label-filter
98+
needs: generate-matrix
99+
runs-on: ubuntu-latest
100+
outputs:
101+
ginkgoLabelFilter: ${{ steps.extract-label-filter.outputs.ginkgoLabelFilter }}
102+
steps:
103+
- name: Extract label filter
104+
id: extract-label-filter
105+
run: |
106+
set -e
107+
# Find labels that start with 'ginkgo-filter:'
108+
LABELS="${{ join(github.event.pull_request.labels.*.name, ',') }}"
109+
110+
# Array to collect all filters
111+
FILTERS=()
112+
113+
# Extract all ginkgo filters
114+
for label in $(echo $LABELS | tr ',' '\n'); do
115+
if [[ $label == ginkgo-filter:* ]]; then
116+
# Extract the filter part after the prefix
117+
FILTER="${label#ginkgo-filter:}"
118+
echo "Found Ginkgo filter in label: $FILTER"
119+
FILTERS+=("$FILTER")
120+
fi
121+
done
122+
123+
# If we have filters, combine them with OR operator
124+
if [ ${#FILTERS[@]} -gt 0 ]; then
125+
COMBINED_FILTER=""
126+
127+
# OR each filter
128+
for i in "${!FILTERS[@]}"; do
129+
if [ $i -eq 0 ]; then
130+
COMBINED_FILTER="(${FILTERS[$i]})"
131+
else
132+
COMBINED_FILTER="$COMBINED_FILTER||(${FILTERS[$i]})"
133+
fi
134+
done
135+
136+
echo "Final ginkgo label filter: $COMBINED_FILTER"
137+
echo "ginkgoLabelFilter=$COMBINED_FILTER" >> $GITHUB_OUTPUT
138+
fi
95139
96140
call-test:
97141
name: test
@@ -108,7 +152,9 @@ jobs:
108152

109153
call-e2e:
110154
name: e2e
111-
needs: generate-matrix
155+
needs:
156+
- generate-matrix
157+
- extract-label-filter
112158
if: |
113159
needs.generate-matrix.outputs.matrix != ''
114160
strategy:
@@ -118,4 +164,5 @@ jobs:
118164
with:
119165
repo: ${{ matrix.repo }}
120166
artifacts: ${{ fromJson(needs.generate-matrix.outputs.artifacts)[matrix.repo] }}
167+
ginkgoLabelFilter: ${{ needs.extract-label-filter.outputs.ginkgoLabelFilter }}
121168
secrets: inherit
Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,4 @@
11
# More info: https://docs.docker.com/engine/reference/builder/#dockerignore-file
22
# Ignore build and test binaries.
33
bin/
4+
tmp/

fleetconfig-controller/Makefile

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -117,6 +117,7 @@ test-unit: manifests generate fmt vet envtest ## Run unit tests.
117117
-coverprofile=$(COVER_DIR)/unit/cover.out \
118118
$(shell go list ./... | grep -v '/test/e2e')
119119

120+
LABEL_FILTER ?= v1beta1
120121
.PHONY: test-e2e
121122
test-e2e: kind kubectl ginkgo support-bundle ## Run e2e tests in the top-level test directory.
122123
@mkdir -p $(COVER_DIR)/e2e
@@ -125,7 +126,7 @@ test-e2e: kind kubectl ginkgo support-bundle ## Run e2e tests in the top-level t
125126
$(GINKGO) run -vv \
126127
--cover \
127128
--coverpkg=./... \
128-
--label-filter="fleetconfig" \
129+
--label-filter="$(if $(LABEL_FILTER),$(LABEL_FILTER),v1beta1)" \
129130
--output-dir=$(COVER_DIR)/e2e \
130131
--timeout 20m \
131132
./test/e2e/

fleetconfig-controller/PROJECT

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -26,7 +26,7 @@ resources:
2626
path: github.com/open-cluster-management-io/lab/fleetconfig-controller/api/v1beta1
2727
version: v1beta1
2828
webhooks:
29-
defaulting: true
29+
defaulting: false
3030
validation: true
3131
webhookVersion: v1
3232
- api:
@@ -36,7 +36,7 @@ resources:
3636
path: github.com/open-cluster-management-io/lab/fleetconfig-controller/api/v1beta1
3737
version: v1beta1
3838
webhooks:
39-
defaulting: true
39+
defaulting: false
4040
validation: true
4141
webhookVersion: v1
4242
version: "3"

fleetconfig-controller/api/v1alpha1/fleetconfig_types.go

Lines changed: 30 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -23,10 +23,11 @@ import (
2323
"sort"
2424
"time"
2525

26-
"open-cluster-management.io/ocm/pkg/operator/helpers/chart"
27-
2826
corev1 "k8s.io/api/core/v1"
2927
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
28+
"open-cluster-management.io/ocm/pkg/operator/helpers/chart"
29+
30+
"github.com/open-cluster-management-io/lab/fleetconfig-controller/internal/args"
3031
)
3132

3233
// FleetConfigSpec defines the desired state of FleetConfig.
@@ -675,6 +676,33 @@ func (r *ResourceValues) String() string {
675676
return ""
676677
}
677678

679+
// GetRequests returns the resource requests.
680+
func (r ResourceSpec) GetRequests() args.ResourceValues {
681+
if r.Requests == nil {
682+
return &ResourceValues{}
683+
}
684+
return r.Requests
685+
}
686+
687+
// GetLimits returns the resource limits.
688+
func (r ResourceSpec) GetLimits() args.ResourceValues {
689+
if r.Limits == nil {
690+
return &ResourceValues{}
691+
}
692+
return r.Limits
693+
}
694+
695+
// GetQosClass returns the QoS class.
696+
func (r ResourceSpec) GetQosClass() string {
697+
return r.QosClass
698+
}
699+
700+
// Ensure ResourceSpec implements args.ResourceSpec interface
701+
var _ args.ResourceSpec = (*ResourceSpec)(nil)
702+
703+
// Ensure ResourceValues implements args.ResourceValues interface
704+
var _ args.ResourceValues = (*ResourceValues)(nil)
705+
678706
// RegistrationAuth provides specifications for registration authentication.
679707
type RegistrationAuth struct {
680708
// The registration authentication driver to use.
Lines changed: 157 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,157 @@
1+
package v1beta1
2+
3+
import (
4+
"fmt"
5+
"time"
6+
7+
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
8+
9+
"github.com/open-cluster-management-io/lab/fleetconfig-controller/internal/args"
10+
)
11+
12+
// Kubeconfig is the configuration for a kubeconfig.
13+
type Kubeconfig struct {
14+
// A reference to an existing secret containing a kubeconfig.
15+
// Must be provided for remote clusters.
16+
// For same-cluster, must be provided unless InCluster is set to true.
17+
// +optional
18+
SecretReference *SecretReference `json:"secretReference,omitempty"`
19+
20+
// If set, the kubeconfig will be read from the cluster.
21+
// Only applicable for same-cluster operations.
22+
// Defaults to false.
23+
// +optional
24+
InCluster bool `json:"inCluster,omitempty"`
25+
26+
// The context to use in the kubeconfig file.
27+
// +optional
28+
Context string `json:"context,omitempty"`
29+
}
30+
31+
// SecretReference describes how to retrieve a kubeconfig stored as a secret in the same namespace as the resource.
32+
type SecretReference struct {
33+
// The name of the secret.
34+
// +required
35+
Name string `json:"name"`
36+
37+
// The map key to access the kubeconfig. Defaults to 'kubeconfig'.
38+
// +kubebuilder:default:="kubeconfig"
39+
// +optional
40+
KubeconfigKey string `json:"kubeconfigKey,omitempty"`
41+
}
42+
43+
// ResourceSpec defines resource limits and requests for all managed clusters.
44+
type ResourceSpec struct {
45+
// The resource limits of all the containers managed by the Cluster Manager or Klusterlet operators.
46+
// +optional
47+
Limits *ResourceValues `json:"limits,omitempty"`
48+
49+
// The resource requests of all the containers managed by the Cluster Manager or Klusterlet operators.
50+
// +optional
51+
Requests *ResourceValues `json:"requests,omitempty"`
52+
53+
// The resource QoS class of all the containers managed by the Cluster Manager or Klusterlet operators.
54+
// One of Default, BestEffort or ResourceRequirement.
55+
// +kubebuilder:validation:Enum=Default;BestEffort;ResourceRequirement
56+
// +kubebuilder:default:="Default"
57+
// +optional
58+
QosClass string `json:"qosClass,omitempty"`
59+
}
60+
61+
// ResourceValues detail container resource constraints.
62+
type ResourceValues struct {
63+
// The number of CPU units to request, e.g., '800m'.
64+
// +optional
65+
CPU string `json:"cpu,omitempty"`
66+
67+
// The amount of memory to request, e.g., '8Gi'.
68+
// +optional
69+
Memory string `json:"memory,omitempty"`
70+
}
71+
72+
// String returns a string representation of the resource values.
73+
func (r *ResourceValues) String() string {
74+
if r.CPU != "" && r.Memory != "" {
75+
return fmt.Sprintf("cpu=%s,memory=%s", r.CPU, r.Memory)
76+
} else if r.CPU != "" {
77+
return fmt.Sprintf("cpu=%s", r.CPU)
78+
} else if r.Memory != "" {
79+
return fmt.Sprintf("memory=%s", r.Memory)
80+
}
81+
return ""
82+
}
83+
84+
// GetRequests returns the resource requests.
85+
func (r ResourceSpec) GetRequests() args.ResourceValues {
86+
if r.Requests == nil {
87+
return &ResourceValues{}
88+
}
89+
return r.Requests
90+
}
91+
92+
// GetLimits returns the resource limits.
93+
func (r ResourceSpec) GetLimits() args.ResourceValues {
94+
if r.Limits == nil {
95+
return &ResourceValues{}
96+
}
97+
return r.Limits
98+
}
99+
100+
// GetQosClass returns the QoS class.
101+
func (r ResourceSpec) GetQosClass() string {
102+
return r.QosClass
103+
}
104+
105+
// Ensure ResourceSpec implements args.ResourceSpec interface
106+
var _ args.ResourceSpec = (*ResourceSpec)(nil)
107+
108+
// Ensure ResourceValues implements args.ResourceValues interface
109+
var _ args.ResourceValues = (*ResourceValues)(nil)
110+
111+
// NewCondition returns a new v1beta1.Condition.
112+
func NewCondition(msg, cType string, status, wantStatus metav1.ConditionStatus) Condition {
113+
return Condition{
114+
Condition: metav1.Condition{
115+
Status: status,
116+
Message: msg,
117+
Reason: ReconcileSuccess,
118+
Type: cType,
119+
LastTransitionTime: metav1.Time{Time: time.Now()},
120+
},
121+
WantStatus: wantStatus,
122+
}
123+
}
124+
125+
// Condition describes the state of a FleetConfig.
126+
type Condition struct {
127+
metav1.Condition `json:",inline"`
128+
WantStatus metav1.ConditionStatus `json:"wantStatus"`
129+
}
130+
131+
// Equal returns true if the condition is identical to the supplied condition, ignoring the LastTransitionTime.
132+
func (c Condition) Equal(other Condition) bool {
133+
return c.Type == other.Type && c.Status == other.Status && c.WantStatus == other.WantStatus &&
134+
c.Reason == other.Reason && c.Message == other.Message
135+
}
136+
137+
// RegistrationAuth provides specifications for registration authentication.
138+
type RegistrationAuth struct {
139+
// The registration authentication driver to use.
140+
// Options are:
141+
// - csr: Use the default CSR-based registration authentication.
142+
// - awsirsa: Use AWS IAM Role for Service Accounts (IRSA) registration authentication.
143+
// The set of valid options is open for extension.
144+
// +kubebuilder:validation:Enum=csr;awsirsa
145+
// +kubebuilder:default:="csr"
146+
// +optional
147+
Driver string `json:"driver,omitempty"`
148+
149+
// The Hub cluster ARN for awsirsa registration authentication. Required when Type is awsirsa, otherwise ignored.
150+
// +optional
151+
HubClusterARN string `json:"hubClusterARN,omitempty"`
152+
153+
// List of AWS EKS ARN patterns so any EKS clusters with these patterns will be auto accepted to join with hub cluster.
154+
// Example pattern: "arn:aws:eks:us-west-2:123456789013:cluster/.*"
155+
// +optional
156+
AutoApprovedARNPatterns []string `json:"autoApprovedARNPatterns,omitempty"`
157+
}

0 commit comments

Comments
 (0)