Skip to content

✨ feat: Support setting EKS authentication mode #5578

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 13 commits into
base: main
Choose a base branch
from
+366 −0
Open

✨ feat: Support setting EKS authentication mode #5578

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
13 commits
Select commit Hold shift + click to select a range
d224304
feat: support setting EKS AuthenticationMode
adammw Aug 27, 2024
092811a
Add GCFLAGS
adammw Sep 6, 2024
b289e83
Add debug log line
adammw Sep 6, 2024
32efafb
Add webhook validation for access config
adammw Sep 12, 2024
52d0e7c
add unit tests
adammw Sep 12, 2024
277cd1a
Make revive happy on naming
adammw Sep 17, 2024
7b51517
fix failing test with invalid mocked input
adammw Sep 17, 2024
67241e7
add accessConfig to v1beta1 type as well
adammw Sep 17, 2024
f5554f1
Revert "add accessConfig to v1beta1 type as well"
joshfrench Jul 1, 2025
910914e
preserve AccessConfig during AWSManagedControlPlane API conversions
joshfrench Jul 1, 2025
7cbcc9f
cruft
joshfrench Jul 2, 2025
d9d523c
fix comment
joshfrench Jul 7, 2025
0b3fa84
rebase main
joshfrench Aug 8, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 15 additions & 0 deletions config/crd/bases/controlplane.cluster.x-k8s.io_awsmanagedcontrolplanes.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2208,6 +2208,21 @@ spec:
description: AWSManagedControlPlaneSpec defines the desired state of an
Amazon EKS Cluster.
properties:
accessConfig:
description: AccessConfig specifies the access configuration information
for the cluster
properties:
authenticationMode:
default: CONFIG_MAP
description: |-
AuthenticationMode specifies the desired authentication mode for the cluster
Defaults to CONFIG_MAP
enum:
- CONFIG_MAP
- API
- API_AND_CONFIG_MAP
type: string
type: object
additionalTags:
additionalProperties:
type: string
Expand Down
15 changes: 15 additions & 0 deletions config/crd/bases/controlplane.cluster.x-k8s.io_awsmanagedcontrolplanetemplates.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,21 @@ spec:
description: AWSManagedControlPlaneSpec defines the desired state
of an Amazon EKS Cluster.
properties:
accessConfig:
description: AccessConfig specifies the access configuration
information for the cluster
properties:
authenticationMode:
default: CONFIG_MAP
description: |-
AuthenticationMode specifies the desired authentication mode for the cluster
Defaults to CONFIG_MAP
enum:
- CONFIG_MAP
- API
- API_AND_CONFIG_MAP
type: string
type: object
additionalTags:
additionalProperties:
type: string
Expand Down
1 change: 1 addition & 0 deletions controlplane/eks/api/v1beta1/conversion.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -117,6 +117,7 @@ func (r *AWSManagedControlPlane) ConvertTo(dstRaw conversion.Hub) error {

dst.Spec.Partition = restored.Spec.Partition
dst.Spec.RestrictPrivateSubnets = restored.Spec.RestrictPrivateSubnets
dst.Spec.AccessConfig = restored.Spec.AccessConfig
dst.Spec.RolePath = restored.Spec.RolePath
dst.Spec.RolePermissionsBoundary = restored.Spec.RolePermissionsBoundary
dst.Status.Version = restored.Status.Version
Expand Down
1 change: 1 addition & 0 deletions controlplane/eks/api/v1beta1/zz_generated.conversion.go
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

13 changes: 13 additions & 0 deletions controlplane/eks/api/v1beta2/awsmanagedcontrolplane_types.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -192,6 +192,10 @@ type AWSManagedControlPlaneSpec struct { //nolint: maligned
// +optional
OIDCIdentityProviderConfig *OIDCIdentityProviderConfig `json:"oidcIdentityProviderConfig,omitempty"`

// AccessConfig specifies the access configuration information for the cluster
// +optional
AccessConfig *AccessConfig `json:"accessConfig,omitempty"`

// VpcCni is used to set configuration options for the VPC CNI plugin
// +optional
VpcCni VpcCni `json:"vpcCni,omitempty"`
Expand Down Expand Up @@ -248,6 +252,15 @@ type EndpointAccess struct {
Private *bool `json:"private,omitempty"`
}

// AccessConfig represents the access configuration information for the cluster
type AccessConfig struct {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
type AccessConfig struct {
type Access struct {

(and JSON field name access)

All YAML is "config", so I'd remove that word unless it's part of "official" AWS terminology, for instance.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The upstream types and API docs both use AccessConfig.

// AuthenticationMode specifies the desired authentication mode for the cluster
// Defaults to CONFIG_MAP
// +kubebuilder:default=CONFIG_MAP
// +kubebuilder:validation:Enum=CONFIG_MAP;API;API_AND_CONFIG_MAP
AuthenticationMode EKSAuthenticationMode `json:"authenticationMode,omitempty"`
}

Comment on lines +256 to +263
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As we are creating a AccessConfig, I suggest we add bootstrapClusterCreatorAdminPermissions as well,
If we do not plan to support this field, do we really need the AccessConfig struct?

WDYT @nrb @damdo ?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

bootstrapClusterCreationAdminPermissions can only be set at cluster creation; the UpdateAccessConfigRequest API doesn't allow changing this. I don't have a strong opinion on adding the new field vs. removing the struct, but we'll need to account for that.

(For additional context, #5583 adds AccessEntry definitions to this struct but now I'm second-guessing where those should live.)

Copy link
Contributor

@punkwalker punkwalker Aug 8, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here is what I think,
We have spec.iamAuthenticatorConfig in ManagedControlPlane. This is used for adding entries in aws-auth configmap.

Similary, we should have separate spec.AccessEntries for role/user mapping and spec.authenticationMode to specify access mode. Because, the default value for mode is CONFIG_MAP which would be required for creating mapping from [spec.iamAuthenticatorConfig].

@joshfrench Let me know what you think?

Also, bootstrapClusterCreatorAdminPermissions can be part of AuthenticationMode struct and have default value true as in CONFIG_MAP mode, regardless of the value, clusterCreator will have admin permissions.

@AndiDog Please share your thoughts.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@punkwalker Agreed that spec.AccessEntries should be a separate field (still planning to handle that in a separate PR.)

To clarify, you are suggesting we add bootstrapClusterCreatorAdminPermissions to AccessConfig and default it to true?

type AccessConfig struct {
    // +kubebuilder:default=CONFIG_MAP
    // +kubebuilder:validation:Enum=CONFIG_MAP;API;API_AND_CONFIG_MAP
    AuthenticationMode EKSAuthenticationMode `json:"authenticationMode,omitempty"`

    // +kubebuilder:default=true
    BootstrapClusterCreatorAdminPermissions bool `json:"bootstrapClusterCreatorAdminPermissions"`
}

That feels right to me, since it matches the shape of the EKS API.

My question was what we should do if the user changes bootstrapClusterCreatorAdminPermissions on an existing cluster. The UpdateAccessConfig API doesn't allow it, because it makes no sense once the cluster has been boostrapped already. The only valid use of UpdateAccessConfig is to change the authentication mode.

So if a user changes bootstrapClusterCreationAdminPermissions on an existing cluster we could:

  1. Ignore bootstrapClusterCreatorAdminPermissions entirely and only submit the request if it changes the authentication mode
  2. Submit the request if it changes the authentication mode and log a warning if bootstrapClusterCreatorAdminPermissions also changes
  3. Fail validation and return an error if bootstrapClusterCreationAdminPermissions changes, regardless of whether the authentication mode also changes

1 feels bad, 2 is low friction, 3 is the most explicit. Thoughts?

Copy link
Contributor

@punkwalker punkwalker Aug 12, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To clarify, you are suggesting we add bootstrapClusterCreatorAdminPermissions to AccessConfig and default it to true?

Yes.

2 feels a golden midway instead of failing. But it should be documented explicitly in the CR.

// EncryptionConfig specifies the encryption configuration for the EKS clsuter.
type EncryptionConfig struct {
// Provider specifies the ARN or alias of the CMK (in AWS KMS)
Expand Down
23 changes: 23 additions & 0 deletions controlplane/eks/api/v1beta2/awsmanagedcontrolplane_webhook.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -140,6 +140,7 @@ func (*awsManagedControlPlaneWebhook) ValidateUpdate(ctx context.Context, oldObj
allErrs = append(allErrs, r.validateEKSClusterNameSame(oldAWSManagedControlplane)...)
allErrs = append(allErrs, r.validateEKSVersion(oldAWSManagedControlplane)...)
allErrs = append(allErrs, r.Spec.Bastion.Validate()...)
allErrs = append(allErrs, r.validateAccessConfig(oldAWSManagedControlplane)...)
allErrs = append(allErrs, r.validateIAMAuthConfig()...)
allErrs = append(allErrs, r.validateSecondaryCIDR()...)
allErrs = append(allErrs, r.validateEKSAddons()...)
Expand Down Expand Up @@ -318,6 +319,28 @@ func validateEKSAddons(eksVersion *string, networkSpec infrav1.NetworkSpec, addo
return allErrs
}

func (r *AWSManagedControlPlane) validateAccessConfig(old *AWSManagedControlPlane) field.ErrorList {
var allErrs field.ErrorList

// If accessConfig is already set, do not allow removal of it.
if old.Spec.AccessConfig != nil && r.Spec.AccessConfig == nil {
allErrs = append(allErrs,
field.Invalid(field.NewPath("spec", "accessConfig"), r.Spec.AccessConfig, "removing AccessConfig is not allowed after it has been enabled"),
)
}

// AuthenticationMode is ratcheting - do not allow downgrades
if old.Spec.AccessConfig != nil && old.Spec.AccessConfig.AuthenticationMode != r.Spec.AccessConfig.AuthenticationMode &&
((old.Spec.AccessConfig.AuthenticationMode == EKSAuthenticationModeAPIAndConfigMap && r.Spec.AccessConfig.AuthenticationMode == EKSAuthenticationModeConfigMap) ||
old.Spec.AccessConfig.AuthenticationMode == EKSAuthenticationModeAPI) {
allErrs = append(allErrs,
field.Invalid(field.NewPath("spec", "accessConfig", "authenticationMode"), r.Spec.AccessConfig.AuthenticationMode, "downgrading authentication mode is not allowed after it has been enabled"),
)
}

return allErrs
}

func (r *AWSManagedControlPlane) validateIAMAuthConfig() field.ErrorList {
return validateIAMAuthConfig(r.Spec.IAMAuthenticatorConfig, field.NewPath("spec.iamAuthenticatorConfig"))
}
Expand Down
90 changes: 90 additions & 0 deletions controlplane/eks/api/v1beta2/awsmanagedcontrolplane_webhook_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -669,6 +669,96 @@ func TestWebhookUpdate(t *testing.T) {
},
expectError: false,
},
{
name: "no change in access config",
oldClusterSpec: AWSManagedControlPlaneSpec{
EKSClusterName: "default_cluster1",
AccessConfig: &AccessConfig{
AuthenticationMode: EKSAuthenticationModeConfigMap,
},
},
newClusterSpec: AWSManagedControlPlaneSpec{
EKSClusterName: "default_cluster1",
AccessConfig: &AccessConfig{
AuthenticationMode: EKSAuthenticationModeConfigMap,
},
},
expectError: false,
},
{
name: "change in access config to nil",
oldClusterSpec: AWSManagedControlPlaneSpec{
EKSClusterName: "default_cluster1",
AccessConfig: &AccessConfig{
AuthenticationMode: EKSAuthenticationModeConfigMap,
},
},
newClusterSpec: AWSManagedControlPlaneSpec{
EKSClusterName: "default_cluster1",
},
expectError: true,
},
{
name: "change in access config from nil to valid",
oldClusterSpec: AWSManagedControlPlaneSpec{
EKSClusterName: "default_cluster1",
},
newClusterSpec: AWSManagedControlPlaneSpec{
EKSClusterName: "default_cluster1",
AccessConfig: &AccessConfig{
AuthenticationMode: EKSAuthenticationModeConfigMap,
},
},
expectError: false,
},
{
name: "change in access config auth mode from ApiAndConfigMap to API is allowed",
oldClusterSpec: AWSManagedControlPlaneSpec{
EKSClusterName: "default_cluster1",
AccessConfig: &AccessConfig{
AuthenticationMode: EKSAuthenticationModeAPIAndConfigMap,
},
},
newClusterSpec: AWSManagedControlPlaneSpec{
EKSClusterName: "default_cluster1",
AccessConfig: &AccessConfig{
AuthenticationMode: EKSAuthenticationModeAPI,
},
},
expectError: false,
},
{
name: "change in access config auth mode from API to Config Map is denied",
oldClusterSpec: AWSManagedControlPlaneSpec{
EKSClusterName: "default_cluster1",
AccessConfig: &AccessConfig{
AuthenticationMode: EKSAuthenticationModeAPI,
},
},
newClusterSpec: AWSManagedControlPlaneSpec{
EKSClusterName: "default_cluster1",
AccessConfig: &AccessConfig{
AuthenticationMode: EKSAuthenticationModeConfigMap,
},
},
expectError: true,
},
{
name: "change in access config auth mode from APIAndConfigMap to Config Map is denied",
oldClusterSpec: AWSManagedControlPlaneSpec{
EKSClusterName: "default_cluster1",
AccessConfig: &AccessConfig{
AuthenticationMode: EKSAuthenticationModeAPIAndConfigMap,
},
},
newClusterSpec: AWSManagedControlPlaneSpec{
EKSClusterName: "default_cluster1",
AccessConfig: &AccessConfig{
AuthenticationMode: EKSAuthenticationModeConfigMap,
},
},
expectError: true,
},
{
name: "change in encryption config to nil",
oldClusterSpec: AWSManagedControlPlaneSpec{
Expand Down
15 changes: 15 additions & 0 deletions controlplane/eks/api/v1beta2/types.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -79,6 +79,21 @@ var (
EKSTokenMethodAWSCli = EKSTokenMethod("aws-cli")
)

// EKSAuthenticationMode defines the authentication mode for the cluster
type EKSAuthenticationMode string

var (
// EKSAuthenticationModeConfigMap indicates that only `aws-auth` ConfigMap will be used for authentication
EKSAuthenticationModeConfigMap = EKSAuthenticationMode("CONFIG_MAP")

// EKSAuthenticationModeAPI indicates that only AWS Access Entries will be used for authentication
EKSAuthenticationModeAPI = EKSAuthenticationMode("API")

// EKSAuthenticationModeAPIAndConfigMap indicates that both `aws-auth` ConfigMap and AWS Access Entries will
// be used for authentication
EKSAuthenticationModeAPIAndConfigMap = EKSAuthenticationMode("API_AND_CONFIG_MAP")
)

var (
// DefaultEKSControlPlaneRole is the name of the default IAM role to use for the EKS control plane
// if no other role is supplied in the spec and if iam role creation is not enabled. The default
Expand Down
20 changes: 20 additions & 0 deletions controlplane/eks/api/v1beta2/zz_generated.deepcopy.go
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

56 changes: 56 additions & 0 deletions pkg/cloud/services/eks/cluster.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -121,6 +121,10 @@ func (s *Service) reconcileCluster(ctx context.Context) error {
return errors.Wrap(err, "failed reconciling cluster config")
}

if err := s.reconcileAccessConfig(ctx, cluster.AccessConfig); err != nil {
return errors.Wrap(err, "failed reconciling access config")
}

if err := s.reconcileLogging(ctx, cluster.Logging); err != nil {
return errors.Wrap(err, "failed reconciling logging")
}
Expand Down Expand Up @@ -422,6 +426,13 @@ func (s *Service) createCluster(ctx context.Context, eksClusterName string) (*ek
return nil, errors.Wrap(err, "couldn't create vpc config for cluster")
}

var accessConfig *ekstypes.CreateAccessConfigRequest
if s.scope.ControlPlane.Spec.AccessConfig != nil && s.scope.ControlPlane.Spec.AccessConfig.AuthenticationMode != "" {
accessConfig = &ekstypes.CreateAccessConfigRequest{
AuthenticationMode: ekstypes.AuthenticationMode(string(s.scope.ControlPlane.Spec.AccessConfig.AuthenticationMode)),
}
}

var netConfig *ekstypes.KubernetesNetworkConfigRequest
if s.scope.VPC().IsIPv6Enabled() {
netConfig = &ekstypes.KubernetesNetworkConfigRequest{
Expand Down Expand Up @@ -465,6 +476,7 @@ func (s *Service) createCluster(ctx context.Context, eksClusterName string) (*ek
Name: aws.String(eksClusterName),
Version: eksVersion,
Logging: logging,
AccessConfig: accessConfig,
EncryptionConfig: encryptionConfigs,
ResourcesVpcConfig: vpcConfig,
RoleArn: role.Arn,
Expand Down Expand Up @@ -542,6 +554,50 @@ func (s *Service) reconcileClusterConfig(ctx context.Context, cluster *ekstypes.
return nil
}

func (s *Service) reconcileAccessConfig(ctx context.Context, accessConfig *ekstypes.AccessConfigResponse) error {
input := &eks.UpdateClusterConfigInput{Name: aws.String(s.scope.KubernetesClusterName())}

if s.scope.ControlPlane.Spec.AccessConfig == nil || s.scope.ControlPlane.Spec.AccessConfig.AuthenticationMode == "" {
return nil
}

expectedAuthenticationMode := ekstypes.AuthenticationMode(string(s.scope.ControlPlane.Spec.AccessConfig.AuthenticationMode))
s.scope.Debug("Reconciling EKS Access Config for cluster", "cluster-name", s.scope.KubernetesClusterName(), "expected", expectedAuthenticationMode, "current", accessConfig.AuthenticationMode)
if expectedAuthenticationMode != accessConfig.AuthenticationMode {
input.AccessConfig = &ekstypes.UpdateAccessConfigRequest{
AuthenticationMode: expectedAuthenticationMode,
}
}

if input.AccessConfig != nil {
if err := wait.WaitForWithRetryable(wait.NewBackoff(), func() (bool, error) {
if _, err := s.EKSClient.UpdateClusterConfig(ctx, input); err != nil {
return false, err
}

// Wait until status transitions to UPDATING because there's a short
// window after UpdateClusterConfig returns where the cluster
// status is ACTIVE and the update would be tried again
if err := s.EKSClient.WaitUntilClusterUpdating(
ctx,
&eks.DescribeClusterInput{Name: aws.String(s.scope.KubernetesClusterName())},
s.scope.MaxWaitActiveUpdateDelete,
); err != nil {
return false, err
}

conditions.MarkTrue(s.scope.ControlPlane, ekscontrolplanev1.EKSControlPlaneUpdatingCondition)
record.Eventf(s.scope.ControlPlane, "InitiatedUpdateEKSControlPlane", "Initiated auth config update for EKS control plane %s", s.scope.KubernetesClusterName())
return true, nil
}); err != nil {
record.Warnf(s.scope.ControlPlane, "FailedUpdateEKSControlPlane", "Failed to update EKS control plane auth config: %v", err)
return errors.Wrapf(err, "failed to update EKS cluster")
}
}

return nil
}

func (s *Service) reconcileLogging(ctx context.Context, logging *ekstypes.Logging) error {
input := &eks.UpdateClusterConfigInput{Name: aws.String(s.scope.KubernetesClusterName())}

Expand Down
Loading