Skip to content

Feature - Directory Bucket Support (S3 Express) #187

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 9 commits into
base: main
Choose a base branch
from
Open

Feature - Directory Bucket Support (S3 Express) #187

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
6e7373d
Remove Ignored Fields, modify locationConstraint override and generat…
Itai-Melamed Oct 22, 2025
060d4ba
fix "type_" scheme override
Itai-Melamed Oct 22, 2025
be57139
modify management operations that are not supported for Directory buc…
Itai-Melamed Oct 23, 2025
41b0082
support tagging operations
Itai-Melamed Oct 23, 2025
eb542a7
fix attribution file mismatch check
Itai-Melamed Oct 29, 2025
80c851e
better validation for invalid Directory Bucket configuration
Itai-Melamed Oct 29, 2025
03dc43d
remove unnecessary operation override in generator.yaml (customFindBu…
Itai-Melamed Oct 29, 2025
b1dde85
regenerate code using s3-controller
Itai-Melamed Oct 29, 2025
0ba503e
Add unit test for IsDirectoryBucketName function
Itai-Melamed Oct 29, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1,882 changes: 1,144 additions & 738 deletions ATTRIBUTION.md
View file
Open in desktop

Large diffs are not rendered by default.

12 changes: 6 additions & 6 deletions apis/v1alpha1/ack-generate-metadata.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,13 +1,13 @@
ack_generate_info:
build_date: "2025-09-19T17:20:43Z"
build_hash: 6b4211163dcc34776b01da9a18217bac0f4103fd
go_version: go1.24.6
version: v0.52.0
api_directory_checksum: 2108338a86d704419192e545c0bfb433bab8c836
build_date: "2025-10-29T15:19:17Z"
build_hash: eaabefb6bd7b2be8a1baf4478f22b3310e6921c8
go_version: go1.25.3
version: v0.52.0-6-geaabefb-dirty
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did you need to modify the ackgenerate tool to make these changes? If not could you rebuild the controller with an unmodified version.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure what caused this change, I assumed the ack-generate tool updated some metadata and didn't think much of it. I don't think it is crucial to the implementation.

Could you maybe elaborate on how to know what the actual values should be?

api_directory_checksum: 0215f0e48eb84dd4b61045aee4ca703d5bf97304
api_version: v1alpha1
aws_sdk_go_version: v1.32.6
generator_config_info:
file_checksum: 00a96da29786586c7a3a59666d5546468e616b48
file_checksum: cd2274c70e78e904884fa8869c5b9fbacc4a0c53
original_file_name: generator.yaml
last_modification:
reason: API generation
10 changes: 8 additions & 2 deletions apis/v1alpha1/generator.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,11 +12,11 @@ ignore:
- "VersioningConfiguration.MFADelete"
# This subfield struct has no members...
- "NotificationConfiguration.EventBridgeConfiguration"
- CreateBucketInput.CreateBucketConfiguration.Location
- CreateBucketInput.CreateBucketConfiguration.Bucket
- LoggingEnabled.TargetObjectKeyFormat
resources:
Bucket:
find_operation:
custom_method_name: customFindBucket
fields:
Name:
is_primary_key: true
Expand Down Expand Up @@ -58,6 +58,12 @@ resources:
# Forcing CRD field to "uRI" to avoid breaking change following
# fix in aws-controller-k8s/pkg dependency for URI -> uri.
go_tag: json:"uRI,omitempty"
CreateBucketConfiguration.Bucket.Type:
# Override json tag to use "type" instead of "type_" to avoid underscore in CRD
go_tag: json:"type,omitempty"
CreateBucketConfiguration.Location.Type:
# Override json tag to use "type" instead of "type_" to avoid underscore in CRD
go_tag: json:"type,omitempty"
Metrics:
custom_field:
list_of: MetricsConfiguration
Expand Down
38 changes: 37 additions & 1 deletion apis/v1alpha1/types.go
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

60 changes: 60 additions & 0 deletions apis/v1alpha1/zz_generated.deepcopy.go
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

30 changes: 29 additions & 1 deletion config/crd/bases/s3.services.k8s.aws_buckets.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
annotations:
controller-gen.kubebuilder.io/version: v0.16.2
controller-gen.kubebuilder.io/version: v0.19.0
name: buckets.s3.services.k8s.aws
spec:
group: s3.services.k8s.aws
Expand Down Expand Up @@ -171,6 +171,34 @@ spec:
createBucketConfiguration:
description: The configuration information for the bucket.
properties:
bucket:
description: |-
Specifies the information about the bucket that will be created. For more
information about directory buckets, see Directory buckets (https://docs.aws.amazon.com/AmazonS3/latest/userguide/directory-buckets-overview.html)
in the Amazon S3 User Guide.

This functionality is only supported by directory buckets.
properties:
dataRedundancy:
type: string
type:
type: string
type: object
location:
description: |-
Specifies the location where the bucket will be created.

For directory buckets, the location type is Availability Zone or Local Zone.
For more information about directory buckets, see Directory buckets (https://docs.aws.amazon.com/AmazonS3/latest/userguide/directory-buckets-overview.html)
in the Amazon S3 User Guide.

This functionality is only supported by directory buckets.
properties:
name:
type: string
type:
type: string
type: object
locationConstraint:
type: string
type: object
Expand Down
10 changes: 8 additions & 2 deletions generator.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,11 +12,11 @@ ignore:
- "VersioningConfiguration.MFADelete"
# This subfield struct has no members...
- "NotificationConfiguration.EventBridgeConfiguration"
- CreateBucketInput.CreateBucketConfiguration.Location
- CreateBucketInput.CreateBucketConfiguration.Bucket
- LoggingEnabled.TargetObjectKeyFormat
resources:
Bucket:
find_operation:
custom_method_name: customFindBucket
fields:
Name:
is_primary_key: true
Expand Down Expand Up @@ -58,6 +58,12 @@ resources:
# Forcing CRD field to "uRI" to avoid breaking change following
# fix in aws-controller-k8s/pkg dependency for URI -> uri.
go_tag: json:"uRI,omitempty"
CreateBucketConfiguration.Bucket.Type:
# Override json tag to use "type" instead of "type_" to avoid underscore in CRD
go_tag: json:"type,omitempty"
CreateBucketConfiguration.Location.Type:
# Override json tag to use "type" instead of "type_" to avoid underscore in CRD
go_tag: json:"type,omitempty"
Metrics:
custom_field:
list_of: MetricsConfiguration
Expand Down
11 changes: 6 additions & 5 deletions go.mod
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,9 +7,10 @@ toolchain go1.24.1
require (
github.com/aws-controllers-k8s/runtime v0.52.0
github.com/aws/aws-sdk-go v1.49.0
github.com/aws/aws-sdk-go-v2 v1.34.0
github.com/aws/aws-sdk-go-v2 v1.39.3
github.com/aws/aws-sdk-go-v2/service/s3 v1.74.1
github.com/aws/smithy-go v1.22.2
github.com/aws/aws-sdk-go-v2/service/s3control v1.66.4
github.com/aws/smithy-go v1.23.1
github.com/go-logr/logr v1.4.2
github.com/pkg/errors v0.9.1
github.com/spf13/pflag v1.0.5
Expand All @@ -25,14 +26,14 @@ require (
github.com/aws/aws-sdk-go-v2/config v1.28.6 // indirect
github.com/aws/aws-sdk-go-v2/credentials v1.17.47 // indirect
github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.16.21 // indirect
github.com/aws/aws-sdk-go-v2/internal/configsources v1.3.29 // indirect
github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.6.29 // indirect
github.com/aws/aws-sdk-go-v2/internal/configsources v1.4.10 // indirect
github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.7.10 // indirect
github.com/aws/aws-sdk-go-v2/internal/ini v1.8.1 // indirect
github.com/aws/aws-sdk-go-v2/internal/v4a v1.3.29 // indirect
github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.12.2 // indirect
github.com/aws/aws-sdk-go-v2/service/internal/checksum v1.5.3 // indirect
github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.12.10 // indirect
github.com/aws/aws-sdk-go-v2/service/internal/s3shared v1.18.10 // indirect
github.com/aws/aws-sdk-go-v2/service/internal/s3shared v1.19.10 // indirect
github.com/aws/aws-sdk-go-v2/service/sso v1.24.7 // indirect
github.com/aws/aws-sdk-go-v2/service/ssooidc v1.28.6 // indirect
github.com/aws/aws-sdk-go-v2/service/sts v1.33.2 // indirect
Expand Down
22 changes: 12 additions & 10 deletions go.sum
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,8 @@ github.com/aws-controllers-k8s/runtime v0.52.0 h1:Q5UIAn6SSBr60t/DiU/zr6NLBlUuK2
github.com/aws-controllers-k8s/runtime v0.52.0/go.mod h1:OkUJN+Ds799JLYZsMJrO2vDJ4snxUeHK2MgrQHbU+Qc=
github.com/aws/aws-sdk-go v1.49.0 h1:g9BkW1fo9GqKfwg2+zCD+TW/D36Ux+vtfJ8guF4AYmY=
github.com/aws/aws-sdk-go v1.49.0/go.mod h1:LF8svs817+Nz+DmiMQKTO3ubZ/6IaTpq3TjupRn3Eqk=
github.com/aws/aws-sdk-go-v2 v1.34.0 h1:9iyL+cjifckRGEVpRKZP3eIxVlL06Qk1Tk13vreaVQU=
github.com/aws/aws-sdk-go-v2 v1.34.0/go.mod h1:JgstGg0JjWU1KpVJjD5H0y0yyAIpSdKEq556EI6yOOM=
github.com/aws/aws-sdk-go-v2 v1.39.3 h1:h7xSsanJ4EQJXG5iuW4UqgP7qBopLpj84mpkNx3wPjM=
github.com/aws/aws-sdk-go-v2 v1.39.3/go.mod h1:yWSxrnioGUZ4WVv9TgMrNUeLV3PFESn/v+6T/Su8gnM=
github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.6.8 h1:zAxi9p3wsZMIaVCdoiQp2uZ9k1LsZvmAnoTBeZPXom0=
github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.6.8/go.mod h1:3XkePX5dSaxveLAYY7nsbsZZrKxCyEuE5pM4ziFxyGg=
github.com/aws/aws-sdk-go-v2/config v1.28.6 h1:D89IKtGrs/I3QXOLNTH93NJYtDhm8SYa9Q5CsPShmyo=
Expand All @@ -12,10 +12,10 @@ github.com/aws/aws-sdk-go-v2/credentials v1.17.47 h1:48bA+3/fCdi2yAwVt+3COvmatZ6
github.com/aws/aws-sdk-go-v2/credentials v1.17.47/go.mod h1:+KdckOejLW3Ks3b0E3b5rHsr2f9yuORBum0WPnE5o5w=
github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.16.21 h1:AmoU1pziydclFT/xRV+xXE/Vb8fttJCLRPv8oAkprc0=
github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.16.21/go.mod h1:AjUdLYe4Tgs6kpH4Bv7uMZo7pottoyHMn4eTcIcneaY=
github.com/aws/aws-sdk-go-v2/internal/configsources v1.3.29 h1:Ej0Rf3GMv50Qh4G4852j2djtoDb7AzQ7MuQeFHa3D70=
github.com/aws/aws-sdk-go-v2/internal/configsources v1.3.29/go.mod h1:oeNTC7PwJNoM5AznVr23wxhLnuJv0ZDe5v7w0wqIs9M=
github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.6.29 h1:6e8a71X+9GfghragVevC5bZqvATtc3mAMgxpSNbgzF0=
github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.6.29/go.mod h1:c4jkZiQ+BWpNqq7VtrxjwISrLrt/VvPq3XiopkUIolI=
github.com/aws/aws-sdk-go-v2/internal/configsources v1.4.10 h1:mj/bdWleWEh81DtpdHKkw41IrS+r3uw1J/VQtbwYYp8=
github.com/aws/aws-sdk-go-v2/internal/configsources v1.4.10/go.mod h1:7+oEMxAZWP8gZCyjcm9VicI0M61Sx4DJtcGfKYv2yKQ=
github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.7.10 h1:wh+/mn57yhUrFtLIxyFPh2RgxgQz/u+Yrf7hiHGHqKY=
github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.7.10/go.mod h1:7zirD+ryp5gitJJ2m1BBux56ai8RIRDykXZrJSp540w=
github.com/aws/aws-sdk-go-v2/internal/ini v1.8.1 h1:VaRN3TlFdd6KxX1x3ILT5ynH6HvKgqdiXoTxAF4HQcQ=
github.com/aws/aws-sdk-go-v2/internal/ini v1.8.1/go.mod h1:FbtygfRFze9usAadmnGJNc8KsP346kEe+y2/oyhGAGc=
github.com/aws/aws-sdk-go-v2/internal/v4a v1.3.29 h1:g9OUETuxA8i/Www5Cby0R3WSTe7ppFTZXHVLNskNS4w=
Expand All @@ -26,18 +26,20 @@ github.com/aws/aws-sdk-go-v2/service/internal/checksum v1.5.3 h1:EP1ITDgYVPM2dL1
github.com/aws/aws-sdk-go-v2/service/internal/checksum v1.5.3/go.mod h1:5lWNWeAgWenJ/BZ/CP9k9DjLbC0pjnM045WjXRPPi14=
github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.12.10 h1:hN4yJBGswmFTOVYqmbz1GBs9ZMtQe8SrYxPwrkrlRv8=
github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.12.10/go.mod h1:TsxON4fEZXyrKY+D+3d2gSTyJkGORexIYab9PTf56DA=
github.com/aws/aws-sdk-go-v2/service/internal/s3shared v1.18.10 h1:fXoWC2gi7tdJYNTPnnlSGzEVwewUchOi8xVq/dkg8Qs=
github.com/aws/aws-sdk-go-v2/service/internal/s3shared v1.18.10/go.mod h1:cvzBApD5dVazHU8C2rbBQzzzsKc8m5+wNJ9mCRZLKPc=
github.com/aws/aws-sdk-go-v2/service/internal/s3shared v1.19.10 h1:DA+Hl5adieRyFvE7pCvBWm3VOZTRexGVkXw33SUqNoY=
github.com/aws/aws-sdk-go-v2/service/internal/s3shared v1.19.10/go.mod h1:L+A89dH3/gr8L4ecrdzuXUYd1znoko6myzndVGZx/DA=
github.com/aws/aws-sdk-go-v2/service/s3 v1.74.1 h1:9LawY3cDJ3HE+v2GMd5SOkNLDwgN4K7TsCjyVBYu/L4=
github.com/aws/aws-sdk-go-v2/service/s3 v1.74.1/go.mod h1:hHnELVnIHltd8EOF3YzahVX6F6y2C6dNqpRj1IMkS5I=
github.com/aws/aws-sdk-go-v2/service/s3control v1.66.4 h1:crP9qsvLlgtGDH8/6Rm8IbyApuP0b9EblY2ZEM47Enw=
github.com/aws/aws-sdk-go-v2/service/s3control v1.66.4/go.mod h1:ghGid4gQ8PYl3B8ip/b25+4yTHzvBa1GvFJ/r/i2CI4=
github.com/aws/aws-sdk-go-v2/service/sso v1.24.7 h1:rLnYAfXQ3YAccocshIH5mzNNwZBkBo+bP6EhIxak6Hw=
github.com/aws/aws-sdk-go-v2/service/sso v1.24.7/go.mod h1:ZHtuQJ6t9A/+YDuxOLnbryAmITtr8UysSny3qcyvJTc=
github.com/aws/aws-sdk-go-v2/service/ssooidc v1.28.6 h1:JnhTZR3PiYDNKlXy50/pNeix9aGMo6lLpXwJ1mw8MD4=
github.com/aws/aws-sdk-go-v2/service/ssooidc v1.28.6/go.mod h1:URronUEGfXZN1VpdktPSD1EkAL9mfrV+2F4sjH38qOY=
github.com/aws/aws-sdk-go-v2/service/sts v1.33.2 h1:s4074ZO1Hk8qv65GqNXqDjmkf4HSQqJukaLuuW0TpDA=
github.com/aws/aws-sdk-go-v2/service/sts v1.33.2/go.mod h1:mVggCnIWoM09jP71Wh+ea7+5gAp53q+49wDFs1SW5z8=
github.com/aws/smithy-go v1.22.2 h1:6D9hW43xKFrRx/tXXfAlIZc4JI+yQe6snnWcQyxSyLQ=
github.com/aws/smithy-go v1.22.2/go.mod h1:irrKGvNn1InZwb2d7fkIRNucdfwR8R+Ts3wxYa/cJHg=
github.com/aws/smithy-go v1.23.1 h1:sLvcH6dfAFwGkHLZ7dGiYF7aK6mg4CgKA/iDKjLDt9M=
github.com/aws/smithy-go v1.23.1/go.mod h1:LEj2LM3rBRQJxPZTB4KuzZkaZYnZPnvgIhb4pu07mx0=
github.com/beorn7/perks v1.0.1 h1:VlbKKnNfV8bJzeqoa4cOKqO6bYr3WgKZxO8Z16+hsOM=
github.com/beorn7/perks v1.0.1/go.mod h1:G2ZrVWU2WbWT9wwq4/hrbKbnv/1ERSJQ0ibhJ6rlkpw=
github.com/cenkalti/backoff/v4 v4.3.0 h1:MyRJ/UdXutAwSAT+s3wNd7MfTIcy71VQueUuFK343L8=
Expand Down
30 changes: 29 additions & 1 deletion helm/crds/s3.services.k8s.aws_buckets.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
annotations:
controller-gen.kubebuilder.io/version: v0.16.2
controller-gen.kubebuilder.io/version: v0.19.0
name: buckets.s3.services.k8s.aws
spec:
group: s3.services.k8s.aws
Expand Down Expand Up @@ -171,6 +171,34 @@ spec:
createBucketConfiguration:
description: The configuration information for the bucket.
properties:
bucket:
description: |-
Specifies the information about the bucket that will be created. For more
information about directory buckets, see Directory buckets (https://docs.aws.amazon.com/AmazonS3/latest/userguide/directory-buckets-overview.html)
in the Amazon S3 User Guide.

This functionality is only supported by directory buckets.
properties:
dataRedundancy:
type: string
type:
type: string
type: object
location:
description: |-
Specifies the location where the bucket will be created.

For directory buckets, the location type is Availability Zone or Local Zone.
For more information about directory buckets, see Directory buckets (https://docs.aws.amazon.com/AmazonS3/latest/userguide/directory-buckets-overview.html)
in the Amazon S3 User Guide.

This functionality is only supported by directory buckets.
properties:
name:
type: string
type:
type: string
type: object
locationConstraint:
type: string
type: object
Expand Down
2 changes: 1 addition & 1 deletion helm/crds/services.k8s.aws_adoptedresources.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
annotations:
controller-gen.kubebuilder.io/version: v0.16.2
controller-gen.kubebuilder.io/version: v0.19.0
name: adoptedresources.services.k8s.aws
spec:
group: services.k8s.aws
Expand Down
Loading