Duplicate Grantee returned from GetBucketAcl

[bwoznicki]

Confirm by changing [ ] to [x] below:

• I've gone though Developer Guide and API reference
• I've checked AWS Forums and StackOverflow for answers
• [ ]
  Hi while working with aws-sdk-go v2 I have noticed that GetBucketAcl is returning array of duplicate Grants where Grantee type/displayname/emailaddress/id/uri/permission is identical. Is there a reason for this ? In the console I can see only one Grant which is current account. (some fields redacted)

example:

Getting bucket acl for:  <some_bucket>
{
  "Grants": [
    {
      "Grantee": {
        "Type": "CanonicalUser",
        "DisplayName": null,
        "EmailAddress": null,
        "ID": "0b1_____91b",
        "URI": null
      },
      "Permission": "FULL_CONTROL"
    },
    {
      "Grantee": {
        "Type": "CanonicalUser",
        "DisplayName": null,
        "EmailAddress": null,
        "ID": "0b1_____91b",
        "URI": null
      },
      "Permission": "FULL_CONTROL"
    },
    {
      "Grantee": {
        "Type": "CanonicalUser",
        "DisplayName": null,
        "EmailAddress": null,
        "ID": "0b1_____91b",
        "URI": null
      },
      "Permission": "FULL_CONTROL"
    },
    {
      "Grantee": {
        "Type": "CanonicalUser",
        "DisplayName": null,
        "EmailAddress": null,
        "ID": "0b1_____91b",
        "URI": null
      },
      "Permission": "FULL_CONTROL"
    },
    {
      "Grantee": {
        "Type": "CanonicalUser",
        "DisplayName": null,
        "EmailAddress": null,
        "ID": "0b1_____91b",
        "URI": null
      },
      "Permission": "FULL_CONTROL"
    }
  ],
  "Owner": {
    "DisplayName": null,
    "ID": "0b1_____91b"
  },
  "ResultMetadata": {}
}

[bwoznicki]

Sorry wrong repo, cant this be transferred to aws-sdk-go-v2

[jasdel]

Thanks for reaching out @bwoznicki. I'm not able to reproduce this issue on my side. Could you enable logging in your client to log the Response HTTP message with the body? You can enable logging per operation call to simplify the logs that are being produced.

	resp, err := client.GetBucketAcl(context.TODO(), &s3.GetBucketAclInput{
		Bucket: &bucket,
	}, func(o *s3.Options) {
                // Add logging of response message and body.
		o.ClientLogMode = aws.LogResponseWithBody
	})

The following is the test app I used to try to reproduce the issue.

package main

import (
	"context"
	"flag"
	"log"

	"github.com/aws/aws-sdk-go-v2/aws"
	"github.com/aws/aws-sdk-go-v2/config"
	"github.com/aws/aws-sdk-go-v2/service/s3"
	"github.com/davecgh/go-spew/spew"
)

func main() {
	var bucket string
	flag.StringVar(&bucket, "b", "", "bucket name")
	flag.Parse()

	cfg, err := config.LoadDefaultConfig(context.TODO())
	if err != nil {
		log.Fatalf("failed to load config, %v", err)
	}

	client := s3.NewFromConfig(cfg)

	resp, err := client.GetBucketAcl(context.TODO(), &s3.GetBucketAclInput{
		Bucket: &bucket,
	}, func(o *s3.Options) {
		o.ClientLogMode = aws.LogResponseWithBody
	})
	if err != nil {
		log.Fatalf("failed to get bucket acl, %v", err)
	}

	spew.Dump(resp.Grants)
}

[bwoznicki]

Hi @jasdel please find debug output bellow, it does not happen on all the buckets just some of them.

SDK 2021/03/04 10:20:37 DEBUG Response
HTTP/1.1 200 OK
Transfer-Encoding: chunked
Content-Type: application/xml
Date: Thu, 04 Mar 2021 10:20:38 GMT
Server: AmazonS3
X-Amz-Id-2: 7toLbalSiqnbE0whkvX+gMgk0dKirLsPqSoHnMlj4sz2itcPnF1GdcBNBd8OEbMQvC7WcUr/ygs=
X-Amz-Request-Id: 3D1EA7AC17200483

55c
<?xml version="1.0" encoding="UTF-8"?>
<AccessControlPolicy xmlns="http://s3.amazonaws.com/doc/2006-03-01/"><Owner><ID>0b1a4cafc06fcd0753d2eff0dd400b81fc190018ec3a79dd4ffd24b2d4ee891b</ID></Owner><AccessControlList><Grant><Grantee xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:type="CanonicalUser"><ID>0b1a4cafc06fcd0753d2eff0dd400b81fc190018ec3a79dd4ffd24b2d4ee891b</ID></Grantee><Permission>FULL_CONTROL</Permission></Grant><Grant><Grantee xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:type="CanonicalUser"><ID>0b1a4cafc06fcd0753d2eff0dd400b81fc190018ec3a79dd4ffd24b2d4ee891b</ID></Grantee><Permission>FULL_CONTROL</Permission></Grant><Grant><Grantee xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:type="CanonicalUser"><ID>0b1a4cafc06fcd0753d2eff0dd400b81fc190018ec3a79dd4ffd24b2d4ee891b</ID></Grantee><Permission>FULL_CONTROL</Permission></Grant><Grant><Grantee xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:type="CanonicalUser"><ID>0b1a4cafc06fcd0753d2eff0dd400b81fc190018ec3a79dd4ffd24b2d4ee891b</ID></Grantee><Permission>FULL_CONTROL</Permission></Grant><Grant><Grantee xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:type="CanonicalUser"><ID>0b1a4cafc06fcd0753d2eff0dd400b81fc190018ec3a79dd4ffd24b2d4ee891b</ID></Grantee><Permission>FULL_CONTROL</Permission></Grant></AccessControlList></AccessControlPolicy>
0

([]types.Grant) (len=5 cap=8) {
 (types.Grant) {
  Grantee: (*types.Grantee)(0xc0002de240)({
   Type: (types.Type) (len=13) "CanonicalUser",
   DisplayName: (*string)(<nil>),
   EmailAddress: (*string)(<nil>),
   ID: (*string)(0xc0002ce0d0)((len=64) "0b1a4cafc06fcd0753d2eff0dd400b81fc190018ec3a79dd4ffd24b2d4ee891b"),
   URI: (*string)(<nil>)
  }),
  Permission: (types.Permission) (len=12) "FULL_CONTROL"
 },
 (types.Grant) {
  Grantee: (*types.Grantee)(0xc0002de2a0)({
   Type: (types.Type) (len=13) "CanonicalUser",
   DisplayName: (*string)(<nil>),
   EmailAddress: (*string)(<nil>),
   ID: (*string)(0xc0002ce0e0)((len=64) "0b1a4cafc06fcd0753d2eff0dd400b81fc190018ec3a79dd4ffd24b2d4ee891b"),
   URI: (*string)(<nil>)
  }),
  Permission: (types.Permission) (len=12) "FULL_CONTROL"
 },
 (types.Grant) {
  Grantee: (*types.Grantee)(0xc0002de330)({
   Type: (types.Type) (len=13) "CanonicalUser",
   DisplayName: (*string)(<nil>),
   EmailAddress: (*string)(<nil>),
   ID: (*string)(0xc0002ce0f0)((len=64) "0b1a4cafc06fcd0753d2eff0dd400b81fc190018ec3a79dd4ffd24b2d4ee891b"),
   URI: (*string)(<nil>)
  }),
  Permission: (types.Permission) (len=12) "FULL_CONTROL"
 },
 (types.Grant) {
  Grantee: (*types.Grantee)(0xc0002de390)({
   Type: (types.Type) (len=13) "CanonicalUser",
   DisplayName: (*string)(<nil>),
   EmailAddress: (*string)(<nil>),
   ID: (*string)(0xc0002ce100)((len=64) "0b1a4cafc06fcd0753d2eff0dd400b81fc190018ec3a79dd4ffd24b2d4ee891b"),
   URI: (*string)(<nil>)
  }),
  Permission: (types.Permission) (len=12) "FULL_CONTROL"
 },
 (types.Grant) {
  Grantee: (*types.Grantee)(0xc0002de3f0)({
   Type: (types.Type) (len=13) "CanonicalUser",
   DisplayName: (*string)(<nil>),
   EmailAddress: (*string)(<nil>),
   ID: (*string)(0xc0002ce110)((len=64) "0b1a4cafc06fcd0753d2eff0dd400b81fc190018ec3a79dd4ffd24b2d4ee891b"),
   URI: (*string)(<nil>)
  }),
  Permission: (types.Permission) (len=12) "FULL_CONTROL"
 }
}

[jasdel]

Thanks for the additional information. From the XML document coming back in the response, it looks like the duplication is on the S3 bucket. Do you see multiple ACL policies on the bucket in the AWS Console? If you don't see multiple entries in the console I suggest reaching out to AWS support about the issue.

<AccessControlPolicy xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
  <Owner><ID>0b1a4cafc06fcd0753d2eff0dd400b81fc190018ec3a79dd4ffd24b2d4ee891b</ID></Owner>
  <AccessControlList>
    <Grant>
      <Grantee xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:type="CanonicalUser">
        <ID>0b1a4cafc06fcd0753d2eff0dd400b81fc190018ec3a79dd4ffd24b2d4ee891b</ID>
      </Grantee>
      <Permission>FULL_CONTROL</Permission>
    </Grant>
    <Grant>
      <Grantee xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:type="CanonicalUser">
        <ID>0b1a4cafc06fcd0753d2eff0dd400b81fc190018ec3a79dd4ffd24b2d4ee891b</ID>
      </Grantee>
      <Permission>FULL_CONTROL</Permission>
    </Grant>
    <Grant>
      <Grantee xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:type="CanonicalUser">
        <ID>0b1a4cafc06fcd0753d2eff0dd400b81fc190018ec3a79dd4ffd24b2d4ee891b</ID>
      </Grantee>
      <Permission>FULL_CONTROL</Permission>
    </Grant>
    <Grant>
      <Grantee xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:type="CanonicalUser">
        <ID>0b1a4cafc06fcd0753d2eff0dd400b81fc190018ec3a79dd4ffd24b2d4ee891b</ID>
      </Grantee>
      <Permission>FULL_CONTROL</Permission>
    </Grant>
    <Grant>
      <Grantee xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:type="CanonicalUser">
        <ID>0b1a4cafc06fcd0753d2eff0dd400b81fc190018ec3a79dd4ffd24b2d4ee891b</ID>
      </Grantee>
      <Permission>FULL_CONTROL</Permission>
    </Grant>
  </AccessControlList>
</AccessControlPolicy>

[bwoznicki]

Hi, yes console shows only one, which is the bucket owner, will reach out to the support see what they say...

[bwoznicki]

Hi @jasdel so I had a response from S3 support. At the moment it is possible to create duplicate entries for ACL as there is no dedupe on PUT/GET. However the team is also actively working on changing this behaviour on S3 so this should be resolved soon.

[leo-labs]

I just ran into the same issue: A duplicate grant of the default grant was present. Because of that I could not apply the bucket owner enforced setting for Object Ownership from the console. What helped was to reset ACLs with aws s3api put-bucket-acl --bucket DOC-EXAMPLE-BUCKET --acl private.