Skip to content

Commit 81a6723

Browse files
Paul-CornellPaul-Cornell
authored
S3 connectors: bucket setup how-to video, copyable access policy (#254)
1 parent 562b11f commit 81a6723

File tree

8 files changed

+115
-9
lines changed

8 files changed

+115
-9
lines changed

‎api-reference/ingest/destination-connector/s3.mdx‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -24,6 +24,12 @@ import S3APIPyV1 from '/snippets/destination_connectors/s3.v1.py.mdx';
2424
<S3APIPyV1 />
2525
</CodeGroup>
2626

27+
## Add an access policy to an existing bucket
28+
29+
import S3BucketPolicy from '/snippets/general-shared-text/s3-bucket-policy.mdx';
30+
31+
<S3BucketPolicy />
32+
2733
## Create a bucket with the AWS CLI
2834

2935
import S3BucketCLI from '/snippets/general-shared-text/s3-cli-setup.mdx';

‎api-reference/ingest/source-connectors/s3.mdx‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -36,6 +36,12 @@ import S3APIPyV1 from '/snippets/source_connectors/s3.v1.py.mdx';
3636
<S3APIPyV1 />
3737
</CodeGroup>
3838

39+
## Add an access policy to an existing bucket
40+
41+
import S3BucketPolicy from '/snippets/general-shared-text/s3-bucket-policy.mdx';
42+
43+
<S3BucketPolicy />
44+
3945
## Create a bucket with the AWS CLI
4046

4147
import S3BucketCLI from '/snippets/general-shared-text/s3-cli-setup.mdx';

‎open-source/ingest/destination-connectors/s3.mdx‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -28,6 +28,12 @@ import SharedPartitionByAPIOSS from '/snippets/ingest-configuration-shared/parti
2828

2929
<SharedPartitionByAPIOSS/>
3030

31+
## Add an access policy to an existing bucket
32+
33+
import S3BucketPolicy from '/snippets/general-shared-text/s3-bucket-policy.mdx';
34+
35+
<S3BucketPolicy />
36+
3137
## Create a bucket with the AWS CLI
3238

3339
import S3BucketCLI from '/snippets/general-shared-text/s3-cli-setup.mdx';

‎open-source/ingest/source-connectors/s3.mdx‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -40,6 +40,12 @@ import SharedPartitionByAPIOSS from '/snippets/ingest-configuration-shared/parti
4040

4141
<SharedPartitionByAPIOSS/>
4242

43+
## Add an access policy to an existing bucket
44+
45+
import S3BucketPolicy from '/snippets/general-shared-text/s3-bucket-policy.mdx';
46+
47+
<S3BucketPolicy />
48+
4349
## Create a bucket with the AWS CLI
4450

4551
import S3BucketCLI from '/snippets/general-shared-text/s3-cli-setup.mdx';

‎platform/destinations/s3.mdx‎

Lines changed: 11 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -23,14 +23,21 @@ import S3Fields from '/snippets/general-shared-text/s3-platform.mdx';
2323

2424
<S3Fields />
2525

26-
## Create a bucket with the AWS CLI
26+
## Add an access policy to an existing bucket
2727

28-
import S3BucketCLI from '/snippets/general-shared-text/s3-cli-setup.mdx';
28+
import S3BucketPolicy from '/snippets/general-shared-text/s3-bucket-policy.mdx';
2929

30-
<S3BucketCLI />
30+
<S3BucketPolicy />
3131

3232
## Create a bucket with AWS CloudFormation
3333

3434
import S3BucketCloudFormation from '/snippets/general-shared-text/s3-cf-setup.mdx';
3535

36-
<S3BucketCloudFormation />
36+
<S3BucketCloudFormation />
37+
38+
## Create a bucket with the AWS CLI
39+
40+
import S3BucketCLI from '/snippets/general-shared-text/s3-cli-setup.mdx';
41+
42+
<S3BucketCLI />
43+

‎platform/sources/s3.mdx‎

Lines changed: 10 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -23,14 +23,20 @@ import S3Fields from '/snippets/general-shared-text/s3-platform.mdx';
2323

2424
<S3Fields />
2525

26-
## Create a bucket with the AWS CLI
26+
## Add an access policy to an existing bucket
2727

28-
import S3BucketCLI from '/snippets/general-shared-text/s3-cli-setup.mdx';
28+
import S3BucketPolicy from '/snippets/general-shared-text/s3-bucket-policy.mdx';
2929

30-
<S3BucketCLI />
30+
<S3BucketPolicy />
3131

3232
## Create a bucket with AWS CloudFormation
3333

3434
import S3BucketCloudFormation from '/snippets/general-shared-text/s3-cf-setup.mdx';
3535

36-
<S3BucketCloudFormation />
36+
<S3BucketCloudFormation />
37+
38+
## Create a bucket with the AWS CLI
39+
40+
import S3BucketCLI from '/snippets/general-shared-text/s3-cli-setup.mdx';
41+
42+
<S3BucketCLI />

‎snippets/general-shared-text/s3-bucket-policy.mdx‎

Lines changed: 53 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,53 @@
1+
To use the Amazon S3 console to add an access policy that allows all authenticated AWS IAM users in the
2+
corresponding AWS account to read and write to an existing S3 bucket, do the following.
3+
4+
<Info>Your organization might have stricter bucket policy requirements. Check with your AWS account
5+
administrator if you are unsure.</Info>
6+
7+
1. Sign in to the [AWS Management Console](https://console.aws.amazon.com/).
8+
2. Open the [Amazon S3 Console](https://console.aws.amazon.com/s3/home).
9+
3. Browse to the existing bucket and open it.
10+
4. Click the **Permissions** tab.
11+
5. In the **Bucket policy** area, click **Edit**.
12+
6. In the **Policy** text area, copy the following JSON-formatted policy.
13+
To change the following policy to restrict it to a specific user in the AWS account, change `root` to that
14+
specific username.
15+
16+
In this policy, replace the following:
17+
18+
- Replace `<my-account-id>` with your AWS account ID.
19+
- Replace `<my-bucket-name>` in two places with the name of your bucket.
20+
21+
```json
22+
{
23+
"Version": "2012-10-17",
24+
"Statement": [
25+
{
26+
"Sid": "AllowAuthenticatedUsersInAccountReadWrite",
27+
"Effect": "Allow",
28+
"Principal": {
29+
"AWS": "arn:aws:iam::<my-account-id>:root"
30+
},
31+
"Action": [
32+
"s3:GetObject",
33+
"s3:PutObject",
34+
"s3:ListBucket",
35+
"s3:DeleteObject"
36+
],
37+
"Resource": [
38+
"arn:aws:s3:::<my-bucket-name>",
39+
"arn:aws:s3:::<my-bucket-name>/*"
40+
],
41+
"Condition": {
42+
"StringEquals": {
43+
"aws:PrincipalType": "IAMUser"
44+
}
45+
}
46+
}
47+
]
48+
}
49+
```
50+
51+
7. Click **Save changes**.
52+
53+

‎snippets/general-shared-text/s3.mdx‎

Lines changed: 17 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,21 @@
11
The Amazon S3 prerequisites:
22

3+
The following video shows how to fulfill the minimum set of S3 prerequisites:
4+
5+
<iframe
6+
width="560"
7+
height="315"
8+
src="https://www.youtube.com/embed/hFTzMR12_A4"
9+
title="YouTube video player"
10+
frameborder="0"
11+
allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture"
12+
allowfullscreen
13+
></iframe>
14+
15+
The preceding video does not show how to create an AWS account; enable anonymous access to the bucket (which is supported but
16+
not recommended); or generate an AWS STS session token for temporary access, if required by your organization's security
17+
requirements. For more information about prerequisites, see the following:
18+
319
- An AWS account. [Create an AWS account](https://aws.amazon.com/free).
420

521
<iframe
@@ -25,7 +41,7 @@ The Amazon S3 prerequisites:
2541
allowfullscreen
2642
></iframe>
2743

28-
- Anonymous or authenticated access to the bucket.
44+
- Anonymous (supported but not recommended) or authenticated access to the bucket.
2945

3046
- [Enable anonymous bucket access](https://docs.aws.amazon.com/AmazonS3/latest/userguide/example-bucket-policies.html#example-bucket-policies-anonymous-user).
3147

0 commit comments

Comments
 (0)