Skip to content

Commit 93cc17c

Browse files
committed
s3 IAM
1 parent ed37e26 commit 93cc17c

File tree

2 files changed

+19
-4
lines changed
  • src/connections

2 files changed

+19
-4
lines changed
Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,5 @@
11
---
22
title: 'AWS S3 Destination'
33
hidden: true
4-
published: false
54
beta: true
65
---

src/connections/storage/catalog/aws-s3/index.md

Lines changed: 19 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,14 +1,20 @@
11
---
22
title: AWS S3 with IAM Role Support Destination
33
redirect_from:
4-
- '/connections/destinations/catalog/amazon-s3/'
4+
- '/connections/destinations/catalog/aws-s3/'
55
hide-personas-partial: true
66
---
77

88
> info "This document is about a destination which is in beta"
99
> This means that the AWS S3 with IAM Role Support destination is in active development, and some functionality may change before it becomes generally available.
1010
1111

12+
## Differences between the Amazon S3 destination and the AWS S3 destination
13+
14+
The AWS S3 destination provides a more secure method of connecting to your S3 buckets. It uses AWS's own IAM Roles to define access to the specified buckets. For more information about IAM Roles, see Amazon's [IAM role](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles.html){:target="_blank"} documentation.
15+
16+
Functionally, the two destinations (Amazon S3 and AWS S3 with IAM Role Support) copy data in a similar manner.
17+
1218
## Getting Started
1319

1420
The AWS S3 destination puts the raw logs of the data Segment receives into your S3 bucket, encrypted, no matter what region the bucket is in.
@@ -24,6 +30,8 @@ The Segment Tracking API processes data from your sources, and collects the Even
2430

2531
![](images/s3processdiagram.png)
2632

33+
34+
2735
## Create a new destination
2836

2937
Complete the following steps to configure the AWS S3 Destination with IAM Role Support.
@@ -97,7 +105,7 @@ If you have server-side encryption enabled, see the [required configuration](#en
97105

98106
To finish configuration, enable the AWS S3 Destination with IAM Role Support destination in your workspace.
99107

100-
1. Add the AWS S3 destination from the Data Storage section of the Destinations catalog.
108+
1. Add the **AWS S3** destination from the Raw Data section of the Destinations catalog. This document is about the **AWS S3** destination. For information about the **Amazon S3** destination, which does not include IAM Role support, see the documentation [here](/docs/connections/storage/catalog/amazon-s3/).
101109
2. Select the data source you'll connect to the destination.
102110
3. Provide a unique name for the destination.
103111
4. Complete the destination settings:
@@ -107,13 +115,21 @@ To finish configuration, enable the AWS S3 Destination with IAM Role Support des
107115
5. Enable the destination.
108116
6. Verify Segment data is stored in the S3 bucket by navigating to the `<your_S3_bucket>/segment-logs` in the AWS console. The bucket will take roughly 1 hour to begin receiving data.
109117

118+
> info ""
119+
> Did you know you can create destinations with the Config API? For more information, see [Create Destination](https://reference.segmentapis.com/#51d965d3-4a67-4542-ae2c-eb1fdddc3df6){:target="_blank"}.
120+
121+
110122
## Migrate an existing destination
123+
124+
> warning "Avoid overwriting data"
125+
> Sending data to the same S3 location from both the existing Amazon S3 destination, and the AWS S3 with IAM Role Support destination will overwrite data in that location. To avoid this, follow the steps below.
126+
111127
To migrate an existing Amazon S3 destination to the AWS S3 with IAM Role Support Destination:
112128

113129
1. Configure the IAM role and IAM policy permissions as described in steps 2 - 4 [above](#create-an-iam-role-in-aws).
114130
2. Add the AWS S3 with IAM Role Support Destination and add the AWS Region and IAM role ARN. For the bucket name, enter `<YOUR_BUCKET_NAME>/segment-logs/test`. Enable the destination, and verify data is received at `<YOUR_BUCKET_NAME>/segment-logs/test/segment-logs`. If the folder receives data, continue to the next step. If you don't see log entries, check the trust relationship document and IAM policy attached to the role.
115131
3. Update the bucket name in the new destination to `<YOUR_BUCKET_NAME>`.
116-
4. After 1 hour, disable the original Amazon S3 destination to avoid data duplication.
132+
4. After 1 hour, disable the original Amazon S3 destination.
117133
5. Verify that the `<YOUR_BUCKET_NAME>/segment-logs` receives data.
118134
6. Remove the test folder created in step 2 from the bucket.
119135

0 commit comments

Comments
 (0)