You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: src/connections/reverse-etl/index.md
+19-26Lines changed: 19 additions & 26 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -5,20 +5,15 @@ redirect_from:
5
5
- '/reverse-etl/'
6
6
---
7
7
8
-
Reverse ETL (Extract, Transform, Load) extracts data from a data warehouse using a query you provide, and delivers the data to your 3rd party destinations. Reverse ETL allows you to connect your data warehouse to the tools that Marketing, Sales, Support, Product, Analytics, and other business teams use. For example, with Reverse ETL, you can sync records from Snowflake to Salesforce. Reverse ETL supports event and object data. This includes customer profile data, subscriptions, product tables, shopping cart tables, and more.
9
-
10
-
As Segment is actively developing this feature, Segment welcomes your feedback on your experience with Reverse ETL. Click the button below to submit your feedback.
11
-
12
-
{% include components/button-fill.html modifier="expand" text="Submit feedback" href=" https://airtable.com/shriQgvkRpBCDN955" %}
8
+
Reverse ETL (Extract, Transform, Load) extracts data from a data warehouse using a query you provide, and syncs the data to your 3rd party destinations. For example, with Reverse ETL, you can sync records from Snowflake to Mixpanel. Reverse ETL supports event and object data. This includes customer profile data, subscriptions, product tables, shopping cart tables, and more.
13
9
14
10
## Example use cases
15
11
Use Reverse ETL when you want to:
16
-
* Sync lead scores created in the warehouse to Salesforce to customize interactions with prospects and optimize sales opportunities.
17
12
* Sync audiences and other data built in the warehouse to Braze, Iterable, Hubspot, or Salesforce Marketing Cloud for personalized marketing campaigns.
18
-
* Connect Google Sheets to a view in the warehouse for other business teams to have access to up-to-date reports.
19
-
* Sync enriched data to Mixpanel for a more complete view.
13
+
* Sync enriched data to Mixpanel for a more complete view of the customer, or enrich Segment Profiles with data from the warehouse.
20
14
* Send data in the warehouse back into Segment as events that can be activated in all supported destinations, including Twilio Engage and other platforms.
21
15
* Pass offline or enriched data to conversion APIs like Facebook, Google Ads, TikTok, or Snapchat.
16
+
* Connect Google Sheets to a view in the warehouse for other business teams to have access to up-to-date reports.
22
17
23
18
## Getting started
24
19
There are four components to Reverse ETL: Sources, Models, Destinations, and Mappings.
@@ -37,9 +32,6 @@ Follow these 4 steps to set up Reverse ETL and learn what each component is abou
37
32
### Step 1: Add a source
38
33
A source is where your data originates from. Traditionally in Segment, a [source](/docs/connections/sources/#what-is-a-source) is a website, server library, mobile SDK, or cloud application which can send data into Segment. In Reverse ETL, your data warehouse is the source.
39
34
40
-
> info ""
41
-
> Reverse ETL supports [these sources] and Segment is actively working on adding more. If you'd like to request Segment to add a particular source, please note it on the [feedback form](https://airtable.com/shriQgvkRpBCDN955){:target="_blank"}.
42
-
43
35
To add your warehouse as a source:
44
36
45
37
> warning ""
@@ -96,16 +88,17 @@ To add your first destination:
96
88
1. Navigate to **Connections > Destinations** and select the **Reverse ETL** tab.
97
89
2. Click **Add Reverse ETL destination**.
98
90
3. Select the destination you want to connect to and click **Configure**.
99
-
4. Select the source you want to connect the destination to.
91
+
4. Select the Reverse ETL source you want to connect the destination to.
100
92
5. Enter the **Destination name** and click **Create Destination**.
101
93
6. Enter the required information on the **Settings** tab of the destination.
94
+
7. Enable the destination.
102
95
103
96
### Step 4: Create mappings
104
97
After you’ve added a destination, you can create mappings from your warehouse to the destination. Mappings enable you to map the data you extract from your warehouse to the fields in your destination.
105
98
106
99
To create a mapping:
107
100
1. Navigate to **Conections > Destinations** and select the **Reverse ETL** tab.
108
-
2. Select the destination that you want to add a mapping to.
101
+
2. Select the destination that you want to create a mapping for.
109
102
3. Click **Add Mapping**.
110
103
2. Select the model to sync from.
111
104
3. Select the **Action** you want to sync and click **Next**.
@@ -120,25 +113,27 @@ To create a mapping:
120
113
* You map the fields that come from your source, to fields that the destination expects to find. Fields on the destination side depend on the type of action selected.
121
114
7. Click **Create Mapping**.
122
115
8. Select the destination you’d like to enable on the **My Destinations** page under **Reverse ETL > Destinations**.
123
-
9. Turn the toggle on for the **Mapping State** to enable the destination. Events that match the trigger condition in the mapping will be sent to the destination.
116
+
9. Turn the toggle on for the **Mapping Status**. Events that match the trigger condition in the mapping will be sent to the destination.
124
117
* If you disable the mapping state to the destination, events that match the trigger condition in the mapping won’t be sent to the destination.
125
118
126
119
To add multiple mappings from your warehouse to your destination, repeat steps 1-9 above.
127
120
128
121
## Using Reverse ETL
129
122
After you've followed [all four steps](/docs/connections/reverse-etl/#getting-started) and set up your source, model, destination, and mappings for Reverse ETL, your data will extract and sync to your destination(s) right away if you chose an interval schedule. If you set your data to extract at a specific day and time, the extraction will take place then.
130
123
131
-
### Sync status and observability
124
+
### Sync history and observability
132
125
Check the status of your data extractions and see details of your syncs. Click into failed records to view additional details on the error, sample payloads to help you debug the issue, and recommended actions.
133
126
134
127
To check the status of your extractions:
135
-
1. Navigate to **Reverse ETL > Destinations**.
128
+
1. Navigate to **Connections > Destinations** and select the **Reverse ETL** tab.
136
129
2. Select the destination you want to view.
137
130
3. Select the mapping you want to view.
138
131
4. Click the sync you want to view to get details of the sync. You can view:
139
-
* The status of the sync
140
-
* How long it took for the sync to complete
132
+
* The status of the sync.
133
+
* Details of how long it took for the sync to complete.
134
+
* How many total records were extracted, as well as a breakdown of the number of records added, updated, and deleted.
141
135
* The load results - how many successful records were synced as well as how many records were updated, deleted, or are new.
136
+
5. If your sync failed, click the failed reason to get more details on the error and view sample payloads to help troubleshoot the issue.
142
137
143
138
144
139
### Edit your model
@@ -168,29 +163,27 @@ In order for Segment to compute the data changes within your warehouse, Segment
168
163
## Segment Connections destination
169
164
If you don’t see your destination listed in the [Reverse ETL catalog], use the [Segment Connections destination](/docs/connections/destinations/catalog/actions-segment/) to send data from your Reverse ETL warehouse to other destinations listed in the [catalog](/docs/connections/destinations/catalog/).
170
165
171
-
The Segment Connections destination enables you to mold data extracted from your warehouse in [Segment Spec](docs/connections/spec/) API calls that are then processed by [Segment’s HTTP Tracking API](/docs/connections/sources/catalog/libraries/server/http-api/). The Segment HTTP Tracking API lets you record analytics data. The requests hit Segment’s servers, and then Segment routes your data to any destination you want. Get started with the [Segment Connections destination](/docs/connections/destinations/catalog/actions-segment/).
166
+
The Segment Connections destination enables you to mold data extracted from your warehouse in [Segment Spec](/docs/connections/spec/) API calls that are then processed by [Segment’s HTTP Tracking API](/docs/connections/sources/catalog/libraries/server/http-api/). The requests hit Segment’s servers, and then Segment routes your data to any destination you want. Get started with the [Segment Connections destination](/docs/connections/destinations/catalog/actions-segment/).
172
167
173
168
> warning ""
174
-
> There are cost implications to using the Segment Connections destination. New users count as new MTUs and each call counts as an API call, which affects your Reverse ETL usage limits and also your Segment costs.
169
+
> The Segment Connections destination sends data to Segment’s Tracking API, which has cost implications. New users count as new MTUs and each call counts as an API call. For information on how Segment calculates MTUs and API calls, please see [MTUs, Throughput and Billing](/docs/guides/usage-and-billing/mtus-and-throughput/).
175
170
176
171
## Limits
177
172
To provide consistent performance and reliability at scale, Segment enforces default use and rate limits for Reverse ETL.
178
173
179
174
### Usage limits
180
-
Reverse ETL usage limits are measured based on the number of records processed to each destination – this includes both successful and failed records. Processed records represents the number of records attempted to send to each destination. Not all processed records are successfully delivered. For example, if you processed 50k records to Braze and 50k records to Mixpanel, then your total usage is 100k records.
175
+
Reverse ETL usage limits are measured based on the number of records processed to each destination – this includes both successful and failed records. Processed records represents the number of records attempted to send to each destination. For example, if you processed 50k records to Braze and 50k records to Mixpanel, then your total usage is 100k records.
181
176
182
-
Your plan determines how many Reverse ETL records you can process in one monthly billing cycle. When your limit is reached before the end of your billing period, your syncs will pause and then resume on your next billing cycle.
177
+
Your plan determines how many Reverse ETL records you can process in one monthly billing cycle. When your limit is reached before the end of your billing period, your syncs will pause and then resume on your next billing cycle. To see how many records you’ve processed using Reverse ETL, navigate to **Settings > Usage & billing** and select the **Reverse ETL** tab.
183
178
184
-
Plan | Number of Reverse ETL records you can process to each destination per month | How to increase your number of Reverse ETL records
179
+
Plan | Number of Reverse ETL records you can process to destinations per month | How to increase your number of Reverse ETL records
Free | 500K | Upgrade to the Teams plan in the Segment app by navigating to **Settings > Usage & billing**.
187
182
Teams | 1 million | Contact your sales representative to upgrade your plan to Business.
188
183
Business | 50 x the number of [MTUs](/docs/guides/usage-and-billing/mtus-and-throughput/#what-is-an-mtu) <br>or .25 x the number of monthly API calls | Contact your sales rep to upgrade your plan.
189
184
190
185
If you have a non-standard or high volume usage plan, you may have unique Reverse ETL limits or custom pricing.
191
186
192
-
To see how many records you’ve processed using Reverse ETL, navigate to **Settings > Usage & billing** and select the **Reverse ETL** tab.
193
-
194
187
### Configuration limits
195
188
196
189
Name | Details | Limit
@@ -201,7 +194,7 @@ Model timestamp column name length | The maximum length for the timestamp column
201
194
Sync frequency | The shortest possible duration Segment allows between syncs. | 15 minutes
202
195
203
196
### Extract limits
204
-
The extract phase is the time spent connecting to your database, executing the model query, updating internal state tables and staging the extracted records for loading. There is a 14-day data retention period to support internal disaster recovery and debugging as needed.
197
+
The extract phase is the time spent connecting to your database, executing the model query, updating internal state tables and staging the extracted records for loading.
0 commit comments