Skip to content

Commit 0d7a0d8

Browse files
committed
Adding first pass of schema overview, updating 'How a sync works' include
1 parent 57321c0 commit 0d7a0d8

File tree

2 files changed

+7
-5
lines changed

2 files changed

+7
-5
lines changed
Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
When Segment loads data into your warehouse, each sync goes through the following steps:
2-
1. **Ping:** Segment servers try to connect to your warehouse.
3-
2. **Scan:** Segment finds new events and updated objects.
4-
3. **Download:** Segment pulls events and objects from AWS S3 or Dynamo into a staging area.
5-
4. **Process:** The raw Segment event and object archive files are pulled from the staging area and transformed into database-specific formats. The [warehouse schema](/docs/connections/storage/warehouses/schema/) is also defined in this step.
6-
5. **Load:** Segment de-duplicates the transformed data and loads it into your warehouse. If you have queries set up in your warehouse, they run after the data was loaded into your warehouse.
2+
1. **Ping:** Segment servers connect to your warehouse. For Redshift warehouses, Segment also runs a query to determine how many slices a cluster has at this step. <br/>Common reasons a sync would fail at this step include a blocked VPN/IP, a warehouse that isn't publicly available, and user permission/credential errors.
3+
2. **Scan:** Segment finds new events in AWS S3 and updated objects in Dynamo.
4+
3. **Download:** Segment pulls the events and objects into a staging area.
5+
4. **Process:** The raw Segment event and object archive files are transformed into database-specific formats. The [warehouse schema](/docs/connections/storage/warehouses/schema/) is also defined in this step.
6+
5. **Load:** Segment de-duplicates the transformed data and loads it into your warehouse. If you have queries set up in your warehouse, they run after the data is loaded into your warehouse. <br/>***This is the only step that connects to your warehouse: all other steps are internal to Segment.***

src/connections/storage/warehouses/schema.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,8 @@
22
title: Warehouse Schemas
33
---
44

5+
A **schema** describes the way a warehouse is organized. Schemas include all database elements: tables, views, indexes, and synonyms, and the relationships that exist between elements.
6+
57
## Warehouse tables
68

79
The table below describes the schema in Segment Warehouses:

0 commit comments

Comments
 (0)