Skip to content

Commit 60f6e45

Browse files
committed
Adding sync frequency, schema context
1 parent 3bbca9a commit 60f6e45

File tree

2 files changed

+19
-5
lines changed

2 files changed

+19
-5
lines changed

src/connections/storage/warehouses/schema.md

Lines changed: 8 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,14 @@
22
title: Warehouse Schemas
33
---
44

5-
A **schema** describes the way a warehouse is organized. Schemas include all database elements: tables, views, indexes, and synonyms, and the relationships that exist between elements.
5+
A **schema** describes the way that the data in a warehouse is organized. Schemas include a detailed description of database elements (tables, views, indexes, synonyms, etc.) and the relationships that exist between elements.
6+
7+
Schemas of warehouse data are organized into the following template: <br/>
8+
`<source>.<collection>.<property>` <br/><br/>
9+
eg. `segment-engineering.tracks.userId`, where Source refers to the source or project name (segment-engineering), collection refers to the event (tracks), and the property refers to the data being collected (userId).
10+
11+
> note "Data warehouse column creation"
12+
> **Note:** Segment creates tables for each of your custom events, and columns for each event's custom properties. Segment does not allow unbounded `event` or `property` spaces in your data. Instead of recording events like "Ordered Product 15", use a single property of "Product Number" or similar.
613
714
## Warehouse tables
815

@@ -393,9 +400,6 @@ ORDER BY day
393400

394401
New event properties and traits create columns. Segment processes the incoming data in batches, based on either data size or an interval of time. If the table doesn't exist we lock and create the table. If the table exists but new columns need to be created, we perform a diff and alter the table to append new columns.
395402

396-
> note "Column creation in Redshift"
397-
> **Note:** Segment creates tables for each of your custom events, and columns for each event's custom properties. Redshift has limits on the number of columns in a table, so Segment does not allow unbounded event or property spaces in your data. Instead of recording events like "Ordered Product 15", use a single property of "Product Number" or similar.
398-
399403
When Segment process a new batch and discover a new column to add, we take the most recent occurrence of a column and choose its datatype.
400404

401405
The data types that we currently support include: 

src/connections/storage/warehouses/warehouse-syncs.md

Lines changed: 11 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -11,7 +11,17 @@ Instead of constantly streaming data to the warehouse destination, Segment loads
1111

1212
{% include content/how-a-sync-works.md %}
1313

14-
Warehouses sync with all data coming from your source and your data is available in your warehouse within 24-48 hours. If you'd like to manage the data you send to your warehouse, use [Warehouse Selective Sync](#warehouse-selective-sync).
14+
Warehouses sync with all data coming from your source.
15+
16+
## Sync frequency
17+
18+
| Plan | Frequency |
19+
| -------- | ----------- |
20+
| Free | Once a day |
21+
| Team | Twice a day |
22+
| Business | Up to 24 times a day. Generally, these syncs are fixed to the top of the hour (:00), but these times can vary. |
23+
24+
If you're a Business plan member and would like to manage the data you send to your warehouse, use [Warehouse Selective Sync](#warehouse-selective-sync).
1525

1626
## Sync History
1727
You can use the Sync History page to see the status and history of data updates in your warehouse. The Sync History page is available for every source connected to each warehouse. This page helps you answer questions like, “Has the data from a specific source been updated recently?” “Did a sync completely fail, or only partially fail?” and “Why wasn’t this sync successful?”

0 commit comments

Comments
 (0)