You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: src/connections/storage/catalog/bigquery/index.md
+20-16Lines changed: 20 additions & 16 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -7,8 +7,7 @@ redirect_from:
7
7
8
8
Segment's [BigQuery](https://cloud.google.com/bigquery/){:target="_blank"} connector makes it easy
9
9
to load web, mobile, and third-party source data like Salesforce, Zendesk, and
10
-
Google AdWords into a BigQuery data warehouse. When you integrate BigQuery with Segment you get a fully managed data pipeline
11
-
loaded into a powerful and cost-effective data warehouse.
10
+
Google AdWords into a BigQuery data warehouse. When you integrate BigQuery with Segment you get a fully managed data pipeline loaded into a powerful and cost-effective data warehouse.
12
11
13
12
The Segment warehouse connector runs a periodic ETL (Extract - Transform - Load) process to pull raw events and objects from your sources and load them into your BigQuery cluster.
14
13
For more information about the ETL process, including how it works and common ETL use cases, refer to [Google Cloud's ETL documentation](https://cloud.google.com/learn/what-is-etl){:target="_blank"}.
@@ -22,20 +21,22 @@ To store your Segment data in BigQuery, complete the following steps:
22
21
23
22
### Create a Project and Enable BigQuery
24
23
24
+
To create a project and enable BigQuery:
25
25
1. Navigate to the [Google Developers Console](https://console.developers.google.com/){:target="_blank"}.
26
26
2. Configure the [Google Cloud Platform](https://console.cloud.google.com/){:target="_blank"}:
27
27
- If you don't have a project already, [create one](https://support.google.com/cloud/answer/6251787?hl=en&ref_topic=6158848){:target="_blank"}.
28
28
- If you have an existing project, [enable the BigQuery API](https://cloud.google.com/bigquery/quickstart-web-ui){:target="_blank"}. Once you've done so, you should see BigQuery in the "Resources" section of Cloud Platform.
29
-
3. Copy the project ID. You will need it when you create a warehouse source in the Segment app.
29
+
3. Copy the project ID. You'll need it when you create a warehouse source in the Segment app.
30
30
31
31
> note "Enable billing"
32
-
> When creating your project, you must [enable billing](https://support.google.com/cloud/answer/6293499#enable-billing){:target="_blank"} so Segment can write into the cluster.
32
+
> When you create your project, you must [enable billing](https://support.google.com/cloud/answer/6293499#enable-billing){:target="_blank"} so Segment can write into the cluster.
33
33
34
34
### Create a service account for Segment
35
35
36
+
To create a service account for Segment:
36
37
1. From the Navigation panel on the left, select **IAM & admin** > **Service accounts**.
37
38
2. Click **Create Service Account**.
38
-
3. Enter a name for the service account (for example `segment-warehouses`) and click **Create**.
39
+
3. Enter a name for the service account (for example,`segment-warehouses`) and click **Create**.
39
40
4. Assign the service account the following roles:
40
41
-`BigQuery Data Owner`
41
42
-`BigQuery Job User`
@@ -46,15 +47,17 @@ If you have trouble creating a new service account, refer to [Google Cloud's doc
46
47
47
48
### Create the Warehouse in Segment
48
49
49
-
1. From the homepage of the Segment app, select **Connections**, click **Add Destination** and search for "BigQuery".
50
+
To create the warehouse in Segment:
51
+
1. From the homepage of the Segment app, select **Connections > Add Destination** and search for **BigQuery**.
50
52
2. Click **BigQuery**.
51
53
3. Select the source(s) you'd like to sync with the BigQuery destination, and click **Next**.
52
54
3. Enter a name for your destination in the **Name your destination** field.
53
55
4. Enter your Project ID in the **Project ID** field.
54
-
<br/>**Optional:** Enter a [region code](https://cloud.google.com/compute/docs/regions-zones/){:target="_blank"} in the **Location** field (the default is "US".)
56
+
<br/>*Optional:* Enter a [region code](https://cloud.google.com/compute/docs/regions-zones/){:target="_blank"} in the **Location** field (the default is *US*.)
55
57
5. Copy the contents of the JSON key that you created for the Segment service account into the **Credentials** field.
56
58
6. Click **Connect**.
57
-
7. If Segment is able to connect to your project, a warehouse will be created and your first sync will begin shortly.
59
+
60
+
If Segment is able to connect to your project, a warehouse will be created and your first sync will begin shortly.
1. Create a [new Service Account for Segment](#create-a-service-account-for-segment) using the linked instructions.
125
129
2. Verify that the data is loading into your warehouse.
126
130
3. Sign in to the [Google Developers Console](https://console.developers.google.com){:target="_blank"}.
@@ -140,7 +144,7 @@ BigQuery charges based on the amount of data scanned by your queries. Views are
140
144
a derived view over your tables that Segment uses for de-duplication of events.
141
145
Therefore, Segment recommends you query a specific view whenever possible to avoid
142
146
duplicate events and historical objects. It's important to note that BigQuery
143
-
views are not cached.
147
+
views aren't cached.
144
148
145
149
> note "Understanding BigQuery views"
146
150
> BigQuery's views are logical views, not materialized views, which means that the query that defines the view is re-executed every time the view is queried. Queries are billed according to the total amount of data in all table fields referenced directly or indirectly by the top-level query.
@@ -151,7 +155,7 @@ destination table.
151
155
152
156
### Query structure
153
157
154
-
If you typically start exploratory data analysis with `SELECT *`, consider
158
+
If you start exploratory data analysis with `SELECT *`, consider
155
159
specifying the fields to reduce costs.
156
160
157
161
Refer to the section on [partitioned tables](#partitioned-tables) for details on
Alerts](https://cloud.google.com/bigquery/cost-controls){:target="_blank"} to help control and
191
195
monitor costs. If you want to learn more about the costs associated with BigQuery,
192
-
Google Cloud has provided[a
196
+
Google Cloud provides[a
193
197
calculator](https://cloud.google.com/products/calculator/){:target="_blank"} to estimate your
194
198
costs.
195
199
@@ -198,13 +202,13 @@ costs.
198
202
You can connect a BI tool like Mode or Looker to BigQuery, or query
199
203
directly from the BigQuery console.
200
204
201
-
BigQuery now supports standard SQL, which you can enable [using Google Cloud's query UI](https://cloud.google.com/bigquery/docs/reference/standard-sql/introduction#changing_from_the_default_dialect){:target="_blank"}.
202
-
This does not work with views, or with a query that uses table range
205
+
BigQuery supports standard SQL, which you can enable [using Google Cloud's query UI](https://cloud.google.com/bigquery/docs/reference/standard-sql/introduction#changing_from_the_default_dialect){:target="_blank"}.
206
+
This doesn't work with views, or with a query that uses table range
203
207
functions.
204
208
205
209
### Does Segment support streaming inserts?
206
210
207
-
Segment's connector does not support streaming inserts at this time. If you have
211
+
Segment's connector doesn't support streaming inserts at this time. If you have
208
212
a need for streaming data into BigQuery, [contact Segment support](https://segment.com/requests/integrations/){:target="_blank"}.
0 commit comments