You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
- If you don't have a project already, [create one](https://support.google.com/cloud/answer/6251787?hl=en&ref_topic=6158848){:target="_blank"}.
31
+
- If you have an existing project, you will need to [enable the BigQuery API](https://cloud.google.com/bigquery/quickstart-web-ui){:target="_blank"}.
32
32
Once you've done so, you should see BigQuery in the "Resources" section of Cloud Platform.
33
-
-**Note:** make sure [billing is enabled](https://support.google.com/cloud/answer/6293499#enable-billing) on your project, or Segment will not be able to write into the cluster.
33
+
-**Note:** make sure [billing is enabled](https://support.google.com/cloud/answer/6293499#enable-billing){:target="_blank"} on your project, or Segment will not be able to write into the cluster.
34
34
3. Copy the project ID. You will need it when you create a warehouse source in the Segment app.
35
35
36
36
### Create a Service Account for Segment
37
37
38
-
Refer to [Google Cloud's documentation about service accounts](https://cloud.google.com/iam/docs/creating-managing-service-accounts)
39
-
for more information.
40
-
41
-
1. From the Navigation panel on the left, go to **IAM & admin** > **Service accounts**
42
-
2. Click **Create Service Account** along the top
43
-
3. Enter a name for the service account (for example: "segment-warehouses") and click **Create**
38
+
1. From the Navigation panel on the left, select **IAM & admin** > **Service accounts**.
39
+
2. Click **Create Service Account** along the top.
40
+
3. Enter a name for the service account (for example: "segment-warehouses") and click **Create**.
44
41
4. When assigning permissions, make sure to grant the following roles:
45
42
-`BigQuery Data Owner`
46
43
-`BigQuery Job User`
47
-
5.[Create a JSON key](https://cloud.google.com/iam/docs/creating-managing-service-account-keys).
48
-
The downloaded file will be used to create your warehouse in the next section.
44
+
5.[Create a JSON key](https://cloud.google.com/iam/docs/creating-managing-service-account-keys){:target="_blank"}.
45
+
The downloaded file will be used to create your warehouse in the Segment app.
46
+
47
+
Refer to [Google Cloud's documentation about service accounts](https://cloud.google.com/iam/docs/creating-managing-service-accounts){:target="_blank"} for more information.
49
48
50
49
### Create the Warehouse in Segment
51
50
52
-
1. In Segment, go to **Workspace** > **Add destination** > Search for "BigQuery"
53
-
2. Select **BigQuery**
54
-
3. Add a name for the destination to the **Name your destination** field
55
-
4. Enter your project ID in the **Project** field
56
-
5. Copy the contents of the credentials (the JSON key) into the **Credentials** field <br/>
57
-
**Optional:** Enter a [region code](https://cloud.google.com/compute/docs/regions-zones/) in the **Location** field (the default will be "US")
58
-
6. Click **Connect**
59
-
7. If Segment can connect with the provided **Project ID** and **Credentials**, a warehouse will be created and your first sync should begin shortly
51
+
1. In Segment, go to **Workspace** > **Add Destination** > Search for "BigQuery"
52
+
2. Click **BigQuery**.
53
+
3. Select the source(s) you'd like to sync with the BigQuery destination, and click **Next**.
54
+
3. Add a name for the destination to the **Name your destination** field.
55
+
4. Enter your project ID in the **Project ID** field.
56
+
**Optional:** Enter a [region code](https://cloud.google.com/compute/docs/regions-zones/){:target="_blank"} in the **Location** field (the default will be "US".)
57
+
5. Copy the contents of the JSON key into the **Credentials** field.
58
+
6. Click **Connect**.
59
+
7. If Segment can connect with the provided **Project ID** and **Credentials**, a warehouse will be created and your first sync should begin shortly.
60
60
61
-
###Schema
61
+
## Schema
62
62
63
63
BigQuery datasets are broken down into **tables** and **views**. **Tables**
64
64
contain duplicate data, **views** do _not_.
65
65
66
-
####Partitioned Tables
66
+
### Partitioned Tables
67
67
68
68
The Segment connector takes advantage of [partitioned
A [view](https://cloud.google.com/bigquery/querying-data#views) is a virtual
90
+
A [view](https://cloud.google.com/bigquery/querying-data#views){:target="_blank"} is a virtual
91
91
table defined by a SQL query. Segment uses views in the de-duplication process to
92
-
ensure that events that you are querying unique events, and the latest objects
92
+
ensure that events that you are querying are unique events and contain the latest objects
93
93
from third-party data. All Segment views are set up to show information from the last
94
94
60 days. Whenever possible, query from these views.
95
95
@@ -125,13 +125,13 @@ You can remove access to the shared Service Account
125
125
126
126
1. Create a [new Service Account for Segment](#create-a-service-account-for-segment) using the linked instructions.
127
127
2. Verify that the data is loading into your warehouse.
128
-
3. Sign in to the [Google Developers Console](https://console.developers.google.com).
128
+
3. Sign in to the [Google Developers Console](https://console.developers.google.com){:target="_blank"}.
129
129
4. Open the IAM & Admin product, and select **IAM**.
130
130
5. From the list of projects, select the project that has BigQuery enabled.
131
131
6. On the project's page, select the **Permissions** tab, and then click **view by PRINCIPALS**.
132
132
7. Select the checkbox for the `[email protected]` account and then click **Remove** to remove access to this shared Service Account.
133
133
134
-
For more information about managing IAM access, see Google's documentation, [Manage access to projects, folders, and organization](https://cloud.google.com/iam/docs/granting-changing-revoking-access).
134
+
For more information about managing IAM access, see Google's documentation, [Manage access to projects, folders, and organization](https://cloud.google.com/iam/docs/granting-changing-revoking-access){:target="_blank"}.
135
135
136
136
137
137
## Best Practices
@@ -150,7 +150,7 @@ views are not cached.
150
150
> referenced directly or indirectly by the top-level query.
151
151
152
152
To save more money, you can query the view and set a [destination
153
-
table](https://cloud.google.com/bigquery/docs/tables), and then query the
153
+
table](https://cloud.google.com/bigquery/docs/tables){:target="_blank"}, and then query the
154
154
destination table.
155
155
156
156
### Query structure
@@ -191,13 +191,13 @@ WHERE ROW_NUMBER = 1
191
191
192
192
BigQuery offers both a scalable, pay-as-you-go pricing plan based on the amount
193
193
of data scanned, or a flat-rate monthly cost. You can learn more about BigQuery
0 commit comments