Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 7 additions & 8 deletions src/segment-app/extensions/dbt.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,38 +6,37 @@ Segment's dbt extension lets you use [Reverse ETL](/docs/connections/reverse-etl

With Segment's dbt extension, you can:

- Securely connect Segment to the GitHub repository that stores your dbt models.
- Securely connect Segment to a Git repository that stores your dbt models.
- Use centralized dbt models to set up Reverse ETL.
- Trigger Reverse ETL syncs from dbt jobs.

This page explains how to set up a dbt Model and then use the model with Reverse ETL.

> info ""
> Extensions, including dbt sync, is currently in public beta and is governed by Segment's [First Access and Beta Preview Terms](https://www.twilio.com/en-us/legal/tos){:target="_blank"}.

## Before you begin

Keep the following in mind as you set up the dbt extension:

- The extension supports [dbt Core v1.7](https://docs.getdbt.com/docs/dbt-versions/core-upgrade/upgrading-to-v1.7){:target="_blank"}.
- You can use [Snowflake](/docs/connections/reverse-etl/reverse-etl-source-setup-guides/snowflake-setup/), [Databricks](/docs/connections/reverse-etl/reverse-etl-source-setup-guides/databricks-setup/), [Redshift](/docs/connections/reverse-etl/reverse-etl-source-setup-guides/redshift-setup/), [Postgres](/docs/connections/reverse-etl/reverse-etl-source-setup-guides/postgres-setup/), and [BigQuery](/docs/connections/reverse-etl/reverse-etl-source-setup-guides/bigquery-setup/) as Reverse ETL sources.
- dbt models aren't synchronized from the dbt cloud. The model sync connects to a Git repository that loads models into Segment for use with Reverse ETL.
- dbt models aren't synchronized from the dbt cloud. The model sync connects to a Git repository that loads models into Segment for use with Reverse ETL.
- You can connect to GitHub using a GitHub App, token, or SSH.
- For [GitLab](https://docs.gitlab.com/ee/user/ssh.html){:target="_blank"} and [Bitbucket](https://support.atlassian.com/bitbucket-cloud/docs/configure-ssh-and-two-step-verification/){:target="_blank"}, use SSH to connect.

## Set up Git dbt Models and dbt Cloud

To set up the dbt extension, you'll need:

- an existing dbt account with a Git repository
- (for job syncs:) dbt cloud with jobs already created
- for job syncs, dbt cloud with jobs already created

### Git repository and dbt Models setup

Follow these steps to connect the Git repository that stores your dbt Models:

1. In your Segment workspace, navigate to **Settings > Extensions**.
2. Click **Set up Git sync**.
3. On the **Configure service credentials** page, select a service and protocol, add your SSH private key or GitHub token, then click **Next**.
4. In the **Connect source** window, select a Reverse ETL warehouse source from the dropdown, then click **Save**.
3. On the **Configure service credentials** page, select a service and protocol, add your GitHub App, SSH private key or GitHub token, then click **Next**.
4. In the **Connect source** window, select an existing Reverse ETL warehouse source from the dropdown, then click **Save**.

After you've saved your setup, you can configure your Git repository's settings to your needs by changing the repository, branch, dbt version, default schema, and project path.

Expand Down
74 changes: 64 additions & 10 deletions src/segment-app/extensions/git.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,23 +2,77 @@
title: Git Sync Extension
---

Segment's Git extension lets you manage versioning by syncing changes you make in Sources and Destinations from your Segment workspace to a Git repository.
Segment's Git extension lets you manage versioning by syncing changes you make in your Segment workspace to a Git repository.

> info ""
> Extensions, including Git sync, is currently in public beta and is governed by Segment's [First Access and Beta Preview Terms](https://www.twilio.com/en-us/legal/tos){:target="_blank"}.
Git Sync supports one-way synchronization from Segment to Git. This sync captures the current state of your workspace through a full sync and includes all new records and changes for supported resources.

## Set up Git sync
Segment doesn't support syncing changes from Git back to Segment.

Follow these steps to set up Git sync:
## Set up Git Sync

Follow these steps to set up Git Sync:

1. In your Segment workspace, navigate to **Settings > Extensions**.
2. Click **Set up Git sync**.
3. On the **Configure service credentials** page, select a service and protocol, add your SSH private key or GitHub token, then click **Next**.
3. On the **Configure service credentials** page, select a service and protocol, add your GitHub App, SSH private key, or GitHub token, then click **Next**.
- To connect to GitLab or Bitbucket, use your SSH private key.

## Working with Git Sync

The Git sync extension syncs the following resources from Segment to your Git repository:

- [Sources](/docs/connections/sources/) and [Destinations](/docs/connections/destinations/)
- [Warehouses](/docs/connections/storage/warehouses/)
- [Destination Filters and Mappings](/docs/connections/destinations/destination-filters/)
- [Tracking Plans](/docs/protocols/tracking-plan/create/)
- [Functions](/docs/connections/functions/)
- [Transformations](/docs/protocols/transform/)
- [Reverse ETL](/docs/connections/reverse-etl/)
- [Users](/docs/segment-app/iam/concepts/#team-members) and [User groups](/docs/segment-app/iam/concepts/#user-groups)
- [Labels](/docs/segment-app/iam/labels/#where-can-i-create-labels)

Reach out to [Segment support](https://app.segment.com/workspaces?contact=1){:target="blank"} to request support for additional Git Sync resources.

After you set up the Git sync extension for the first time, Segment performs an initial sync that sends the current state of your Segment workspace to the Git repository you connected. Segment automatically tracks all following workspace updates.

You can manually trigger syncs at any time by clicking **Full Sync** on the Git Sync page. To disable Git Sync from the Git Sync page, switch the **Enabled** toggle to off.

## Git Sync architecture and data model

Because a Segment workspace can represent a distinct environment (testing, staging, production), each workspace is mapped directly to a single Git repository. This direct mapping ensures a clear and organized relationship between workspace resources and a Git repository.

Segment uses its [Terraform provider](https://registry.terraform.io/providers/segmentio/segment/1.0.3){:target="_blank"} to manage key functions like tracking changes and retrieving information about those changes in Segment. Segment stores changes in HashiCorp Configuration Language (HCL), the format used by Terraform. To learn more about HCL and how it compares to JSON or YAML, visit [HashiCorp's HCL repository on GitHub](https://github.com/hashicorp/hcl){:target="_blank"}.

Using HCL makes it easier to document Segment's data model, especially for users managing versioning and Git Sync with Terraform. It also helps manage Segment configurations directly from Git. For more details on the Git Sync data model, read [Segment's Terraform provider documentation](https://registry.terraform.io/providers/segmentio/segment/latest/docs){:target="_blank"}.

## Managing your Segment workspace with Terraform and Git Sync

Segment supports one-way synchronization from Segment to Git, but you can set up two-way synchronization using the Segment Terraform provider.

Terraform offers an open-source way to manage Segment resources through a Git repository as an alternative to a fully managed two-way sync. However, this method requires third-party tools like [Atlantis](https://www.runatlantis.io/){:target="_blank"} for CI integration, which Segment doesn’t officially support.

To manage Segment resources using Git and Terraform, follow these steps:

1. Copy the generated Terraform configuration for the resources you want to manage into a separate Git repository dedicated to Terraform.
2. Include the following provider configuration blocks:

```hcl
# providers.tf

## Working with Git syncs
terraform {
required_providers {
segment = {
source = "segmentio/segment"
version = "1.0.4"
}
}
}

The Git sync extension syncs [Connections](/docs/connections/) ([Sources](/docs/connections/sources/) and [Destinations](/docs/connections/destinations/)) from Segment to your Git repository.
provider "segment" {
# Provide the token directly or load it from an environment variable
}
```
3. Apply configuration changes by running Terraform locally or using a tool like Atlantis to run it directly from your Git provider.

After you set up the Git sync extension for the first time, Segment performs an initial sync that sends source and destination information in your Segment workspace to the Git repository you connected.

You can run syncs at any time by clicking **Full resync** on the Git sync page. To disable Git sync from the Git sync page, switch the **Enabled** toggle to off.
For more information on using Terraform, visit [Terraform's documentation](https://developer.hashicorp.com/terraform/docs){:target="_blank"}.
11 changes: 5 additions & 6 deletions src/segment-app/extensions/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,12 +2,11 @@
title: Extensions
---

Extensions integrate third-party tools into your existing Segment workspace to help you automate tasks.
Extensions let you integrate third-party tools into your existing Segment workspace, helping you automate tasks, manage data flows, and maintain version control.

> info ""
> Extensions is currently in public beta and is governed by Segment's [First Access and Beta Preview Terms](https://www.twilio.com/en-us/legal/tos){:target="_blank"}. During Public Beta, Extensions is available for Team and Developer plans only. [Reach out to Segment](mailto:[email protected]) if you're on a Business Tier plan and would like to participate in the Public Beta.
Segment offers the following extensions:

During public beta, Segment offers the following extensions:
- [dbt models and dbt Cloud](/docs/segment-app/extensions/dbt): Sync your dbt Labs models with Segment to streamline model management, versioning, and CI checks. This extension lets you securely connect Segment to a Git repository, making it easier to integrate and manage dbt models across different environments like testing, staging, and production.
- [Git Sync](/docs/segment-app/extensions/git): Manage versioning and track changes by syncing your Segment workspace a Git repository. The Git Sync extension helps maintain a clear and organized relationship between your workspace and its corresponding Git repository, ensuring that your resources are consistently managed and versioned across your environments.

- [dbt models and dbt Cloud](/docs/segment-app/extensions/dbt), which lets you sync models with dbt Labs.
- [Git sync](/docs/segment-app/extensions/git), which you can use to manage versioning and track changes in your Segment workspace.
Segment built Extensions to help you get the most out of your Segment workspace, allowing you to keep your projects organized, efficient, and aligned with best practices for data management and version control.