|
| 1 | +--- |
| 2 | +title: "DeltaStream setup" |
| 3 | +description: "Read this guide to learn about the DeltaStream warehouse setup in dbt." |
| 4 | +meta: |
| 5 | + maintained_by: Community |
| 6 | + authors: 'DeltaStream Team' |
| 7 | + github_repo: 'deltastreaminc/dbt-deltastream' |
| 8 | + pypi_package: 'dbt-deltastream' |
| 9 | + min_core_version: 'v1.10.0' |
| 10 | + cloud_support: Not supported |
| 11 | + min_supported_version: '?' |
| 12 | + slack_channel_name: '#db-deltastream' |
| 13 | + platform_name: 'DeltaStream' |
| 14 | + config_page: '/reference/resource-configs/deltastream-configs' |
| 15 | +--- |
| 16 | + |
| 17 | +import SetUpPages from '/snippets/_setup-pages-intro.md'; |
| 18 | + |
| 19 | +<SetUpPages meta={frontMatter.meta} /> |
| 20 | + |
| 21 | +## Connecting to DeltaStream with **dbt-deltastream** |
| 22 | + |
| 23 | +To connect to DeltaStream from dbt, you'll need to add a [profile](/docs/core/connect-data-platform/connection-profiles) |
| 24 | +to your `profiles.yml` file. A DeltaStream profile conforms to the following syntax: |
| 25 | + |
| 26 | +<File name='profiles.yml'> |
| 27 | + |
| 28 | +```yaml |
| 29 | +<profile-name>: |
| 30 | + target: <target-name> |
| 31 | + outputs: |
| 32 | + <target-name>: |
| 33 | + type: deltastream |
| 34 | + |
| 35 | + # Required parameters |
| 36 | + token: [ your-api-token ] # Authentication token for DeltaStream API |
| 37 | + database: [ your-database ] # Target database name |
| 38 | + schema: [ your-schema ] # Target schema name |
| 39 | + organization_id: [ your-org-id ] # Organization identifier |
| 40 | + |
| 41 | + # Optional parameters |
| 42 | + url: [ https://api.deltastream.io/v2 ] # DeltaStream API URL, defaults to https://api.deltastream.io/v2 |
| 43 | + timezone: [ UTC ] # Timezone for operations, defaults to UTC |
| 44 | + session_id: [ <empty string> ] # Custom session identifier for debugging purpose |
| 45 | + role: [ <empty string> ] # User role |
| 46 | + store: [ <empty string> ] # Target store name |
| 47 | + compute_pool: [ <empty string> ] # Compute pool name to be used if any else use the default compute pool |
| 48 | +``` |
| 49 | +
|
| 50 | +</File> |
| 51 | +
|
| 52 | +### Description of DeltaStream profile fields |
| 53 | +
|
| 54 | +| Field | Required | Description | |
| 55 | +|-------------------|----------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| |
| 56 | +| `type` | ✅ | This must be included either in `profiles.yml` or in the `dbt_project.yml` file. Must be set to `deltastream`. | |
| 57 | +| `token` | ✅ | Authentication token for DeltaStream API. This should be stored securely, preferably as an environment variable. | |
| 58 | +| `database` | ✅ | Target default database name in DeltaStream where your dbt models will be created. | |
| 59 | +| `schema` | ✅ | Target default schema name within the specified database. | |
| 60 | +| `organization_id` | ✅ | Organization identifier that determines which DeltaStream organization you're connecting to. | |
| 61 | +| `url` | ❌ | DeltaStream API URL. Defaults to `https://api.deltastream.io/v2` if not specified. | |
| 62 | +| `timezone` | ❌ | Timezone for operations. Defaults to `UTC` if not specified. | |
| 63 | +| `session_id` | ❌ | Custom session identifier for debugging purposes. Helps track operations in DeltaStream logs. | |
| 64 | +| `role` | ❌ | User role within the DeltaStream organization. If not specified, uses the default role associated with the token. | |
| 65 | +| `store` | ❌ | Target default store name. Stores represent external system connections (Kafka, PostgreSQL, etc.) in DeltaStream. | |
| 66 | +| `compute_pool` | ❌ | Compute pool name to be used for models that require computational resources. If not specified, uses the default compute pool. | |
| 67 | + |
| 68 | +## Security best practices |
| 69 | + |
| 70 | +When configuring your project for production, it is strongly recommended to use environment variables to store sensitive information such as the authentication token: |
| 71 | + |
| 72 | +<File name='profiles.yml'> |
| 73 | + |
| 74 | +```yaml |
| 75 | +your_profile_name: |
| 76 | + target: prod |
| 77 | + outputs: |
| 78 | + prod: |
| 79 | + type: deltastream |
| 80 | + token: "{{ env_var('DELTASTREAM_API_TOKEN') }}" |
| 81 | + database: "{{ env_var('DELTASTREAM_DATABASE') }}" |
| 82 | + schema: "{{ env_var('DELTASTREAM_SCHEMA') }}" |
| 83 | + organization_id: "{{ env_var('DELTASTREAM_ORG_ID') }}" |
| 84 | +``` |
| 85 | + |
| 86 | +</File> |
| 87 | + |
| 88 | +## Troubleshooting connections |
| 89 | + |
| 90 | +If you encounter issues connecting to DeltaStream from dbt, verify the following: |
| 91 | + |
| 92 | +### Authentication issues |
| 93 | + |
| 94 | +- Ensure your API token is valid and has not expired |
| 95 | +- Verify the token has appropriate permissions for the target organization |
| 96 | +- Check that the `organization_id` matches your DeltaStream organization |
0 commit comments