You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
This dbt package is part of Elementary, the dbt-native data observability solution for data and analytics engineers.
17
-
Set up in minutes, gain immediate visibility, detect data issues, send actionable alerts, and understand impact and root cause.
18
-
Available as self-hosted or Cloud service with premium features.
16
+
This dbt-native package powers **Elementary**, helping data and analytics engineers **detect data anomalies** and build **rich metadata tables** from their dbt runs and tests. Gain immediate visibility into data quality trends, **detect anomalies**, and uncover potential issues—all within dbt.
19
17
20
-
#### Table of Contents
18
+
Choose the observability tool that fits your needs:
-[Get more out of Elementary](#get-more-out-of-elementary-dbt-package)
24
-
-[Run results and dbt artifacts](#run-results-and-dbt-artifacts)
25
-
-[Data anomaly detection as dbt tests](#data-anomaly-detection-as-dbt-tests)
26
-
-[How Elementary works?](#how-elementary-works)
27
-
-[Community & Support](#community--support)
28
-
-[Contribution](#contributions)
20
+
✅ [**Elementary Open Source**](https://docs.elementary-data.com/oss/oss-introduction) – A powerful, self-hosted tool for teams that want full control.
29
21
30
-
## Quick start - dbt package
22
+
✅ [**Elementary Cloud Platform**](https://docs.elementary-data.com/cloud/introduction) – A fully managed, enterprise-ready solution with **automated ML-powered anomaly detection, flexible data discovery, integrated incident management, and collaboration tools**—all with minimal setup and infrastructure maintenance.
23
+
24
+
### Table of Contents
25
+
26
+
-[What's Inside the Elementary dbt Package?](#whats-inside-the-elementary-dbt-package)
-[Get more out of Elementary dbt package](#get-more-out-of-elementary-dbt-package)
29
+
-[Elementary tables - Run Results and dbt artifacts](#elementary-tables---run-results-and-dbt-artifacts)
30
+
-[Data anomaly detection as dbt tests](#data-anomaly-detection-as-dbt-tests)
31
+
-[Community & Support](#community--support)
32
+
-[Contributions](#contributions)
33
+
34
+
35
+
### **What's Inside the Elementary dbt Package?**
36
+
37
+
The **Elementary dbt package** is designed to enhance data observability within your dbt workflows. It includes two core components:
38
+
39
+
-**Elementary Tests** – A collection of **anomaly detection tests** and other data quality checks that help identify unexpected trends, missing data, or schema changes directly within your dbt runs.
40
+
-**Metadata & Test Results Tables** – The package automatically generates and updates **metadata tables** in your data warehouse, capturing valuable information from your dbt runs and test results. These tables act as the backbone of your **observability setup**, enabling **alerts and reports** when connected to an Elementary observability platform.
41
+
42
+
## Quickstart - dbt package
31
43
32
44
1. Add to your `packages.yml`:
33
45
34
-
```yml packages.yml
46
+
```
35
47
packages:
36
48
- package: elementary-data/elementary
37
49
version: 0.17.0
38
-
## Docs: https://docs.elementary-data.com
50
+
## Docs: <https://docs.elementary-data.com>
51
+
39
52
```
40
53
41
54
2. Run `dbt deps`
42
-
43
55
3. Add to your `dbt_project.yml`:
44
56
45
-
```yml
57
+
```
46
58
models:
47
59
## elementary models will be created in the schema '<your_schema>_elementary'
48
-
## for details, see docs: https://docs.elementary-data.com/
60
+
## for details, see docs: <https://docs.elementary-data.com/>
49
61
elementary:
50
62
+schema: "elementary"
63
+
51
64
```
52
65
53
66
4. Run `dbt run --select elementary`
@@ -56,69 +69,88 @@ Check out the [full documentation](https://docs.elementary-data.com/).
56
69
57
70
## Get more out of Elementary dbt package
58
71
59
-
Elementary has 3 offerings: This dbt package, Elementary Community (OSS) and Elementary (cloud service).
72
+
The **Elementary dbt package** helps you find anomalies in your data and build metadata tables from your dbt runs and tests—but there's even more you can do.
73
+
74
+
To generate observability reports, send alerts, and govern your data quality effectively, connect your dbt package to one of the following options:
60
75
61
-
- **dbt package**
62
-
- For basic data monitoring and dbt artifacts collection, Elementary offers a dbt package. The package adds logging, artifacts uploading, and Elementary tests (anomaly detection and schema) to your project.
63
-
- **Elementary Community**
64
-
- An open-source CLI tool you can deploy and orchestrate to send alerts and self-host the Elementary report. Best for data and analytics engineers that require basic observability capabilities or for evaluating features without vendor approval. Our community can provide great support on [Slack](https://www.elementary-data.com/community) if needed.
76
+
-**Elementary OSS**
77
+
- An open-source CLI tool you can **deploy and orchestrate to send alerts** and **self-host the Elementary report**. Best for data and analytics engineers that require basic observability capabilities or for evaluating features without vendor approval. Quickstart [here](https://docs.elementary-data.com/oss/quickstart/quickstart-cli), and our team and community can provide great support on [Slack](https://www.elementary-data.com/community) if needed.
65
78
-**Elementary Cloud**
66
-
- Ideal for teams monitoring mission-critical data pipelines, requiring guaranteed uptime and reliability, short-time-to-value, advanced features, collaboration, and professional support. The solution is secure by design, and requires no access to your data from cloud. To learn more, [book a demo](https://cal.com/maayansa/elementary-intro-github-package) or [start a trial](https://www.elementary-data.com/signup).
79
+
- A **fully managed, enterprise-ready** solution designed for **scalability and automation**. It offers automated **ML-powered anomaly detection**, flexible **data discovery**, an integrated **incident management system**, and **collaboration features.** Delivering **high value with minimal setup and infrastructure maintenance**, it's ideal for teams looking to enhance data reliability without operational overhead. To learn more, [book a demo](https://cal.com/maayansa/elementary-intro-github-package) or [start a trial](https://www.elementary-data.com/signup).
The package automatically uploads dbt artifacts and run results to tables in your data warehouse:
85
+
## Elementary tables - Run Results and dbt artifacts
71
86
72
-
Run resultstables:
87
+
The **Elementary dbt package** automatically stores **dbt artifacts and run results** in your data warehouse, creating structured tables that provide visibility into your dbt runs and metadata.
73
88
74
-
- dbt_run_results
75
-
- model_run_results
76
-
- snapshot_run_results
77
-
- dbt_invocations
78
-
- elementary_test_results (all dbt test results)
89
+
### **Metadata Tables - dbt artifacts**
79
90
80
-
Metadata tables:
91
+
These tables provide a comprehensive view of your dbt project structure and configurations:
81
92
82
-
- dbt_models
83
-
- dbt_tests
84
-
- dbt_sources
85
-
- dbt_exposures
86
-
- dbt_metrics
87
-
- dbt_snapshots
93
+
-**dbt_models** – Details on all dbt models.
94
+
-**dbt_tests** – Stores information about dbt tests.
95
+
-**dbt_sources** – Tracks source tables and freshness checks.
-**dbt_seeds -** Stores current metadata about seed files in the dbt project.
100
+
-**dbt_columns** - Stores detailed information about columns across the dbt project.
88
101
89
-
Here you can find [additional details about the tables](https://docs.elementary-data.com/guides/modules-overview/dbt-package).
102
+
### **Run Results Tables**
90
103
91
-
## Data anomaly detection as dbt tests
104
+
These tables track execution details, test outcomes, and performance metrics from your dbt runs:
105
+
106
+
-**dbt_run_results** – Captures high-level details of each dbt run.
107
+
-**model_run_results** – Stores execution data for dbt models.
108
+
-**snapshot_run_results** – Logs results from dbt snapshots.
109
+
-**dbt_invocations** – Tracks each instance of dbt being run.
110
+
-**elementary_test_results** – Consolidates all dbt test results, including Elementary anomaly tests.
92
111
93
-
Elementary dbt tests collect metrics and metadata over time, such as freshness, volume, schema changes, distribution, cardinality, etc.
94
-
Executed as any other dbt tests, the Elementary tests alert on anomalies and outliers.
112
+
For a full breakdown of these tables, see the [documentation](https://docs.elementary-data.com/dbt/package-models).
113
+
114
+
## Data anomaly detection as dbt tests
95
115
96
116
**Elementary tests are configured and executed like native tests in your project!**
97
117
98
-
Example of Elementary test config in `properties.yml`:
118
+
Elementary dbt tests help track and alert on schema changes as well as key metrics and metadata over time, including freshness, volume, distribution, cardinality, and more.
119
+
120
+
**Seamlessly configured and run like native dbt tests,** Elementary tests detect anomalies and outliers, helping you catch data issues early.
99
121
100
-
```yml
122
+
Example of an Elementary test config in `schema.yml`:
123
+
124
+
```
101
125
models:
102
126
- name: your_model_name
103
127
config:
104
128
elementary:
105
129
timestamp_column: updated_at
106
130
tests:
107
-
- elementary.table_anomalies
108
-
- elementary.all_columns_anomalies
131
+
- elementary.all_columns_anomalies
132
+
109
133
```
110
134
111
-
Read about the available [Elementary tests and configuration](https://docs.elementary-data.com/data-tests/introduction).
135
+
Elementary tests include:
112
136
113
-
## How Elementary works?
137
+
### **Anomaly detection tests**
114
138
115
-
Elementary dbt package creates tables of metadata and test results in your data warehouse, as part of your dbt runs.
139
+
-**Volume anomalies -** Monitors the row count of your table over time per time bucket.
140
+
-**Freshness anomalies -** Monitors the freshness of your table over time, as the expected time between data updates.
141
+
-**Event freshness anomalies -** Monitors the freshness of event data over time, as the expected time it takes each event to load - that is, the time between when the event actually occurs (the **`event timestamp`**), and when it is loaded to the database (the **`update timestamp`**).
142
+
-**Dimension anomalies -** Monitors the count of rows grouped by given **`dimensions`** (columns/expressions).
143
+
-**Column anomalies -** Executes column level monitors on a certain column, with a chosen metric.
144
+
-**All columns anomalies** - Executes column level monitors and anomaly detection on all the columns of the table.
116
145
117
-
The cloud service or the CLI tool read the data from these tables, send alerts and present the results in the UI.
-**Schema changes -** Alerts on a deleted table, deleted or added columns, or change of data type of a column.
149
+
-**Schema changes from baseline** - Checks for schema changes against baseline columns defined in a source’s or model’s configuration.
150
+
-**JSON schema** - Allows validating that a string column matches a given JSON schema.
151
+
-**Exposure validation test -** Detects changes in your models’ columns that break downstream exposure.
152
+
153
+
Read more about the available [Elementary tests and configuration](https://docs.elementary-data.com/data-tests/introduction).
122
154
123
155
## Community & Support
124
156
@@ -129,4 +161,4 @@ The cloud service or the CLI tool read the data from these tables, send alerts a
129
161
130
162
Thank you :orange_heart: Whether it’s a bug fix, new feature, or additional documentation - we greatly appreciate contributions!
131
163
132
-
Check out the [contributions guide](https://docs.elementary-data.com/general/contributions) and [open issues](https://github.com/elementary-data/elementary/issues) in the main repo.
164
+
Check out the [contributions guide](https://docs.elementary-data.com/oss/general/contributions) and [open issues](https://github.com/elementary-data/elementary/issues) in the main repo.
0 commit comments