You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
@@ -39,54 +39,82 @@ The **Elementary dbt package** is designed to enhance data observability within
39
39
-**Elementary Tests** – A collection of **anomaly detection tests** and other data quality checks that help identify unexpected trends, missing data, or schema changes directly within your dbt runs.
40
40
-**Metadata & Test Results Tables** – The package automatically generates and updates **metadata tables** in your data warehouse, capturing valuable information from your dbt runs and test results. These tables act as the backbone of your **observability setup**, enabling **alerts and reports** when connected to an Elementary observability platform.
41
41
42
-
## Quickstart - dbt package
43
42
44
-
1. Add to your `packages.yml`:
43
+
## Get more out of Elementary dbt package
45
44
46
-
```
47
-
packages:
48
-
- package: elementary-data/elementary
49
-
version: 0.17.0
50
-
## Docs: <https://docs.elementary-data.com>
45
+
The **Elementary dbt package** helps you find anomalies in your data and build metadata tables from your dbt runs and tests—but there's even more you can do.
51
46
52
-
```
47
+
To generate observability reports, send alerts, and govern your data quality effectively, connect your dbt package to one of the following options:
53
48
54
-
2. Run `dbt deps`
55
-
3. Add to your `dbt_project.yml`:
49
+
-**Elementary OSS**
50
+
- An open-source CLI tool you can **deploy and orchestrate to send alerts** and **self-host the Elementary report**. Best for data and analytics engineers that require basic observability capabilities or for evaluating features without vendor approval. Quickstart [here](https://docs.elementary-data.com/oss/quickstart/quickstart-cli), and our team and community can provide great support on [Slack](https://www.elementary-data.com/community) if needed.
51
+
-**Elementary Cloud**
52
+
- A **fully managed, enterprise-ready** solution designed for **scalability and automation**. It offers automated **ML-powered anomaly detection**, flexible **data discovery**, an integrated **incident management system**, and **collaboration features.** Delivering **high value with minimal setup and infrastructure maintenance**, it's ideal for teams looking to enhance data reliability without operational overhead. To learn more, [book a demo](https://cal.com/maayansa/elementary-intro-github-package) or [start a trial](https://www.elementary-data.com/signup).
## Data Anomaly Detection & Schema changes as dbt Tests
60
+
61
+
**Elementary tests are configured and executed like native tests in your project!**
62
+
63
+
Elementary dbt tests help track and alert on schema changes as well as key metrics and metadata over time, including freshness, volume, distribution, cardinality, and more.
64
+
65
+
**Seamlessly configured and run like native dbt tests,** Elementary tests detect anomalies and outliers, helping you catch data issues early.
66
+
67
+
Example of an Elementary test config in `schema.yml`:
56
68
57
69
```
70
+
58
71
models:
59
-
## elementary models will be created in the schema '<your_schema>_elementary'
60
-
## for details, see docs: <https://docs.elementary-data.com/>
61
-
elementary:
62
-
+schema: "elementary"
72
+
- name: all_events
73
+
config:
74
+
elementary:
75
+
timestamp_column: 'loaded_at'
76
+
columns:
77
+
- name: event_count
78
+
tests:
79
+
- elementary.column_anomalies:
80
+
column_anomalies:
81
+
- average
82
+
where_expression: "event_type in ('event_1', 'event_2') and country_name != 'unwanted country'"
83
+
anomaly_sensitivity: 2
84
+
time_bucket:
85
+
period: day
86
+
count:1
63
87
64
88
```
65
89
66
-
4. Run `dbt run --select elementary`
90
+
Elementary tests include:
67
91
68
-
Check out the [full documentation](https://docs.elementary-data.com/).
92
+
### **Anomaly Detection Tests**
69
93
70
-
## Get more out of Elementary dbt package
94
+
-**Volume anomalies -** Monitors the row count of your table over time per time bucket.
95
+
-**Freshness anomalies -** Monitors the freshness of your table over time, as the expected time between data updates.
96
+
-**Event freshness anomalies -** Monitors the freshness of event data over time, as the expected time it takes each event to load - that is, the time between when the event actually occurs (the **`event timestamp`**), and when it is loaded to the database (the **`update timestamp`**).
97
+
-**Dimension anomalies -** Monitors the count of rows grouped by given **`dimensions`** (columns/expressions).
98
+
-**Column anomalies -** Executes column level monitors on a certain column, with a chosen metric.
99
+
-**All columns anomalies** - Executes column level monitors and anomaly detection on all the columns of the table.
71
100
72
-
The**Elementary dbt package** helps you find anomalies in your data and build metadata tables from your dbt runs and tests—but there's even more you can do.
101
+
###**Schema Tests**
73
102
74
-
To generate observability reports, send alerts, and govern your data quality effectively, connect your dbt package to one of the following options:
103
+
-**Schema changes -** Alerts on a deleted table, deleted or added columns, or change of data type of a column.
104
+
-**Schema changes from baseline** - Checks for schema changes against baseline columns defined in a source’s or model’s configuration.
105
+
-**JSON schema** - Allows validating that a string column matches a given JSON schema.
106
+
-**Exposure validation test -** Detects changes in your models’ columns that break downstream exposure.
107
+
108
+
Read more about the available [Elementary tests and configuration](https://docs.elementary-data.com/data-tests/introduction).
75
109
76
-
-**Elementary OSS**
77
-
- An open-source CLI tool you can **deploy and orchestrate to send alerts** and **self-host the Elementary report**. Best for data and analytics engineers that require basic observability capabilities or for evaluating features without vendor approval. Quickstart [here](https://docs.elementary-data.com/oss/quickstart/quickstart-cli), and our team and community can provide great support on [Slack](https://www.elementary-data.com/community) if needed.
78
-
-**Elementary Cloud**
79
-
- A **fully managed, enterprise-ready** solution designed for **scalability and automation**. It offers automated **ML-powered anomaly detection**, flexible **data discovery**, an integrated **incident management system**, and **collaboration features.** Delivering **high value with minimal setup and infrastructure maintenance**, it's ideal for teams looking to enhance data reliability without operational overhead. To learn more, [book a demo](https://cal.com/maayansa/elementary-intro-github-package) or [start a trial](https://www.elementary-data.com/signup).
## Elementary tables - Run Results and dbt artifacts
112
+
113
+
## Elementary Tables - Run Results and dbt Artifacts
86
114
87
115
The **Elementary dbt package** automatically stores **dbt artifacts and run results** in your data warehouse, creating structured tables that provide visibility into your dbt runs and metadata.
88
116
89
-
### **Metadata Tables - dbt artifacts**
117
+
### **Metadata Tables - dbt Artifacts**
90
118
91
119
These tables provide a comprehensive view of your dbt project structure and configurations:
92
120
@@ -111,46 +139,34 @@ These tables track execution details, test outcomes, and performance metrics fro
111
139
112
140
For a full breakdown of these tables, see the [documentation](https://docs.elementary-data.com/dbt/package-models).
113
141
114
-
## Data anomaly detection as dbt tests
142
+
## Quickstart - dbt Package
115
143
116
-
**Elementary tests are configured and executed like native tests in your project!**
144
+
1. Add to your `packages.yml`:
117
145
118
-
Elementary dbt tests help track and alert on schema changes as well as key metrics and metadata over time, including freshness, volume, distribution, cardinality, and more.
146
+
```
147
+
packages:
148
+
- package: elementary-data/elementary
149
+
version: 0.17.0
150
+
## Docs: <https://docs.elementary-data.com>
119
151
120
-
**Seamlessly configured and run like native dbt tests,** Elementary tests detect anomalies and outliers, helping you catch data issues early.
152
+
```
121
153
122
-
Example of an Elementary test config in `schema.yml`:
154
+
2. Run `dbt deps`
155
+
3. Add to your `dbt_project.yml`:
123
156
124
157
```
125
158
models:
126
-
- name: your_model_name
127
-
config:
128
-
elementary:
129
-
timestamp_column: updated_at
130
-
tests:
131
-
- elementary.all_columns_anomalies
159
+
## elementary models will be created in the schema '<your_schema>_elementary'
160
+
## for details, see docs: <https://docs.elementary-data.com/>
161
+
elementary:
162
+
+schema: "elementary"
132
163
133
164
```
134
165
135
-
Elementary tests include:
136
-
137
-
### **Anomaly detection tests**
138
-
139
-
-**Volume anomalies -** Monitors the row count of your table over time per time bucket.
140
-
-**Freshness anomalies -** Monitors the freshness of your table over time, as the expected time between data updates.
141
-
-**Event freshness anomalies -** Monitors the freshness of event data over time, as the expected time it takes each event to load - that is, the time between when the event actually occurs (the **`event timestamp`**), and when it is loaded to the database (the **`update timestamp`**).
142
-
-**Dimension anomalies -** Monitors the count of rows grouped by given **`dimensions`** (columns/expressions).
143
-
-**Column anomalies -** Executes column level monitors on a certain column, with a chosen metric.
144
-
-**All columns anomalies** - Executes column level monitors and anomaly detection on all the columns of the table.
145
-
146
-
### **Schema tests**
166
+
4. Run `dbt run --select elementary`
147
167
148
-
-**Schema changes -** Alerts on a deleted table, deleted or added columns, or change of data type of a column.
149
-
-**Schema changes from baseline** - Checks for schema changes against baseline columns defined in a source’s or model’s configuration.
150
-
-**JSON schema** - Allows validating that a string column matches a given JSON schema.
151
-
-**Exposure validation test -** Detects changes in your models’ columns that break downstream exposure.
168
+
Check out the [full documentation](https://docs.elementary-data.com/).
152
169
153
-
Read more about the available [Elementary tests and configuration](https://docs.elementary-data.com/data-tests/introduction).
0 commit comments