Skip to content

Commit 3501fbc

Browse files
committed
Update README with AI-powered data validation section
Add documentation for AI-powered data validation and unstructured data tests, including an example configuration and a link to further documentation. Enhance the README's overview of Elementary's capabilities.
1 parent ade2ddd commit 3501fbc

File tree

1 file changed

+28
-3
lines changed

1 file changed

+28
-3
lines changed

README.md

Lines changed: 28 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -23,6 +23,7 @@ Available as self-hosted or Cloud service with premium features.
2323
- [Get more out of Elementary](#get-more-out-of-elementary-dbt-package)
2424
- [Run results and dbt artifacts](#run-results-and-dbt-artifacts)
2525
- [Data anomaly detection as dbt tests](#data-anomaly-detection-as-dbt-tests)
26+
- [AI-powered data validation and unstructured data tests](#ai-powered-data-validation-and-unstructured-data-tests)
2627
- [How Elementary works?](#how-elementary-works)
2728
- [Community & Support](#community--support)
2829
- [Contribution](#contributions)
@@ -61,9 +62,9 @@ Elementary has 3 offerings: This dbt package, Elementary Community (OSS) and Ele
6162
- **dbt package**
6263
- For basic data monitoring and dbt artifacts collection, Elementary offers a dbt package. The package adds logging, artifacts uploading, and Elementary tests (anomaly detection and schema) to your project.
6364
- **Elementary Community**
64-
- An open-source CLI tool you can deploy and orchestrate to send alerts and self-host the Elementary report. Best for data and analytics engineers that require basic observability capabilities or for evaluating features without vendor approval. Our community can provide great support on [Slack](https://www.elementary-data.com/community) if needed.
65+
- An open-source CLI tool you can deploy and orchestrate to send alerts and self-host the Elementary report. Best for data and analytics engineers that require basic observability capabilities or for evaluating features without vendor approval. Our community can provide great support on [Slack](https://www.elementary-data.com/community) if needed.
6566
- **Elementary Cloud**
66-
- Ideal for teams monitoring mission-critical data pipelines, requiring guaranteed uptime and reliability, short-time-to-value, advanced features, collaboration, and professional support. The solution is secure by design, and requires no access to your data from cloud. To learn more, [book a demo](https://cal.com/maayansa/elementary-intro-github-package) or [start a trial](https://www.elementary-data.com/signup).
67+
- Ideal for teams monitoring mission-critical data pipelines, requiring guaranteed uptime and reliability, short-time-to-value, advanced features, collaboration, and professional support. The solution is secure by design, and requires no access to your data from cloud. To learn more, [book a demo](https://cal.com/maayansa/elementary-intro-github-package) or [start a trial](https://www.elementary-data.com/signup).
6768

6869
## Run Results and dbt artifacts
6970

@@ -110,6 +111,30 @@ models:
110111

111112
Read about the available [Elementary tests and configuration](https://docs.elementary-data.com/data-tests/introduction).
112113

114+
## AI-powered data validation and unstructured data tests
115+
116+
Elementary leverages AI to enhance data reliability with natural language test definitions:
117+
118+
- **AI data validation**: Define expectations in plain English to validate structured data
119+
- **Unstructured data validation**: Validate text, JSON, and other non-tabular data types
120+
121+
Example:
122+
123+
```yml
124+
# AI data validation example
125+
models:
126+
- name: crm
127+
description: "A table containing contract details."
128+
columns:
129+
- name: contract_date
130+
description: "The date when the contract was signed."
131+
tests:
132+
- elementary.ai_data_validation:
133+
expectation_prompt: "There should be no contract date in the future"
134+
```
135+
136+
Learn more in our [AI data validation documentation](hhttps://docs.elementary-data.com/data-tests/ai-data-tests/ai_data_validations).
137+
113138
## How Elementary works?
114139

115140
Elementary dbt package creates tables of metadata and test results in your data warehouse, as part of your dbt runs.
@@ -127,6 +152,6 @@ The cloud service or the CLI tool read the data from these tables, send alerts a
127152

128153
## Contributions
129154

130-
Thank you :orange_heart: Whether its a bug fix, new feature, or additional documentation - we greatly appreciate contributions!
155+
Thank you :orange_heart: Whether it's a bug fix, new feature, or additional documentation - we greatly appreciate contributions!
131156

132157
Check out the [contributions guide](https://docs.elementary-data.com/general/contributions) and [open issues](https://github.com/elementary-data/elementary/issues) in the main repo.

0 commit comments

Comments
 (0)