Skip to content

Ai data tests docs edits #1854

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
MikaKerman merged 2 commits into from
Mar 12, 2025
Merged

Ai data tests docs edits #1854

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
1475141
Add version requirement note to AI and unstructured data validation docs
oravi Mar 12, 2025
dd5e415
Reorganize AI data tests documentation with platform-specific guides
oravi Mar 12, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 6 additions & 4 deletions docs/data-tests/ai-data-tests/ai_data_validations.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,8 @@ title: "AI Data Validations"

<Note type="warning">
**Beta Feature**: AI data validation tests is currently in beta. The functionality and interface may change in future releases.

**Version Requirement**: This feature requires Elementary dbt package version 0.18.0 or above.
</Note>

# AI Data Validation with Elementary
Expand All @@ -29,24 +31,24 @@ Before you can use Elementary's AI data validations, you need to set up AI and L
### Snowflake
- **Prerequisite**: Enable Snowflake Cortex AI LLM functions
- **Recommended Model**: `claude-3-5-sonnet`
- [View Snowflake's Guide](/data-tests/unstructured-data-tests/snowflake)
- [View Snowflake's Guide](/data-tests/ai-data-tests/supported-platforms/snowflake)

### Databricks
- **Prerequisite**: Ensure Databricks AI Functions are available
- **Recommended Model**: `databricks-meta-llama-3-3-70b-instruct`
- [View Databrick's Setup Guide](/data-tests/unstructured-data-tests/databricks)
- [View Databrick's Setup Guide](/data-tests/ai-data-tests/supported-platforms/databricks)

### BigQuery
- **Prerequisite**: Configure BigQuery to use Vertex AI models
- **Recommended Model**: `gemini-1.5-pro`
- [View BigQuery's Setup Guide](/data-tests/unstructured-data-tests/bigquery)
- [View BigQuery's Setup Guide](/data-tests/ai-data-tests/supported-platforms/bigquery)

### Redshift
- Support coming soon

### Data Lakes
- Currently supported through Snowflake, Databricks, or BigQuery external object tables
- [View Data Lakes Information](/data-tests/unstructured-data-tests/data-lakes)
- [View Data Lakes Information](/data-tests/ai-data-tests/supported-platforms/data-lakes)

## Using the AI Data Validation Test

Expand Down
0 docs/data-tests/ai-data-tests/bigquery.mdx → ...ta-tests/supported-platforms/bigquery.mdx
View file
Open in desktop
File renamed without changes.
0 docs/data-tests/ai-data-tests/data-lakes.mdx → ...-tests/supported-platforms/data-lakes.mdx
View file
Open in desktop
File renamed without changes.
0 docs/data-tests/ai-data-tests/databricks.mdx → ...-tests/supported-platforms/databricks.mdx
View file
Open in desktop
File renamed without changes.
0 docs/data-tests/ai-data-tests/redshift.mdx → ...ta-tests/supported-platforms/redshift.mdx
View file
Open in desktop
File renamed without changes.
0 docs/data-tests/ai-data-tests/snowflake.mdx → ...a-tests/supported-platforms/snowflake.mdx
View file
Open in desktop
File renamed without changes.
10 changes: 6 additions & 4 deletions docs/data-tests/ai-data-tests/unstructured_data_validations.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,8 @@ title: "Unstructured Data Validations"

<Note type="warning">
**Beta Feature**: Unstructured data validation tests is currently in beta. The functionality and interface may change in future releases.

**Version Requirement**: This feature requires Elementary dbt package version 0.18.0 or above.
</Note>

# Validating Unstructured Data with Elementary
Expand All @@ -29,24 +31,24 @@ Before you can use Elementary's unstructured data validations, you need to set u
### Snowflake
- **Prerequisite**: Enable Snowflake Cortex AI LLM functions
- **Recommended Model**: `claude-3-5-sonnet`
- [View Snowflake's Guide](/data-tests/unstructured-data-tests/snowflake)
- [View Snowflake's Guide](/data-tests/ai-data-tests/supported-platforms/snowflake)

### Databricks
- **Prerequisite**: Ensure Databricks AI Functions are available
- **Recommended Model**: `databricks-meta-llama-3-3-70b-instruct`
- [View Databrick's Setup Guide](/data-tests/unstructured-data-tests/databricks)
- [View Databrick's Setup Guide](/data-tests/ai-data-tests/supported-platforms/databricks)

### BigQuery
- **Prerequisite**: Configure BigQuery to use Vertex AI models
- **Recommended Model**: `gemini-1.5-pro`
- [View BigQuery's Setup Guide](/data-tests/unstructured-data-tests/bigquery)
- [View BigQuery's Setup Guide](/data-tests/ai-data-tests/supported-platforms/bigquery)

### Redshift
- Support coming soon

### Data Lakes
- Currently supported through Snowflake, Databricks, or BigQuery external object tables
- [View Data Lakes Information](/data-tests/unstructured-data-tests/data-lakes)
- [View Data Lakes Information](/data-tests/ai-data-tests/supported-platforms/data-lakes)


## Using the Validation Test
Expand Down
15 changes: 10 additions & 5 deletions docs/mint.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -374,11 +374,16 @@
"pages": [
"data-tests/ai-data-tests/ai_data_validations",
"data-tests/ai-data-tests/unstructured_data_validations",
"data-tests/ai-data-tests/snowflake",
"data-tests/ai-data-tests/databricks",
"data-tests/ai-data-tests/bigquery",
"data-tests/ai-data-tests/redshift",
"data-tests/ai-data-tests/data-lakes"
{
"group": "Supported Platforms",
"pages": [
"data-tests/ai-data-tests/supported-platforms/snowflake",
"data-tests/ai-data-tests/supported-platforms/databricks",
"data-tests/ai-data-tests/supported-platforms/bigquery",
"data-tests/ai-data-tests/supported-platforms/redshift",
"data-tests/ai-data-tests/supported-platforms/data-lakes"
]
}
]
},
{
Expand Down
Loading