Skip to content

Commit bdb4eed

Browse files
gckogcko
authored
Merge pull request #50 from DataRecce/feature/pla-514-update-doc-for-supporting-super
update
2 parents d335b09 + 9be9bf1 commit bdb4eed

File tree

3 files changed

+138
-153
lines changed

3 files changed

+138
-153
lines changed

‎docs/2-getting-started/oss-vs-cloud.md‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,10 +1,10 @@
11
---
2-
title: Choose Your Approach
2+
title: Choose best for you
33
---
44

5-
# Choose Your Approach
5+
# Choose what's best for you
66

7-
Recce offers two ways to validate your data changes. **We recommend starting with Recce Cloud** for the easiest setup and team collaboration.
7+
Recce offers two ways to validate your data changes: Open Source and Recce Cloud. **We recommend starting with Recce Cloud** for the easiest setup and team collaboration.
88

99
## Quick Comparison
1010

‎docs/2-getting-started/start-free-with-cloud.md‎

Lines changed: 98 additions & 109 deletions
Original file line numberDiff line numberDiff line change
@@ -1,92 +1,77 @@
11
---
2-
title: Start Free with Cloud
2+
title: Start with Recce Cloud
33
---
44

5-
# Start Free with Recce Cloud
5+
# Start with Recce Cloud
66

7-
**Launch Recce in under 2 minutes**. Each following feature provides additional value. The progressive features help you
8-
get more value from Recce over time.
7+
Validate data changes and collaborate with your team with automation.
98

10-
👉 **[Start Free →](https://cloud.reccehq.com){target="_blank"}**
9+
👉 **[Start with Recce Cloud](https://cloud.reccehq.com){target="_blank"}**
1110

12-
## Model Changes and Impact Analysis
11+
Setup steps:
1312

14-
Recce shows what changed between **base** and **current** environments and helps assess potential impact. The most
15-
common case is comparing your development branch against your production or main branch to see what your changes will
16-
impact.
13+
1. [Git Platform Integration](#git-integration)
14+
2. [Data Warehouse Diffing](#data-diffing)
15+
3. [CI/CD Automation](#cicd-automation)
1716

18-
You can:
17+
Fall back to [manual](#manual) if you unable to finish the setup.
1918

20-
- Explore with the pre-loaded Jaffle Shop data
21-
- Upload your metadata (see below)
22-
- **Skip manual upload go directly to [CI/CD automation](#cicd-automation)**
2319

24-
<!-- insert a video -->
20+
## Prerequisites
21+
1. Admin access for git platform
22+
2. Data warehouse credentials with read access
23+
3.
2524

26-
### Upload Metadata
2725

28-
- Web interface: Click "Update" on the session you want to update in Recce Cloud.
29-
1. Click "Update" in base session to upload baseline metadata
30-
2. Click "Update" in current session to upload comparison metadata
31-
3. Click "Launch" to compare current against base
32-
- CLI command:
33-
34-
```
35-
recce upload-session --session-id <your_session_id>
36-
```
37-
38-
Find your session ID in Recce Cloud web interface when clicking "Update" on any session.
39-
40-
### Required Files
41-
42-
Recce needs `manifest.json` and `catalog.json` from both **base** and **current** environments for comparison.
26+
## Git Platform Integration {#git-integration}
4327

44-
#### Base Metadata
28+
Connect your GitHub or GitLab repository to see all PRs/MRs in one place and validate changes before they hit
29+
production.
4530

46-
Production environment is commonly used as the baseline, but any environment can serve as the base.
31+
### Setup Requirements
4732

48-
Choose one method:
33+
| GitHub | GitLab |
34+
|------------------------------------------------------------------------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------|
35+
| • GitHub repository with dbt project<br>• Repository admin access for initial setup<br>• Active PRs with model changes | • GitLab project with dbt project<br>• Project maintainer or owner access for initial setup<br>• Active merge requests with model changes |
4936

50-
**Method 1: Generate locally**
37+
!!!Note
38+
You'll need administrative access to the Organization/Group you want to connect. Please ensure you have the necessary
39+
permissions for **App installations** (GitHub) or **Providing a Personal Access Token** (GitLab).
5140

52-
```
53-
dbt docs generate --target-path target-base --target <your_prod_target>
54-
```
41+
### Connection
5542

56-
**Method 2: dbt Cloud**<br>
57-
Deploy → Jobs → Production job → Recent run → Download artifacts
43+
Connect your repository to track pull requests/merge requests and validate changes.
5844

59-
**Method 3: dbt Docs server**<br>
60-
Download the artifacts directly from dbt docs server:
45+
!!!Note
46+
Keep **Connection setup** note the same as before if there was one specific to this section.
6147

62-
- `<dbt_docs_url>/manifest.json`
63-
- `<dbt_docs_url>/catalog.json`
48+
**Connection setup:**
6449

65-
#### Current Metadata
50+
| GitHub | GitLab |
51+
|--------------------------------------------------------------------------------------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
52+
| 1. Navigate to settings<br>2. Connect GitHub repository<br>3. Authorize Recce access<br>4. Select repository | 1. Navigate to settings<br>2. Connect GitLab by providing a Personal Access Token ([see our directions here](../7-cicd/gitlab-pat-guide.md))<br>3. Connect a project by adding a GitLab Project or URL to a Recce Project |
6653

67-
Use development environment or PR branch as current to compare against the base.
6854

69-
Choose one method:
55+
### How to Use PR/MR Tracking
7056

71-
**Method 1: Generate locally**
57+
Once connected, Recce displays all open and draft PRs/MRs in your Recce Cloud project.
7258

73-
```
74-
dbt docs generate --target <your_dev_target>
75-
```
59+
### PR/MR Validation Workflow
7660

77-
**Method 2: dbt Cloud**<br>
78-
Deploy → Jobs → CI job → Recent run → Download artifacts
61+
- View open/draft PRs/MRs in dashboard
62+
- Select PR/MR to validate
63+
- Upload PR/MR metadata (until CI/CD is configured)
64+
- Launch Recce to analyze changes
7965

8066
## Data Warehouse Diffing {#data-diffing}
8167

82-
Go beyond metadata to see how changes affect your actual data. Configure your data warehouse connection to compare query
68+
Do data diffing to see how changes affect your actual data. Configure your data warehouse connection to compare query
8369
results and catch issues before they impact production.
8470

8571
### Setup Requirements
8672

8773
- Data warehouse credentials with read access
8874
- Network connectivity to your warehouse
89-
- Base and current environments configured in previous step
9075

9176
### Supported Warehouses
9277

@@ -100,66 +85,13 @@ Configure connection to your data warehouse to enable query result comparisons.
10085

10186
**Quick setup:**
10287

103-
1. In your [project home](https://cloud.datarecce.io/), click the gear icon next to **Warehouse Connection**
88+
1. In your [project home](https://cloud.reccehq.com/), click the gear icon next to **Warehouse Connection**
10489
2. Create a new connection or select an existing one from the dropdown
10590
3. Your connection is linked and ready to use
10691

10792
For detailed connection settings, see [Connect to Warehouse](../5-data-diffing/connect-to-warehouse.md). Connection
108-
credentials are encrypted and secure - see our [security practices](https://reccehq.com/security/).
109-
110-
<!-- insert a video -->
111-
112-
### How to Use Data Diffing
113-
114-
Recce supports several data diffing methods. See Data Diffing sections for details:
115-
116-
- [Row Count Diff](/5-data-diffing/row-count-diff)
117-
- [Profile Diff](/5-data-diffing/profile-diff/)
118-
- [Value Diff](/5-data-diffing/value-diff/)
119-
- [Top-K Diff](/5-data-diffing/topK-diff/)
120-
- [Histogram Diff](/5-data-diffing/histogram-diff/)
121-
- [Query](/5-data-diffing/query/)
93+
credentials are encrypted and secure, see our [security practices](https://reccehq.com/security/).
12294

123-
## Cloud-based Git Platform Integration {#git-integration}
124-
125-
Connect your GitHub or GitLab repository to see all PRs/MRs in one place and validate changes before they hit
126-
production.
127-
128-
### Setup Requirements
129-
130-
| GitHub | GitLab |
131-
|------------------------------------------------------------------------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------|
132-
| • GitHub repository with dbt project<br>• Repository admin access for initial setup<br>• Active PRs with model changes | • GitLab project with dbt project<br>• Project maintainer or owner access for initial setup<br>• Active merge requests with model changes |
133-
134-
!!!Note
135-
You'll need administrative access to the Organization/Group you want to connect. Please ensure you have the necessary
136-
permissions for **App installations** (GitHub) or **Providing a Personal Access Token** (GitLab).
137-
138-
### Connection
139-
140-
Connect your repository to track pull requests/merge requests and validate changes.
141-
142-
!!!Note
143-
Keep **Connection setup** note the same as before if there was one specific to this section.
144-
145-
**Connection setup:**
146-
147-
| GitHub | GitLab |
148-
|--------------------------------------------------------------------------------------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
149-
| 1. Navigate to settings<br>2. Connect GitHub repository<br>3. Authorize Recce access<br>4. Select repository | 1. Navigate to settings<br>2. Connect GitLab by providing a Personal Access Token ([see our directions here](../7-cicd/gitlab-pat-guide.md))<br>3. Connect a project by adding a GitLab Project or URL to a Recce Project |
150-
151-
<!-- insert a video -->
152-
153-
### How to Use PR/MR Tracking
154-
155-
Once connected, Recce displays all open and draft PRs/MRs in your dashboard.
156-
157-
### PR/MR Validation Workflow
158-
159-
- View open/draft PRs/MRs in dashboard
160-
- Select PR/MR to validate
161-
- Upload PR/MR metadata (until CI/CD is configured)
162-
- Launch Recce to analyze changes
16395

16496
## CI/CD Automation {#cicd-automation}
16597

@@ -189,5 +121,62 @@ See the CI/CD sections for complete setup guides:
189121
- Validation results directly in PR
190122

191123

124+
## Manual Upload Metadata {#manual}
125+
126+
Recce shows what changed between base and current environments and helps assess potential impact. The most common case is comparing your development branch against your production or main branch to see what your changes will impact.
127+
128+
If you don’t use the GitHub/GitLab or havn’t setup CI/CD yet, you can manual upload
129+
130+
- Web interface: Click "Update" on the session you want to update in Recce Cloud.
131+
1. Click "Update" in base session to upload baseline metadata
132+
2. Click "Update" in current session to upload comparison metadata
133+
3. Click "Launch" to compare current against base
134+
- CLI command:
135+
136+
```
137+
recce upload-session --session-id <your_session_id>
138+
```
139+
140+
Find your session ID in Recce Cloud web interface when clicking "Update" on any session.
141+
142+
### Required Files
143+
144+
Recce needs `manifest.json` and `catalog.json` from both **base** and **current** environments for comparison.
145+
146+
#### Base Metadata
147+
148+
Production environment is commonly used as the baseline, but any environment can serve as the base.
149+
150+
Choose one method:
151+
152+
**Method 1: Generate locally**
153+
154+
```
155+
dbt docs generate --target-path target-base --target <your_prod_target>
156+
```
157+
158+
**Method 2: dbt Cloud**<br>
159+
Deploy → Jobs → Production job → Recent run → Download artifacts
160+
161+
**Method 3: dbt Docs server**<br>
162+
Download the artifacts directly from dbt docs server:
163+
164+
- `<dbt_docs_url>/manifest.json`
165+
- `<dbt_docs_url>/catalog.json`
166+
167+
#### Current Metadata
168+
169+
Use development environment or PR branch as current to compare against the base.
170+
171+
Choose one method:
172+
173+
**Method 1: Generate locally**
174+
175+
```
176+
dbt docs generate --target <your_dev_target>
177+
```
178+
179+
**Method 2: dbt Cloud**<br>
180+
Deploy → Jobs → CI job → Recent run → Download artifacts
192181

193182

‎docs/7-cicd/ci-cd-getting-started.md‎

Lines changed: 37 additions & 41 deletions
Original file line numberDiff line numberDiff line change
@@ -10,82 +10,78 @@ Automate data validation in your development workflow. Catch data issues before
1010

1111
Set up automated workflows that:
1212

13-
- **Maintain current baselines** - Auto-update comparison baselines on every merge to main
14-
- **Validate every PR/MR** - Run data validation checks automatically when changes are proposed
13+
- **Save time on reviews** - Eliminate manual validation steps for every change
14+
- **Run data validations on every pull request/merge request** - Run data validation checks automatically when changes are proposed
1515
- **Prevent regressions** - Catch data quality issues before they reach production
16-
- **Save team time** - Eliminate manual validation steps for every change
1716

1817
!!!note
19-
CI/CD automation requires Recce Cloud Team plan. A free trial is available.
18+
CI/CD automation requires a Cloud Plan. Get started for free [here](https://cloud.reccehq.com/).
2019

21-
## Understanding CI vs CD
20+
## What is CI/CD?
2221

23-
Recce uses both continuous integration and continuous delivery to automate data validation:
22+
Recce uses both continuous integration (CI) and continuous delivery (CD) to automate data validation:
2423

2524
**Continuous Integration (CI)**
2625

27-
- **When**: Runs on every PR/MR update
28-
- **Purpose**: Validates proposed changes against baseline
26+
- **When**: Runs when you open a new or update a Pull Request/Merge Request
27+
- **Purpose**: Validates proposed changes against baseline (typically this mean production)
2928
- **Benefit**: Catches issues before merge, with results in your PR/MR
3029

3130
**Continuous Delivery (CD)**
3231

3332
- **When**: Runs after merge to main branch
34-
- **Purpose**: Updates your baseline Recce session with latest production state
33+
- **Purpose**: Updates baseline artifacts Recce uses to with latest production state
3534
- **Benefit**: Ensures future comparisons use current baseline
3635

37-
## Choose your platform
36+
## What does look like with Recce?
3837

39-
Recce integrates with both GitHub Actions and GitLab CI/CD.
40-
41-
Select your Git platform to get started:
38+
Both CI and CD workflows follow the same pattern:
4239

43-
### GitHub
44-
If your dbt project uses GitHub:
40+
1. **Trigger event** (merge to main, or PR/MR opened/updated)
41+
2. **Generate dbt artifacts** (`dbt docs generate` or external source)
42+
3. **Upload to Recce Cloud** (automatic via workflow action)
43+
4. **Validation results** appear in Recce dashboard and PR/MR
4544

46-
1. [Setup CI](./github/setup-ci.md) - Auto-validate changes in every PR
47-
2. [Setup CD](./github/setup-cd.md) - Auto-update baseline on merge to main
45+
<figure markdown>
46+
![Recce CI/CD architecture](../assets/images/7-cicd/ci-cd.png){: .shadow}
47+
<figcaption>Automated validation workflow for pull requests</figcaption>
48+
</figure>
4849

49-
### GitLab
50-
If your dbt project uses GitLab:
50+
## Getting Started with your CI/CD
5151

52-
2. [Setup CI](./gitlab/setup-ci.md) - Auto-validate changes in every MR
53-
1. [Setup CD](./gitlab/setup-cd.md) - Auto-update baseline on merge to main
54-
3. [GitLab Personal Access Token Guide](./gitlab/gitlab-pat-guide.md) - Required for GitLab integration
52+
Recce currently integrates with both GitHub Actions and GitLab CI/CD. If you use another CI/CD product and interested in Recce, [let us know](https://cal.com/team/recce/chat).
5553

5654
## Prerequisites
5755

5856
Before setting up, ensure you have:
5957

60-
- **Recce Cloud account** with Team plan or free trial
58+
- **Recce Cloud account** You can signup and start your free trial [here](https://cloud.reccehq.com/)
6159
- **Repository connected** to Recce Cloud ([setup guide](../2-getting-started/start-free-with-cloud.md#git-integration))
62-
- **dbt artifacts** (`manifest.json` and `catalog.json`) from your project
60+
- **dbt artifacts generated** (`manifest.json` and `catalog.json`) from your project
6361

64-
## Architecture overview
62+
### GitHub
63+
If your dbt project uses GitHub:
6564

66-
Both CI and CD workflows follow the same pattern:
65+
1. [Setup CD](./github/setup-cd.md) - Auto-update baseline on merge to main
66+
2. [Setup CI](./github/setup-ci.md) - Auto-validate changes in every PR
6767

68-
1. **Trigger event** (merge to main, or PR/MR opened/updated)
69-
2. **Generate dbt artifacts** (`dbt docs generate` or external source)
70-
3. **Upload to Recce Cloud** (automatic via workflow action)
71-
4. **Validation results** appear in Recce dashboard and PR/MR
68+
### GitLab
69+
If your dbt project uses GitLab:
7270

73-
<figure markdown>
74-
![Recce CI/CD architecture](../assets/images/7-cicd/ci-cd.png){: .shadow}
75-
<figcaption>Automated validation workflow for pull requests</figcaption>
76-
</figure>
71+
1. [Setup CD](./gitlab/setup-cd.md) - Auto-update baseline on merge to main
72+
2. [Setup CI](./gitlab/setup-ci.md) - Auto-validate changes in every MR
73+
3. [GitLab Personal Access Token Guide](./gitlab/gitlab-pat-guide.md) - Required for GitLab integration
7774

7875
## Next steps
7976

80-
1. Choose your platform (GitHub or GitLab)
81-
2. Start with CD setup to establish baseline updates
82-
3. Add CI setup to enable PR/MR validation
83-
4. Review [best practices](./best-practices-prep-env.md) for environment preparation
77+
1. Start with relevant CD setup ([Gitlab](./gitlab/setup-cd.md) or [Github](./github/setup-cd.md)) to establish automatic baseline (production artifacts) updates.
78+
2. Configure CI setup ([Gitlab](./gitlab/setup-ci.md) or [Github](./github/setup-ci.md)) to enable PR/MR validation
79+
3. Review [best practices](./best-practices-prep-env.md) for environment preparation
8480

8581
## Related workflows
8682

8783
After setting up CI/CD automation, explore these workflow guides:
8884

89-
- [Development workflow](./scenario-dev.md) - Validate changes during development
90-
- [PR/MR review workflow](./scenario-pr-review.md) - Collaborate on validation results
91-
- [Preset checks](./preset-checks.md) - Configure automatic validation checks
85+
- [Development workflow](./scenario-dev.md) - How to validate data impact during development (pre-PR/MR)
86+
- [PR/MR review workflow](./scenario-pr-review.md) - How to collaborate with teammates using Recce in PRs/MRs
87+
- [Preset checks](./preset-checks.md) - How to configure automatic validation checks

0 commit comments

Comments
 (0)