Skip to content

Commit ef79ef4

Browse files
even-weiclaude
even-wei
and
claude
committed
Update CI/CD documentation with simplified Recce Cloud integration
- Enhanced setup-cd.md with scheduled runs and concurrency control - Added comprehensive setup-ci.md for PR validation - Streamlined workflows using recce-cloud-cicd-action - Added verification screenshots for both CD and CI setup - Simplified artifact preparation with fresh build as default - Removed complex configuration in favor of automated GitHub token auth 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]> Signed-off-by: even-wei <[email protected]>
1 parent 361cfdd commit ef79ef4

File tree

5 files changed

+222
-8
lines changed

5 files changed

+222
-8
lines changed

‎docs/7-cicd/setup-cd.md‎

Lines changed: 104 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,5 +2,108 @@
22
title: Setup CD
33
---
44

5-
coming soon
5+
# Setup CD
66

7+
Automated continuous deployment (CD) for Recce Cloud base session updates. Ensures your comparison baseline stays current without manual intervention.
8+
9+
> **Note**: This guide uses `recce-cloud-cicd-action` which is specifically designed for Recce Cloud integration.
10+
11+
## Purpose
12+
13+
**Automated Base Session Management** → Eliminates manual baseline maintenance
14+
15+
- **Triggers**: PR merge to main + scheduled updates
16+
- **Action**: Auto-update base Recce session
17+
- **Benefit**: Current comparison baseline for future PRs
18+
19+
## Implementation
20+
21+
### 1. Core Workflow
22+
23+
Create `.github/workflows/cd-workflow.yml`:
24+
25+
```yaml
26+
name: Update Base Recce Session
27+
28+
on:
29+
push:
30+
branches: ["main"]
31+
schedule:
32+
- cron: "0 2 * * *" # Daily at 2 AM UTC
33+
workflow_dispatch:
34+
35+
concurrency:
36+
group: ${{ github.workflow }}
37+
cancel-in-progress: true
38+
39+
jobs:
40+
update-base-session:
41+
runs-on: ubuntu-latest
42+
timeout-minutes: 30
43+
44+
steps:
45+
- name: Checkout code
46+
uses: actions/checkout@v4
47+
48+
- name: Setup Python
49+
uses: actions/setup-python@v5
50+
with:
51+
python-version: "3.11"
52+
cache: "pip"
53+
54+
- name: Install dependencies
55+
run: |
56+
pip install -r requirements.txt
57+
58+
- name: Prepare dbt artifacts
59+
run: |
60+
# Install dbt packages
61+
dbt deps
62+
63+
# Optional: Build tables to ensure they're materialized and updated
64+
# dbt build --target prod
65+
66+
# Required: Generate artifacts (provides all we need)
67+
dbt docs generate --target prod
68+
env:
69+
DBT_ENV_SECRET_KEY: ${{ secrets.DBT_ENV_SECRET_KEY }}
70+
71+
- name: Update Recce Cloud Base Session
72+
uses: DataRecce/[email protected]
73+
# This action automatically uploads artifacts to Recce Cloud
74+
```
75+
76+
### 2. Artifact Preparation Options
77+
78+
**Default: Fresh Build** (shown in example above)
79+
80+
- `dbt docs generate` → Required (provides all needed artifacts)
81+
- `dbt build` → Optional (ensures tables are materialized and updated)
82+
83+
**Alternative Methods:**
84+
85+
- **External Download**: Download from dbt Cloud, Paradime, or other platforms
86+
- **Pipeline Integration**: Use existing dbt build workflows
87+
88+
**Key Requirement**: `manifest.json` and `catalog.json` available for Recce Cloud upload.
89+
90+
### 3. Verification
91+
92+
#### Manual Trigger Test
93+
94+
1. Go to **Actions** tab in your repository
95+
2. Select "Update Base Recce Session" workflow
96+
3. Click **Run workflow** button
97+
4. Monitor the run for successful completion
98+
99+
#### Verify Success
100+
101+
- ✅ **Workflow completes** without errors in Actions tab
102+
- ✅ **Base session updated** in Recce Cloud dashboard
103+
104+
![Recce Cloud dashboard showing updated base sessions](../assets/images/7-cicd/verify-setup-cd.png){: .shadow}
105+
106+
## Next Steps
107+
108+
- ✅ **CD Setup Complete** → Base sessions auto-update on merge + schedule
109+
- 🔄 **Add CI**: [Setup CI](./setup-ci.md) for PR validation

‎docs/7-cicd/setup-ci.md‎

Lines changed: 111 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,111 @@
1+
---
2+
title: Setup CI
3+
---
4+
5+
# Setup CI
6+
7+
Automated data validation for pull requests using Recce Cloud. Catch data issues before merging to main.
8+
9+
> **Note**: This guide shows CI integration with Recce Cloud for centralized validation management.
10+
11+
## Purpose
12+
13+
**Automated PR Validation** → Prevent data regressions before merge
14+
15+
- **Triggers**: PR opened/updated against main
16+
- **Action**: Auto-update Recce session for PR validation
17+
- **Benefit**: Automated data validation and comparison
18+
19+
## Implementation
20+
21+
### 1. Core Workflow
22+
23+
Create `.github/workflows/ci-workflow.yml`:
24+
25+
```yaml
26+
name: Validate PR Changes
27+
28+
on:
29+
pull_request:
30+
branches: ["main"]
31+
32+
concurrency:
33+
group: ${{ github.workflow }}-${{ github.ref }}
34+
cancel-in-progress: true
35+
36+
jobs:
37+
validate-changes:
38+
runs-on: ubuntu-latest
39+
timeout-minutes: 45
40+
41+
steps:
42+
- name: Checkout PR branch
43+
uses: actions/checkout@v4
44+
with:
45+
fetch-depth: 2
46+
47+
- name: Setup Python
48+
uses: actions/setup-python@v5
49+
with:
50+
python-version: "3.11"
51+
cache: "pip"
52+
53+
- name: Install dependencies
54+
run: |
55+
pip install -r requirements.txt
56+
57+
# Step 1: Prepare current branch artifacts
58+
- name: Build current branch artifacts
59+
run: |
60+
# Install dbt packages
61+
dbt deps
62+
63+
# Optional: Build tables to ensure they're materialized
64+
# dbt build --target ci
65+
66+
# Required: Generate artifacts for comparison
67+
dbt docs generate --target ci
68+
env:
69+
DBT_ENV_SECRET_KEY: ${{ secrets.DBT_ENV_SECRET_KEY }}
70+
71+
- name: Update Recce PR Session
72+
uses: DataRecce/[email protected]
73+
# This action automatically creates a PR session in Recce Cloud
74+
```
75+
76+
### 2. Artifact Preparation Options
77+
78+
**Default: Fresh Build** (shown in example above)
79+
80+
- `dbt docs generate` → Required (provides all needed artifacts)
81+
- `dbt build` → Optional (ensures tables are materialized and updated)
82+
83+
**Alternative Methods:**
84+
85+
- **External Download**: Download from dbt Cloud, Paradime, or other platforms
86+
- **Pipeline Integration**: Use existing dbt build workflows
87+
88+
**Key Requirement**: `manifest.json` and `catalog.json` available for Recce Cloud action.
89+
90+
### 3. Verification
91+
92+
#### Test with a PR
93+
94+
1. Create a test PR with small data changes
95+
2. Check **Actions** tab for CI workflow execution
96+
3. Verify validation runs successfully
97+
98+
#### Verify Success
99+
100+
- ✅ **Workflow completes** without errors in Actions tab
101+
- ✅ **PR session updated** in Recce Cloud dashboard
102+
103+
![Recce Cloud showing PR validation session](../assets/images/7-cicd/verify-setup-ci.png){: .shadow}
104+
105+
#### Review PR Session
106+
107+
To analyze the PR changes in detail:
108+
109+
- Go to your Recce Cloud dashboard
110+
- Find the PR session that was created
111+
- Launch Recce instance to explore data differences

‎docs/assets/images/7-cicd/verify-setup-cd.png‎

72.5 KB
Loading

‎docs/assets/images/7-cicd/verify-setup-ci.png‎

48.8 KB
Loading

‎mkdocs.yml‎

Lines changed: 7 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -73,17 +73,17 @@ nav:
7373
- 6-collaboration/checklist.md
7474
- 6-collaboration/share.md
7575
- CI/CD:
76-
#- 7-cicd/setup-cd.md
76+
- 7-cicd/setup-cd.md
7777
- Development:
78-
- 7-cicd/scenario-dev.md
78+
- 7-cicd/scenario-dev.md
7979
#- 7-cicd/recce-debug.md # content outdated
8080
- Continuous Integration:
81-
#- 7-cicd/setup-ci.md
82-
#- 7-cicd/cloud-recce-summary.md
83-
#- 7-cicd/cloud-preset-checks.md
84-
- 7-cicd/scenario-pr-review.md
81+
- 7-cicd/setup-ci.md
82+
#- 7-cicd/cloud-recce-summary.md
83+
#- 7-cicd/cloud-preset-checks.md
84+
- 7-cicd/scenario-pr-review.md
8585
- CI/CD for Open Source:
86-
- 7-cicd/scenario-ci.md
86+
- 7-cicd/scenario-ci.md
8787
#- 7-cicd/recce-summary.md content outdated
8888
- 7-cicd/preset-checks.md
8989
- 7-cicd/best-practices-prep-env.md

0 commit comments

Comments
 (0)