Skip to content

Commit b2ef31a

Browse files
even-weiclaude
andcommitted
Update CI/CD documentation to use recce-cloud CLI
Replace GitHub Actions and GitLab CI/CD components with the new recce-cloud CLI for simpler, more direct CI/CD integration. Changes: - Replace DataRecce/recce-cloud-cicd-action with recce-cloud upload command - Replace GitLab component with direct CLI usage - Remove RECCE_API_TOKEN requirements (auto-detects GITHUB_TOKEN/CI_JOB_TOKEN) - Simplify authentication - no manual token setup needed - Add platform-specific verification images for GitHub/GitLab CI/CD - Update configuration examples to show CLI flags instead of component inputs Benefits: - Simpler setup without external dependencies - Consistent approach across GitHub and GitLab - Automatic platform and context detection - Direct pip install + CLI execution 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <[email protected]> Signed-off-by: even-wei <[email protected]>
1 parent bdb4eed commit b2ef31a

File tree

8 files changed

+82
-92
lines changed

8 files changed

+82
-92
lines changed

docs/7-cicd/github/setup-cd.md

Lines changed: 5 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -70,9 +70,10 @@ jobs:
7070
env:
7171
DBT_ENV_SECRET_KEY: ${{ secrets.DBT_ENV_SECRET_KEY }}
7272

73-
- name: Update Recce Cloud Base Session
74-
uses: DataRecce/[email protected]
75-
# This action automatically uploads artifacts to Recce Cloud
73+
- name: Upload to Recce Cloud
74+
run: |
75+
pip install recce-cloud
76+
recce-cloud upload --type prod
7677
```
7778
7879
### 2. Artifact Preparation Options
@@ -102,7 +103,7 @@ jobs:
102103
- ✅ **Workflow completes** without errors in Actions tab
103104
- ✅ **Base session updated** in Recce Cloud
104105

105-
![Recce Cloud showing updated base sessions](/assets/images/7-cicd/verify-setup-cd.png){: .shadow}
106+
![Recce Cloud showing updated base sessions](../../assets/images/7-cicd/verify-setup-github-cd.png){: .shadow}
106107

107108
## Next Steps
108109

docs/7-cicd/github/setup-ci.md

Lines changed: 5 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -70,9 +70,10 @@ jobs:
7070
env:
7171
DBT_ENV_SECRET_KEY: ${{ secrets.DBT_ENV_SECRET_KEY }}
7272

73-
- name: Update Recce PR Session
74-
uses: DataRecce/[email protected]
75-
# This action automatically creates a PR session in Recce Cloud
73+
- name: Upload to Recce Cloud
74+
run: |
75+
pip install recce-cloud
76+
recce-cloud upload
7677
```
7778
7879
### 2. Artifact Preparation Options
@@ -100,7 +101,7 @@ jobs:
100101
- ✅ **Workflow completes** without errors in Actions tab
101102
- ✅ **PR session updated** in Recce Cloud
102103

103-
![Recce Cloud showing PR validation session](/assets/images/7-cicd/verify-setup-ci.png){: .shadow}
104+
![Recce Cloud showing PR validation session](../../assets/images/7-cicd/verify-setup-github-ci.png){: .shadow}
104105

105106
#### Review PR Session
106107

docs/7-cicd/gitlab/setup-cd.md

Lines changed: 29 additions & 40 deletions
Original file line numberDiff line numberDiff line change
@@ -22,25 +22,15 @@ You need `manifest.json` and `catalog.json` files (dbt artifacts) for Recce Clou
2222

2323
### 1. Core Workflow
2424

25-
GitLab's CD setup uses the same Recce Cloud component as CI, but with different trigger rules. Add to your `.gitlab-ci.yml`:
25+
Add to your `.gitlab-ci.yml`:
2626
```yaml
27-
include:
28-
- component: gitlab.com/recce/recce-cloud-cicd-component/[email protected]
29-
inputs:
30-
stage: upload
31-
3227
stages:
3328
- build
3429
- upload
3530

3631
variables:
3732
DBT_TARGET_PROD: "prod"
3833

39-
# Disable the default component job
40-
recce-cloud-upload:
41-
rules:
42-
- when: never
43-
4434
# Production build - runs on schedule or manual trigger
4535
prod-build:
4636
stage: build
@@ -65,11 +55,14 @@ prod-build:
6555
when: manual
6656

6757
# Production Recce upload
68-
recce-cloud-upload-prod:
69-
extends: recce-cloud-upload
70-
needs:
71-
- job: prod-build
72-
artifacts: true
58+
recce-upload-prod:
59+
stage: upload
60+
image: python:3.11-slim
61+
script:
62+
- pip install recce-cloud
63+
- recce-cloud upload --type prod
64+
dependencies:
65+
- prod-build
7366
rules:
7467
- if: $CI_PIPELINE_SOURCE == "schedule"
7568
- if: $CI_PIPELINE_SOURCE == "push" && $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH
@@ -85,22 +78,12 @@ This configuration:
8578
8679
### 2. Unified CI/CD Configuration
8780
88-
The Recce Cloud component can handle both CI (MR validation) and CD (base session updates) in a single configuration. Here's the combined approach:
81+
You can handle both CI (MR validation) and CD (base session updates) in a single configuration:
8982
```yaml
90-
include:
91-
- component: gitlab.com/recce/recce-cloud-cicd-component/[email protected]
92-
inputs:
93-
stage: upload
94-
9583
stages:
9684
- build
9785
- upload
9886

99-
# Disable the default component job
100-
recce-cloud-upload:
101-
rules:
102-
- when: never
103-
10487
# MR build - runs on merge requests
10588
mr-build:
10689
stage: build
@@ -135,28 +118,34 @@ prod-build:
135118
- if: $CI_PIPELINE_SOURCE == "push" && $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH
136119

137120
# MR Recce upload
138-
recce-cloud-upload-mr:
139-
extends: recce-cloud-upload
140-
needs:
141-
- job: mr-build
142-
artifacts: true
121+
recce-upload-mr:
122+
stage: upload
123+
image: python:3.11-slim
124+
script:
125+
- pip install recce-cloud
126+
- recce-cloud upload
127+
dependencies:
128+
- mr-build
143129
rules:
144130
- if: $CI_PIPELINE_SOURCE == "merge_request_event"
145131

146132
# Production Recce upload
147-
recce-cloud-upload-prod:
148-
extends: recce-cloud-upload
149-
needs:
150-
- job: prod-build
151-
artifacts: true
133+
recce-upload-prod:
134+
stage: upload
135+
image: python:3.11-slim
136+
script:
137+
- pip install recce-cloud
138+
- recce-cloud upload --type prod
139+
dependencies:
140+
- prod-build
152141
rules:
153142
- if: $CI_PIPELINE_SOURCE == "schedule"
154143
- if: $CI_PIPELINE_SOURCE == "push" && $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH
155144
```
156145
157146
This unified approach:
158147
159-
- Uses the same component for both CI and CD
148+
- Uses the same `recce-cloud upload` command for both CI and CD
160149
- Separates MR validation from production updates via `rules`
161150
- Maintains different dbt targets for each environment
162151
- Reduces configuration duplication
@@ -200,7 +189,7 @@ To enable automatic baseline updates:
200189
- ✅ **Pipeline completes** without errors in CI/CD → Pipelines
201190
- ✅ **Base session updated** in Recce Cloud
202191

203-
![Recce Cloud showing updated base sessions](../../assets/images/7-cicd/verify-setup-cd.png){: .shadow}
192+
![Recce Cloud showing updated base sessions](../../assets/images/7-cicd/verify-setup-gitlab-cd.png){: .shadow}
204193

205194
#### Verify Scheduled Runs
206195

@@ -253,7 +242,7 @@ recce-cloud-upload-prod:
253242

254243
## Complete Example
255244

256-
See the [complete working example](https://gitlab.com/recce/jaffle-shop-snowflake/-/blob/main/.gitlab-ci.yml) showing unified CI/CD configuration with the Recce Cloud component.
245+
Here's a complete working example with unified CI/CD configuration using `recce-cloud` CLI. This example can be used as a reference for your own setup.
257246

258247
## Next Steps
259248

docs/7-cicd/gitlab/setup-ci.md

Lines changed: 43 additions & 44 deletions
Original file line numberDiff line numberDiff line change
@@ -24,14 +24,9 @@ You need `manifest.json` and `catalog.json` files (dbt artifacts) for Recce Clou
2424

2525
Add to your `.gitlab-ci.yml`:
2626
```yaml
27-
include:
28-
- component: gitlab.com/recce/recce-cloud-cicd-component/[email protected]
29-
inputs:
30-
stage: test
31-
3227
stages:
3328
- build
34-
- test
29+
- upload
3530

3631
variables:
3732
DBT_TARGET: "ci"
@@ -56,46 +51,47 @@ dbt-build:
5651
expire_in: 1 week
5752
rules:
5853
- if: $CI_PIPELINE_SOURCE == "merge_request_event"
54+
55+
recce-upload:
56+
stage: upload
57+
image: python:3.11-slim
58+
script:
59+
- pip install recce-cloud
60+
- recce-cloud upload
61+
dependencies:
62+
- dbt-build
63+
rules:
64+
- if: $CI_PIPELINE_SOURCE == "merge_request_event"
5965
```
6066
61-
The included Recce Cloud component automatically:
67+
The `recce-cloud upload` command automatically:
6268

69+
- Detects your GitLab CI environment
6370
- Creates a session in Recce Cloud for the merge request
6471
- Uploads your dbt artifacts (`manifest.json` and `catalog.json`)
65-
- Provides session URL for validation review
72+
- Links the session to your MR
73+
74+
### 2. Configuration Options
6675

67-
### 2. Component Configuration Options
76+
The `recce-cloud upload` command supports customization via flags:
6877

69-
The component accepts optional inputs for customization:
7078
```yaml
71-
include:
72-
- component: gitlab.com/recce/recce-cloud-cicd-component/[email protected]
73-
inputs:
74-
stage: test # Pipeline stage (default: test)
75-
dbt_target_path: target # Path to dbt artifacts (default: target)
76-
base_branch: main # Base branch for comparison (default: main)
77-
gitlab_token: $CUSTOM_GITLAB_TOKEN # Custom GitLab token (default: $CI_JOB_TOKEN)
79+
recce-upload:
80+
script:
81+
- pip install recce-cloud
82+
- recce-cloud upload --target-path custom-target --type cr
7883
```
7984

80-
**Default Configuration** (shown in example above):
85+
**Available options:**
8186

82-
- Component uses `$CI_JOB_TOKEN` automatically (no manual token setup required)
83-
- Uploads from `target/` directory by default
84-
- Compares against `main` branch
87+
- `--target-path`: Path to dbt artifacts directory (default: `target`)
88+
- `--type`: Session type - `cr`, `prod`, or `dev` (auto-detected by default)
89+
- `--cr`: Override MR number (auto-detected by default)
90+
- `--dry-run`: Preview what would be uploaded without actually uploading
8591

86-
**Custom Token** (optional):
92+
**Authentication:**
8793

88-
If you need to use a custom GitLab token instead of the default `$CI_JOB_TOKEN`:
89-
90-
1. Create a [Project Access Token](../gitlab-pat-guide.md) with `api` scope
91-
2. Add it as a [CI/CD variable](https://docs.gitlab.com/ee/ci/variables/) in your project
92-
3. Reference it in the component inputs:
93-
```yaml
94-
include:
95-
- component: gitlab.com/recce/recce-cloud-cicd-component/[email protected]
96-
inputs:
97-
gitlab_token: $CUSTOM_GITLAB_TOKEN
98-
```
94+
The command automatically uses your GitLab CI environment for authentication (via `CI_JOB_TOKEN`). No additional token setup required.
9995

10096
### 3. Artifact Preparation Options
10197

@@ -123,7 +119,7 @@ include:
123119
- ✅ **MR session updated** in Recce Cloud
124120
- ✅ **Session URL** appears in pipeline job output
125121

126-
![Recce Cloud showing MR validation session](../../assets/images/7-cicd/verify-setup-ci.png){: .shadow}
122+
![Recce Cloud showing MR validation session](../../assets/images/7-cicd/verify-setup-gitlab-ci.png){: .shadow}
127123

128124
#### Review MR Session
129125

@@ -187,24 +183,18 @@ dbt-build:
187183

188184
Here's a full working example combining dbt build and Recce validation:
189185
```yaml
190-
include:
191-
- component: gitlab.com/recce/recce-cloud-cicd-component/[email protected]
192-
inputs:
193-
stage: test
194-
195186
stages:
196187
- build
197-
- test
188+
- upload
198189
199190
variables:
200191
DBT_TARGET: "ci"
201192
202193
dbt-build:
203194
stage: build
204195
image: python:3.11-slim
205-
before_script:
206-
- pip install -r requirements.txt
207196
script:
197+
- pip install -r requirements.txt
208198
- dbt deps
209199
- dbt build --target $DBT_TARGET
210200
- dbt docs generate --target $DBT_TARGET
@@ -214,6 +204,15 @@ dbt-build:
214204
expire_in: 1 week
215205
rules:
216206
- if: $CI_PIPELINE_SOURCE == "merge_request_event"
217-
```
218207
219-
See the [complete example project](https://gitlab.com/recce/jaffle-shop-snowflake/-/blob/main/.gitlab-ci.yml) for a full working configuration.
208+
recce-upload:
209+
stage: upload
210+
image: python:3.11-slim
211+
script:
212+
- pip install recce-cloud
213+
- recce-cloud upload
214+
dependencies:
215+
- dbt-build
216+
rules:
217+
- if: $CI_PIPELINE_SOURCE == "merge_request_event"
218+
```
7.83 KB
Loading
89.4 KB
Loading
6.56 KB
Loading
41.6 KB
Loading

0 commit comments

Comments
 (0)