Skip to content

Commit 98be497

Browse files
committed
Remove dbt-audit-helper and dbt-profiler related info
Signed-off-by: even-wei <[email protected]>
1 parent 558a48a commit 98be497

File tree

3 files changed

+20
-75
lines changed

3 files changed

+20
-75
lines changed

docs/features/lineage.md

Lines changed: 19 additions & 47 deletions
Original file line numberDiff line numberDiff line change
@@ -1,22 +1,21 @@
11
---
22
title: Lineage
33
icon: material/file-tree
4-
54
---
65

76
The Lineage Diff is the main interface to Recce and allows you to quickly see the potential area of impact from your dbt data modeling changes.
87

98
## Lineage Diff
10-
It's from the Lineage Diff that you will determine which models to investigate further; and also perform the various data validation checks that will serve as proof-of-correctness of your work.
119

10+
It's from the Lineage Diff that you will determine which models to investigate further; and also perform the various data validation checks that will serve as proof-of-correctness of your work.
1211

1312
<figure markdown>
1413
![Recce Lineage Diff](../assets/images/features/lineage-diff.gif){: .shadow}
1514
<figcaption>Lineage Diff</figcaption>
1615
</figure>
1716

18-
1917
### Node Summary
18+
2019
![](../assets/images/features/node.png){: .shadow}
2120

2221
Models are color-coded to indicate their **status**:
@@ -42,56 +41,52 @@ The two icons at the bottom right of each node indicate if a `row count` or `sch
4241
Click a model to open the [node details](#node-detail) panel and perform other data validation checks.
4342

4443
### Filter Nodes
44+
4545
In the top control bar, you can change the rule to filter the nodes:
4646

4747
1. **Mode:**
48-
- **Changed Models:** Modified nodes and their downstream + 1st degree of their parents.
49-
- **All:** Show all nodes.
48+
- **Changed Models:** Modified nodes and their downstream + 1st degree of their parents.
49+
- **All:** Show all nodes.
5050
1. **Package:** Filter by dbt package names.
5151
1. **Select:** Select nodes by [node selection](./node-selection.md).
5252
1. **Exclude:** Exclude nodes by [node selection](./node-selection.md).
5353

5454
### Select Nodes
5555

56-
Click a node to select it, or click the **Select nodes** button at the top-right corner to select multiple nodes for further operations. For detail, see the [Multi Nodes Selections](#multi-nodes-selection) section
56+
Click a node to select it, or click the **Select nodes** button at the top-right corner to select multiple nodes for further operations. For detail, see the [Multi Nodes Selections](#multi-nodes-selection) section
5757

5858
### Row Count Diff
5959

6060
A row count diff can be performed on nodes selected using the `select` and `exclude` options:
6161

6262
![](../assets/images/features/row-count-diff-selector.gif){: .shadow}
6363

64-
6564
After selecting nodes, run the row count diff by:
6665

6766
1. Clicking the 3 dots (**...**) button at the top-right corner.
6867
2. Clicking **Row Count Diff by Selector**.
6968

70-
7169
## Node Details
7270

73-
The node details panel shows information about a node, such as node type, schema and row count changes, and allows you to perform diffs on the node using the options accessed via the `Explore Change` button.
71+
The node details panel shows information about a node, such as node type, schema and row count changes, and allows you to perform diffs on the node using the options accessed via the `Explore Change` button.
7472

7573
### Schema Diff
7674

7775
Schema Diff shows added, removed, and renamed columns. Click a model in the Lineage Diff to open the node details and view the Schema Diff.
7876

7977
!!! Note
80-
Schema Diff requires `catalog.json` in both environments.
81-
78+
Schema Diff requires `catalog.json` in both environments.
8279

8380
<figure markdown>
8481
![Recce Schema Diff](../assets/images/features/schema-diff.gif){: .shadow}
8582
<figcaption>Schema Diff</figcaption>
8683
</figure>
8784

88-
8985
<figure markdown>
9086
![Recce Schema Diff](../assets/images/features/schema-diff.png)
9187
<figcaption>Schema Diff showing renamed column</figcaption>
9288
</figure>
9389

94-
9590
### Row Count Diff
9691

9792
Row Count Diff shows the difference in row count between the base and current environments.
@@ -121,7 +116,6 @@ Value Diff shows the matched count and percentage for each column in the table.
121116

122117
The primary key is automatically inferred by the first column with the [unique](https://docs.getdbt.com/reference/resource-properties/data-tests#unique) test. If no primary key is detected at least one column is required to be specified as the primary key.
123118

124-
125119
<figure markdown>
126120
![Recce Value Diff](../assets/images/features/value-diff.png)
127121
<figcaption>Value Diff</figcaption>
@@ -132,17 +126,6 @@ The primary key is automatically inferred by the first column with the [unique](
132126
- **Matched**: For a column, the count of matched value of common PKs.
133127
- **Matched %**: For a column, the ratio of matched over common PKs.
134128

135-
!!! note
136-
137-
Value Diff uses the `compare_column_values` from [audit-helper](https://hub.getdbt.com/dbt-labs/audit_helper/latest/). To use Value Diff, ensure that `audit-helper` is installed in your project.
138-
139-
```yaml
140-
packages:
141-
- package: dbt-labs/audit_helper
142-
version: <version>
143-
```
144-
145-
146129
View mismatched values at the row level by clicking the `show mismatched values` option on a column name:
147130

148131
![](../assets/images/features/value-diff-detail.gif){: .shadow}
@@ -155,26 +138,26 @@ Profile Diff compares the basic statistic (e.g. count, distinct count, min, max,
155138
2. Click the `Expore Change` button.
156139
3. Click `Profile Diff`.
157140

158-
159141
<figure markdown>
160142
![Recce Profile Diff](../assets/images/features/profile-diff.png)
161143
<figcaption>Profile Diff</figcaption>
162144
</figure>
163145

164-
Please refer to the [dbt-profiler](https://hub.getdbt.com/data-mie/dbt_profiler/latest/#dbt-profiler) documentation for the definitions of profiling stats.
165-
166-
!!! Note
167-
Profile diff uses the `get_profile` from [dbt-profiler](https://hub.getdbt.com/data-mie/dbt_profiler/latest/). To use Profile Diff, ensure that dbt-profiler is installed in your project.
146+
The Statistics:
168147

169-
```yaml
170-
packages:
171-
- package: data-mie/dbt_profiler
172-
version: <version>
173-
```
148+
- Row count
149+
- Not null proportion
150+
- Distinct proportion
151+
- Distinct count
152+
- Is unique
153+
- Minimum
154+
- Maximum
155+
- Average
156+
- Median
174157

175158
### Histogram Diff
176159

177-
Histogram Diff compares the distribution of a numeric column in an overlay histogram chart.
160+
Histogram Diff compares the distribution of a numeric column in an overlay histogram chart.
178161

179162
<figure markdown>
180163
![Recce Histogram Diff](../assets/images/features/histogram-diff.png)
@@ -185,7 +168,6 @@ A Histogram Diff can be generated in two ways.
185168

186169
**Via the Explore Change button menu:**
187170

188-
189171
1. Select the model from the Lineage DAG.
190172
2. Click the `Explore Change` button.
191173
3. Click `Histogram Diff`.
@@ -199,13 +181,11 @@ A Histogram Diff can be generated in two ways.
199181
3. Click the vertical 3 dots `...`
200182
4. Click `Histogram Diff`.
201183

202-
203184
<figure markdown>
204185
![Generate a Recce Histogram Diff ](../assets/images/features/histogram-diff.gif){: .shadow}
205186
<figcaption>Generate a Recce Histogram Diff from the column options</figcaption>
206187
</figure>
207188

208-
209189
### Top-K Diff
210190

211191
Top-K Diff compares the distribution of a categorical column. The top 10 elements are shown by default, which can be expanded to the top 50 elements.
@@ -215,12 +195,10 @@ Top-K Diff compares the distribution of a categorical column. The top 10 element
215195
<figcaption>Recce Top-K Diff</figcaption>
216196
</figure>
217197

218-
219198
A Top-K Diff can be generated in two ways.
220199

221200
**Via the Explore Change button menu:**
222201

223-
224202
1. Select the model from the Lineage DAG.
225203
2. Click the `Explore Change` button.
226204
3. Click `Top-K Diff`.
@@ -234,14 +212,11 @@ A Top-K Diff can be generated in two ways.
234212
3. Click the vertical 3 dots `...`
235213
4. Click `Top-K Diff`.
236214

237-
238215
<figure markdown>
239216
![Generate a Recce Top-K Diff ](../assets/images/features/top-k-diff.gif){: .shadow}
240217
<figcaption>Generate a Recce Top-K Diff </figcaption>
241218
</figure>
242219

243-
244-
245220
## Multi-Node Selection
246221

247222
Multiple nodes can be selected in the Lineage DAG. This enables actions to be performed on multiple nodes at the same time such as Row Count Diff, or Value Diff.
@@ -295,8 +270,6 @@ An example of selecting multiple nodes to perform a multi-node Value Diff:
295270
<figcaption>Perform a Value Diff on multiple nodes</figcaption>
296271
</figure>
297272

298-
299-
300273
## Screenshot
301274

302275
In the diff result, we can find a **Copy to Clipboard** button. it's a handy feature to copy the result image to clipboard and paste in your PR comment.
@@ -339,7 +312,6 @@ For the majority of diffs, which are performed via the Explore Change dropdown m
339312
<figcaption>Add a Check by clicking the Add to Checklist button in the diff results panel</figcaption>
340313
</figure>
341314

342-
343315
An example performing a Top-K diff and adding the results to the Checklist:
344316

345317
<figure markdown>

docs/installation.md

Lines changed: 0 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -11,18 +11,3 @@ Install `Recce` in your dbt project with pip:
1111
pip install recce
1212
```
1313

14-
To take full advantage of all the features of `Recce`, ensure that [dbt_profiler](https://hub.getdbt.com/data-mie/dbt_profiler/latest/) and [audit-helper](https://hub.getdbt.com/dbt-labs/audit_helper/latest/) are installed via the `packages.yml` file in your dbt project .
15-
16-
1. Add these two packages in the packages.yml
17-
2. Do `dbt deps` to install these 2 packages.
18-
19-
```yaml
20-
packages:
21-
- package: dbt-labs/audit_helper
22-
version: <version>
23-
- package: data-mie/dbt_profiler
24-
version: <version>
25-
26-
```
27-
28-
For full instructions on using `Recce`, check the [Getting Started](get-started.md) guide.

docs/recce-cloud/getting-started-recce-cloud.md

Lines changed: 1 addition & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -77,18 +77,6 @@ Set up the Jaffle Shop project and install Recce.
7777
+ schema: prod
7878
+ threads: 24
7979
```
80-
1. Add the following packages required by Recce for some features (highly recommended). Create a `./packages.yml` file in the root of your project with the following packages:
81-
```
82-
packages:
83-
- package: dbt-labs/audit_helper
84-
version: 0.12.0
85-
- package: data-mie/dbt_profiler
86-
version: 0.8.2
87-
```
88-
Install the packages:
89-
```
90-
dbt deps
91-
```
9280

9381
## Prepare the base environment
9482

@@ -256,4 +244,4 @@ Back on the GitHub PR page, you'll notice that the Recce Cloud check status has
256244
![Recce Cloud - All Checks are Approved](../assets/images/recce-cloud/pr-checks-all-approved.png){: .shadow}
257245

258246

259-
In a real-world situation you'd now be able to merge the PR with the confidence that the PR author had checked their work, and the reviewer both understands and has signed-off on any changes.
247+
In a real-world situation you'd now be able to merge the PR with the confidence that the PR author had checked their work, and the reviewer both understands and has signed-off on any changes.

0 commit comments

Comments
 (0)