You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: README.md
+34Lines changed: 34 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -42,6 +42,40 @@ The [--config-path] is an optional argument. You can provide a yaml file with ov
42
42
43
43
Note: The dbt docs generate requires an active database connection and may take a long time for projects with large number of models.
44
44
45
+
### Pre-commit Hook Integration
46
+
47
+
DataPilot provides a pre-commit hook that automatically runs health checks on changed files before each commit. This ensures code quality and catches issues early in the development process.
Create a ``datapilot-config.yaml`` file in your project root:
75
+
76
+
.. code-block:: yaml
77
+
78
+
# DataPilot Configuration
79
+
disabled_insights:
80
+
- "hard_coded_references"
81
+
- "duplicate_sources"
82
+
83
+
# Custom settings for your project
84
+
project_settings:
85
+
max_fanout: 10
86
+
require_tests: true
30
87
31
88
3. Install the pre-commit hook:
32
89
33
-
```
34
-
pre-commit install
35
-
```
90
+
.. code-block:: shell
91
+
92
+
pre-commit install
36
93
37
94
Usage
38
95
-----
39
96
40
-
Once the hook is installed, it will run automatically before each commit. If any issues are detected, the commit will be aborted, and you will be prompted to fix the issues before retrying the commit.
97
+
Once the hook is installed, it will run automatically before each commit. The hook will:
98
+
99
+
1. **Validate Configuration**: Check that your config file exists and is valid
100
+
2. **Authenticate**: Use your provided token and instance name to authenticate
101
+
3. **Load DBT Artifacts**: Load manifest and catalog files for analysis
102
+
4. **Analyze Changes**: Only analyze files that have changed in the commit
103
+
5. **Report Issues**: Display any issues found and prevent the commit if problems are detected
104
+
105
+
**Required DBT Artifacts:**
106
+
107
+
The pre-commit hook requires DBT manifest and catalog files to function properly:
41
108
109
+
- **Manifest File**: Generated by running `dbt compile` or `dbt run`. Default location: `./target/manifest.json`
If you want to manually run all pre-commit hooks on a repository, run `pre-commit run --all-files`. To run individual hooks use `pre-commit run <hook_id>`.
112
+
**Note**: The catalog file is optional but recommended for comprehensive analysis. If not available, the hook will continue without catalog information.
44
113
114
+
**Manual Execution:**
115
+
116
+
To manually run all pre-commit hooks on a repository:
117
+
118
+
.. code-block:: shell
119
+
120
+
pre-commit run --all-files
121
+
122
+
To run individual hooks:
123
+
124
+
.. code-block:: shell
125
+
126
+
pre-commit run datapilot_run_dbt_checks
127
+
128
+
**Troubleshooting:**
129
+
130
+
- **Authentication Issues**: Ensure your token and instance name are correctly set
131
+
- **Empty Config Files**: The hook will fail if your config file is empty or invalid
132
+
- **Missing Manifest File**: Ensure you have run `dbt compile` or `dbt run` to generate the manifest.json file
133
+
- **Missing Catalog File**: Run `dbt docs generate` to create the catalog.json file (optional but recommended)
134
+
- **No Changes**: If no relevant files have changed, the hook will skip execution
135
+
- **Network Issues**: Ensure you have access to the DataPilot API
136
+
137
+
Best Practices
138
+
-------------
139
+
140
+
1. **Use Version Tags**: Always specify a version tag in the ``rev`` field, never use ``main``
141
+
2. **Environment Variables**: Use environment variables for sensitive information like tokens
142
+
3. **Configuration Files**: Create a dedicated config file for your project settings
143
+
4. **Regular Updates**: Update to new versions when they become available
144
+
5. **Team Coordination**: Ensure all team members use the same configuration
145
+
146
+
Example Complete Setup
147
+
---------------------
148
+
149
+
Here's a complete example of a ``.pre-commit-config.yaml`` file:
args: [--config-name, "[The Config Name in the SaaS UI]", --token, "[YOUR API KEY]", --instance-name, "[Your tenant name]"]
22
+
23
+
24
+
Performance
25
+
-----------
26
+
7
27
The primary objective is to ensure the pre-commit hook operates swiftly and efficiently, preventing any delay in the development workflow. To achieve this, various optimizations have been applied, focusing on minimizing the time and resources required during execution.
0 commit comments