[CDF-27563] 😁 Standardize validation by doctrino · Pull Request #2792 · cognitedata/toolkit

doctrino · 2026-03-26T13:38:00Z

Description

Reorganization of the validation to match the data structures we now use.

In progress

Completed

Bump

Patch
Skip

gemini-code-assist · 2026-03-26T13:38:06Z

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here.

doctrino · 2026-03-26T13:38:54Z

cognite_toolkit/_cdf_tk/commands/build_v2/build_v2.py

            cdf_project = config.environment.project
            validation_type = config.environment.validation_type

-        # Todo optimize by only searching for yaml files in the selected modules paths if selection is provided.


We do not want to do this. Instead, we should find all yaml files so we can give proper error output. It is a trade-off between performance and useability, and then we should side with useability.

doctrino · 2026-03-26T13:39:12Z

cognite_toolkit/_cdf_tk/commands/build_v2/data_classes/_insights.py

    ...


-class ConsistencyWarning(Insight):


github-actions · 2026-03-26T13:45:17Z

☂️ Python Coverage

current status: ✅

Overall Coverage

Lines	Covered	Coverage	Threshold	Status
39525	33439	85%	80%	🟢

New Files

File	Coverage	Status
cognite_toolkit/_cdf_tk/rules/_auth.py	100%	🟢
cognite_toolkit/_cdf_tk/rules/_dependencies.py	88%	🟢
cognite_toolkit/_cdf_tk/rules/_neat.py	84%	🟢
TOTAL	90%	🟢

Modified Files

File	Coverage	Status
cognite_toolkit/_cdf_tk/client/api/views.py	94%	🟢
cognite_toolkit/_cdf_tk/commands/build_v2/build_v2.py	76%	🟢
cognite_toolkit/_cdf_tk/commands/build_v2/data_classes/init.py	100%	🟢
cognite_toolkit/_cdf_tk/commands/build_v2/data_classes/_build.py	96%	🟢
cognite_toolkit/_cdf_tk/commands/build_v2/data_classes/_insights.py	85%	🟢
cognite_toolkit/_cdf_tk/commands/build_v2/data_classes/_lineage.py	71%	🟢
cognite_toolkit/_cdf_tk/commands/build_v2/data_classes/_module.py	95%	🟢
cognite_toolkit/_cdf_tk/cruds/_resource_cruds/datamodel.py	88%	🟢
cognite_toolkit/_cdf_tk/rules/init.py	100%	🟢
cognite_toolkit/_cdf_tk/rules/_base.py	97%	🟢
cognite_toolkit/_cdf_tk/rules/_orchestrator.py	91%	🟢
TOTAL	90%	🟢

updated for commit: fcc11d6 by action🐍

codecov · 2026-03-26T13:45:32Z

Codecov Report

❌ Patch coverage is 84.11215% with 51 lines in your changes missing coverage. Please review.
✅ Project coverage is 84.60%. Comparing base (5b3dcd7) to head (fcc11d6).

Files with missing lines	Patch %	Lines
...nite_toolkit/_cdf_tk/commands/build_v2/build_v2.py	67.77%	29 Missing ⚠️
cognite_toolkit/_cdf_tk/rules/_neat.py	83.58%	11 Missing ⚠️
cognite_toolkit/_cdf_tk/rules/_dependencies.py	87.50%	5 Missing ⚠️
cognite_toolkit/_cdf_tk/client/api/views.py	83.33%	2 Missing ⚠️
cognite_toolkit/_cdf_tk/rules/_orchestrator.py	93.33%	2 Missing ⚠️
...t/_cdf_tk/commands/build_v2/data_classes/_build.py	95.23%	1 Missing ⚠️
cognite_toolkit/_cdf_tk/rules/_base.py	96.77%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #2792      +/-   ##
==========================================
- Coverage   84.66%   84.60%   -0.07%     
==========================================
  Files         447      447              
  Lines       39428    39525      +97     
==========================================
+ Hits        33383    33439      +56     
- Misses       6045     6086      +41

Files with missing lines	Coverage Δ
..._cdf_tk/commands/build_v2/data_classes/__init__.py	`100.00% <100.00%> (ø)`
...cdf_tk/commands/build_v2/data_classes/_insights.py	`84.90% <100.00%> (ø)`
..._cdf_tk/commands/build_v2/data_classes/_lineage.py	`71.09% <ø> (+1.56%)`	⬆️
.../_cdf_tk/commands/build_v2/data_classes/_module.py	`95.03% <100.00%> (+0.07%)`	⬆️
...toolkit/_cdf_tk/cruds/_resource_cruds/datamodel.py	`88.18% <100.00%> (ø)`
cognite_toolkit/_cdf_tk/rules/__init__.py	`100.00% <100.00%> (ø)`
cognite_toolkit/_cdf_tk/rules/_auth.py	`100.00% <100.00%> (ø)`
...t/_cdf_tk/commands/build_v2/data_classes/_build.py	`96.05% <95.23%> (+3.64%)`	⬆️
cognite_toolkit/_cdf_tk/rules/_base.py	`96.87% <96.77%> (-3.13%)`	⬇️
cognite_toolkit/_cdf_tk/client/api/views.py	`94.44% <83.33%> (-5.56%)`	⬇️
... and 4 more

... and 4 files with indirect coverage changes

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

doctrino · 2026-03-27T06:36:58Z

cognite_toolkit/_cdf_tk/commands/build_v2/data_classes/_build.py

-    def resource_by_type_by_kind(self) -> dict[ResourceType, list[Path]]:
-        """Organizes built files by their resource type and kind."""
-        resource_by_type: dict[ResourceType, list[Path]] = defaultdict(list)
-        for resource in self.resources:


I avoid having logic on the DTO objects that are not used or only used in one location.

doctrino · 2026-03-27T06:37:21Z

cognite_toolkit/_cdf_tk/commands/build_v2/data_classes/_plugins.py

Moved to the NeatRule

doctrino · 2026-03-27T06:39:52Z

cognite_toolkit/_cdf_tk/commands/build_v2/build_v2.py

-        dependency_insights = self._dependency_validation(built_modules, client)
-
-        global_insights = self._global_validation(built_modules, client)
+        plan = self._create_validation_plan(built_modules, client)
+        if parameters.verbose:
+            self._display_validation_plan(plan)
+        validation_results = self._run_validation(plan, console)


This is the main change, instead of having one function for each validation (dependency, neat (only called in global validation). Standardize them into a ToolkitGlobalRulSet. Then, use the create plan, display, and run validation.

doctrino · 2026-03-27T06:40:35Z

cognite_toolkit/_cdf_tk/rules/_auth.py

+BASE_CODE = "AUTH"
+
+
+class CheckDataSetMissing(ToolkitLocalRule):


Moved from the specialist rule only applied to Workflow. This applies to all resources that have a data set.

doctrino · 2026-03-27T06:41:11Z

cognite_toolkit/_cdf_tk/rules/_dummy.py

No need to keep this :)

doctrino · 2026-03-27T06:41:35Z

cognite_toolkit/_cdf_tk/rules/_neat.py

Combining the previous neat plugin with the logic to execute it that was in the build v2 function.

doctrino · 2026-03-27T06:44:43Z

/gemini review

gemini-code-assist

Code Review

This pull request refactors the validation logic in the build process by introducing a new validation plan system and replacing the previous RulesOrchestrator with LocalRulesOrchestrator and NeatRules. It also updates data classes to support these changes, including the addition of ValidationResult and FailedValidation models. A critical bug was identified in the validation loop where the status check incorrectly compared a RuleSetStatus object to a string instead of checking its code attribute.

cognite_toolkit/_cdf_tk/commands/build_v2/build_v2.py

doctrino · 2026-03-27T11:04:55Z

cognite_toolkit/_cdf_tk/client/api/views.py

+
+        results: list[ViewResponse] = []
+        # The API does not support mixing ViewId and ViewNoVersionId in the same request.
+        if view_ids:


This is an random issue I found

doctrino added 20 commits March 26, 2026 13:37

refactor: logic for all issues

7e4868f

refactor: finished data structures

6f77998

refator: reord

2d25c59

tests: added test suite

e9d5c27

fix; issue in get misplaced modules

c87db77

refactor: update display

fb08a4f

fix: raise if errors

3aa6692

refactor: more display code

a28bb19

refactor: fix

859fbbe

fix: bug

cd61268

tests: updated tests

7036e99

fix

fc88d49

fix: random error

b616dee

fix; parent variables

af2a1ac

refactor: rename

6a02e29

refactro: update demo

3fe9555

more fixes

63a3c66

refactro: cleanup

d72084f

fix: handle misbehaving neat

9be8226

refator: clean/plan

6d34826

doctrino commented Mar 26, 2026

View reviewed changes

doctrino added 5 commits March 26, 2026 14:49

refactor: reorg

23899da

refactor: reorg rules

ea05b1f

wip

a3868ca

refactor: implement ruleset

4f68f5a

refactor: simplify neat ruleset

b88e3e5

refactor: update display

36fb8d7

doctrino commented Mar 27, 2026

View reviewed changes

cognite_toolkit/_cdf_tk/commands/build_v2/data_classes/_plugins.py Outdated

Copy link
Copy Markdown

Contributor Author

doctrino Mar 27, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Moved to the NeatRule

feat: display plan

51ce595

doctrino commented Mar 27, 2026

View reviewed changes

cognite_toolkit/_cdf_tk/rules/_dummy.py Outdated

Copy link
Copy Markdown

Contributor Author

doctrino Mar 27, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No need to keep this :)

doctrino commented Mar 27, 2026

View reviewed changes

refactor; bad renaming

e8d16bd

doctrino marked this pull request as ready for review March 27, 2026 06:44

doctrino requested review from a team as code owners March 27, 2026 06:44

gemini-code-assist bot reviewed Mar 27, 2026

View reviewed changes

cognite_toolkit/_cdf_tk/commands/build_v2/build_v2.py Outdated Show resolved Hide resolved

doctrino added 7 commits March 27, 2026 07:53

refactor: review feedback + tests

2e8edc6

fix: import rulesets

c032d65

fix: manual testing

f9364a3

tests: adjust test data

aae1064

regen

cc31616

refactor: adjust

fa43895

refactor: adjust

1191f12

Base automatically changed from module-validation to main March 27, 2026 10:49

Merge remote-tracking branch 'origin/main' into standardizing-validation

29de533

doctrino enabled auto-merge (squash) March 27, 2026 11:00

doctrino commented Mar 27, 2026

View reviewed changes

doctrino mentioned this pull request Mar 27, 2026

[CDF-27564] 🙉Test build+deploy for v2 commands. #2800

Open

2 tasks

Merge branch 'main' into standardizing-validation

fcc11d6

		BASE_CODE = "AUTH"


		class CheckDataSetMissing(ToolkitLocalRule):

Conversation

doctrino commented Mar 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

In progress

Completed

Bump

Uh oh!

gemini-code-assist bot commented Mar 26, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Mar 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

☂️ Python Coverage

Overall Coverage

New Files

Modified Files

Uh oh!

codecov bot commented Mar 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

doctrino commented Mar 27, 2026

Uh oh!

gemini-code-assist bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

doctrino commented Mar 26, 2026 •

edited

Loading

github-actions bot commented Mar 26, 2026 •

edited

Loading

codecov bot commented Mar 26, 2026 •

edited

Loading