Fix package-lock.yml format, remove dbt-vertica from all extra, add release validation (#2159)

* Fix package-lock.yml format for dbt 1.8+ and remove restrictive adapters from all extra

- Remove deprecated 'name' fields from package-lock.yml that dbt 1.8+ rejects
- Remove dbt-clickhouse, dbt-sqlserver, dbt-vertica from 'all' extra as they
  pin dbt-core to 1.8.x (still available as individual extras)

Co-Authored-By: Itamar Hartstein <haritamar@gmail.com>

* Add release validation: reject package-lock.yml with deprecated 'name' fields

Co-Authored-By: Itamar Hartstein <haritamar@gmail.com>

* Revert lock file and release validation changes

The 'name' field in package-lock.yml is not deprecated - it is the
current format generated by modern dbt versions. The issue was specific
to dbt 1.8.x which has a bug rejecting this field. The real fix is
removing restrictive adapters from the 'all' extra so dbt resolves to
1.11+ where this bug doesn't exist.

Co-Authored-By: Itamar Hartstein <haritamar@gmail.com>

* Fix package-lock.yml and release validation wording

- Remove 'name' fields from package-lock.yml (unsupported in dbt 1.8)
- Re-add release validation with corrected wording: 'unsupported in dbt 1.8'
  instead of incorrectly saying 'deprecated'

Co-Authored-By: Itamar Hartstein <haritamar@gmail.com>

* Add dbt-clickhouse and dbt-sqlserver back to all extra

Only dbt-vertica (dbt-core==1.8.5 exact pin) and dbt-fabricspark
(broken upstream) need to stay out. dbt-clickhouse 1.10.0 requires
dbt-core>=1.9 and dbt-sqlserver 1.9.0 requires dbt-core>=1.9.0,<2.0,
so both are compatible with newer dbt-core versions.

Co-Authored-By: Itamar Hartstein <haritamar@gmail.com>

* Add comments explaining why fabricspark and vertica are excluded from all extra

Co-Authored-By: Itamar Hartstein <haritamar@gmail.com>

* Bump version to 0.23.1

Co-Authored-By: Itamar Hartstein <haritamar@gmail.com>

* Add pre-commit hook to auto-strip 'name' fields from package-lock.yml

The hook runs on package-lock.yml changes, strips any 'name' fields
(unsupported in dbt 1.8), and exits 1 so the user re-stages the fix.
Placed before the existing verify-dbt-project-packages-lock hook.

Co-Authored-By: Itamar Hartstein <haritamar@gmail.com>

* Remove release validation for package-lock.yml name fields

The pre-commit hook handles this automatically now, so the release
validation step is redundant.

Co-Authored-By: Itamar Hartstein <haritamar@gmail.com>

---------

Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: Itamar Hartstein <haritamar@gmail.com>

Author: devin-ai-integration[bot]

.pre-commit-config.yaml

@@ -46,6 +46,29 @@ repos:
 
   - repo: local
     hooks:
+      - id: strip-name-from-package-lock
+        name: Strip 'name' fields from package-lock.yml (unsupported in dbt 1.8)
+        entry: |
+          python -c "
+          import yaml, sys
+          lock_file = 'elementary/monitor/dbt_project/package-lock.yml'
+          with open(lock_file) as f:
+              data = yaml.safe_load(f)
+          changed = False
+          for pkg in data.get('packages', []):
+              if 'name' in pkg:
+                  del pkg['name']
+                  changed = True
+          if changed:
+              with open(lock_file, 'w') as f:
+                  yaml.dump(data, f, default_flow_style=False, sort_keys=False)
+              print(f'Removed unsupported name fields from {lock_file}')
+              sys.exit(1)
+          "
+        language: system
+        files: package-lock\.yml$
+        pass_filenames: false
+
       - id: verify-dbt-project-packages-lock
         name: Verify internal dbt project package lock
         entry: |

elementary/monitor/dbt_project/package-lock.yml

@@ -1,8 +1,6 @@
 packages:
-  - name: dbt_utils
-    package: dbt-labs/dbt_utils
+  - package: dbt-labs/dbt_utils
     version: 0.8.6
-  - name: elementary
-    package: elementary-data/elementary
+  - package: elementary-data/elementary
     version: 0.23.0
 sha1_hash: 0edc67beba193ced6784779894083b5af00f73ec

pyproject.toml

@@ -1,6 +1,6 @@
 [tool.poetry]
 name = "elementary-data"
-version = "0.23.0"
+version = "0.23.1"
 description = "Data monitoring and lineage"
 authors = ["Elementary"]
 keywords = ["data", "lineage", "data lineage", "data warehouse", "DWH", "observability", "data monitoring", "data observability", "Snowflake", "BigQuery", "Redshift", "data reliability", "analytics engineering"]
@@ -76,7 +76,10 @@ fabric = ["dbt-fabric"]
 fabricspark = ["dbt-fabricspark"]
 sqlserver = ["dbt-sqlserver"]
 vertica = ["dbt-vertica"]
-all = ["dbt-snowflake", "dbt-bigquery", "dbt-redshift", "dbt-postgres", "dbt-databricks", "dbt-spark", "dbt-clickhouse", "dbt-athena-community", "dbt-trino", "dbt-duckdb", "dbt-dremio", "dbt-fabric", "dbt-sqlserver", "dbt-vertica"]
+# dbt-fabricspark is excluded due to broken upstream dependencies (azure-cli pre-release pins).
+# dbt-vertica is excluded because it pins dbt-core==1.8.5, forcing the entire resolution to dbt 1.8.
+# Both are still available as individual extras (e.g. pip install elementary-data[vertica]).
+all = ["dbt-snowflake", "dbt-bigquery", "dbt-redshift", "dbt-postgres", "dbt-databricks", "dbt-spark", "dbt-athena-community", "dbt-trino", "dbt-clickhouse", "dbt-duckdb", "dbt-dremio", "dbt-fabric", "dbt-sqlserver"]
 
 [build-system]
 requires = ["poetry-core>=1.0.0"]