v0.7.0

Release/0.7.0 core (#311)

* Fix typos in configs and notebook docstrings

- Fix GOV-20 check name: "Unity Sharing" → "Unity Catalog"
- Fix "metasores" → "metastores" in GOV-20 logic description
- Fix "cofiguration" → "configuration" in 5 manual check descriptions
  (DP-4, IA-1, IA-2, IA-3, INFO-7)
- Fix "configruation" → "configuration" in initialize.py and
  sat_checks_config.py docstrings

* Fix "recepient" → "recipient" typo in delta sharing code

- Rename functions: get_sharing_recepients_list, get_sharing_recepient,
  get_sharing_recepient_permissions → recipient spelling
- Rename intermediate table: delta_sharing_recepients_list →
  delta_sharing_recipients_list (safe: table is in intermediate schema
  which is dropped after each SAT run)
- Update all callers in workspace_bootstrap, workspace_analysis, and tests

* Fix typos in security checks CSV and workspace analysis notebook

- configs/security_best_practices.csv: fix "custer"→"cluster" (GOV-9) and "diplayName"→"displayName" (INFO-6)
- notebooks/Includes/workspace_analysis.py: fix "uc_metasore*" → "uc_metastore*" in local variable and inner function names

* Revert non-typo changes swept into previous commit

Restores CHANGELOG.md and configs/sat_dasf_mapping.csv to their
release/0.7.0_core state, and removes the unintended backup file.
This branch should only contain the two typo fixes:
  - security_best_practices.csv: custer→cluster, diplayName→displayName
  - workspace_analysis.py: uc_metasore*→uc_metastore*

* Fix 2 CSV typos cleanly (no extra changes)

Restores security_best_practices.csv to its pre-ac66c3d state and
re-applies only the 2 intended typo fixes:
  - GOV-9: 'custer' → 'cluster' in recommendation field
  - INFO-6: 'diplayName' → 'displayName' in logic field

* Bump SDK to 0.1.41 and rebuild wheel

- Increment __version__ in setup.py: 0.1.40 → 0.1.41
- Replace lib/dbl_sat_sdk-0.1.40-*.whl with newly built 0.1.41 wheel
- Update SDK_VERSION in install_sat_sdk.py to match

* feat(SFE-4539): add UC schema/table/column comments for Genie

Adds apply_schema_comments() to common.py with data-verified descriptions
for all 12 tables and ~110 columns in the security_analysis schema.
Called from initialize.py after load_sat_dasf_mapping() so comments are
applied on every SAT run.

- Covers 9 SAT core tables + 3 BrickHound tables (BH wrapped in try/except)
- Single quotes escaped as '' in SQL literals
- Idempotent: safe to re-run on existing deployments

* docs: add CLAUDE.md to repo and schema comment/validation rules

- Add CLAUDE.md to version control (force-add, was previously gitignored)
- Add Schema Comment Sync rule: keep apply_schema_comments() in sync
  whenever tables/columns are added, removed, or renamed
- Add security_best_practices.csv uniqueness validation rule: check that
  id and check_id are both unique before every CSV commit
- Add Pre-Commit Typo Check rule using codespell
- Update security_best_practices table/column comments to explicitly
  document that id and check_id are unique identifiers

* chore: allow CLAUDE.md to be tracked by git

Remove CLAUDE.md from .gitignore so it is versioned alongside the codebase.

* refactor(SFE-4539): co-locate table/column comments with table creation

Move all UC Genie column/table comments from the monolithic
apply_schema_comments() into the function that creates each table,
so documentation lives next to the DDL that defines it.

- Each create_*() in common.py now calls _set_table_comment +
  _set_column_comments immediately after its CREATE TABLE DDL
- readBestPracticesConfigsFile() and load_sat_dasf_mapping() set
  comments inside their saveAsTable blocks
- apply_schema_comments() deleted; callers in initialize.py removed
- Schema-level COMMENT ON SCHEMA moved into create_schema()
- BrickHound: get_vertex_schema() and get_edge_schema() in schema.py
  get inline COMMENT clauses on every column
- New GraphSchema.get_metadata_schema() DDL with inline column comments
  called in permission_analysis_data_collection.py before the first
  brickhound_collection_metadata saveAsTable, fixing the timing bug
  where BrickHound table comments were silently skipped at init time

* fix(SFE-4539): guard GraphSchema call when brickhound not installed

When brickhound is not installed the try/except ImportError block leaves
GraphSchema undefined, causing a NameError at the get_metadata_schema() call
added in the SFE-4539 refactor.  Fix: set GraphSchema = None in the except
branch and guard the spark.sql() call with `if GraphSchema is not None`.

When brickhound IS installed the UC column comments are applied as before.
When it is not installed the metadata table is still created via saveAsTable
without column comments, restoring the original works-with-or-without contract.

* feat(SFE-4539): add UC table/column comments for brickhound tables

Replace unreliable GraphSchema DDL approach with explicit ALTER TABLE
ALTER COLUMN COMMENT calls after each saveAsTable, mirroring the SAT
pattern used in common.py. Comments now apply regardless of whether
brickhound is installed and work on pre-existing tables.

Tables covered: brickhound_vertices (14 cols), brickhound_edges (8 cols),
brickhound_collection_metadata (10 cols).

* feat(SFE-4548): add INFO-42 Git repository allowlist check

- Add check id=113, check_id=INFO-42 to security_best_practices.csv
- Implement enableProjectsAllowList rule in workspace_settings.py
- Add DASF-52 mapping to sat_dasf_mapping.csv
- Append Phase 5 (bugs) and Phase 6 (new check backlog) to sat_checks_audit.md

* feat(SFE-4548): include Git URL allowlist entries in INFO-42 check details

When enableProjectsAllowList passes, also return the projectsAllowList
comma-separated URL prefixes in additional details so reviewers can verify
the allowlist is properly scoped. Add projectsAllowList to ws_keymap and
expand the SQL/rule function to fetch both keys. Rebuild wheel 0.1.41.

* feat(SFE-4548): add /add-sat-check Claude skill and expand checks audit doc

- Add .claude/commands/add-sat-check.md: project-level skill that guides
  end-to-end implementation of a new SAT security check (CSV → SDK →
  notebook check block → DASF mapping → validations)
- Expand docs/sat_checks_audit.md with INFO-42 additional-details section
  and Phase 7 planned checks (NS-12, IA-10, GOV-44, NS-13)

* fix: correct typo 'respones' -> 'responses' in notebook header comments

* chore: remove sat_checks_audit.md from branch

* feat(SFE-4549): remove 24 unrelated checks and self-assessment functionality

- Remove 24 checks from security_best_practices.csv: DP-4, GOV-1/6/7/8/9/13/23/24/26, IA-1/2/3/7, INFO-1/2/4/7/12/13/14/17, NS-1/2
- Remove corresponding DASF mapping entries and self_assessment_checks.yaml
- Delete self-assessment notebook (Setup/9) and its two functions in sat_checks_config.py
- Remove schema fields: object_storage_encrypted, vpc_peering_done, table_access_control_enabled, sso_enabled, scim_enabled from account_workspaces DDL, CSV schema, drivers, setup notebooks, and config utilities
- Remove all check implementations from workspace_analysis.py and workspace_settings.py

* feat(SFE-4549): remove legacy Databricks SQL API EOL endpoints and bump SDK to 0.1.42

Remove deprecated /api/2.0/sql/alerts, /sql/queries, /sql/config/warehouses,
/preview/sql/permissions, and /preview/sql/data_sources usages. Dead-code
bootstrap calls for dbsql_workspaceconfig and dbsql_alerts are dropped from
workspace_bootstrap.py. configure_alerts_template.py now uses the warehouse ID
directly as data_source_id. SDK version bumped 0.1.41 -> 0.1.42 with rebuilt wheel.

* feat(SFE-4549): remove alerts feature and drop alert column from schema

- Delete notebooks/Setup/6. configure_alerts_template.py (deprecated SQL API endpoints at EOL)
- Remove dangling references to notebooks 6 and 9 from security_analysis_initializer.py
- Drop alert column from configs/security_best_practices.csv (all 55 rows had alert=0)
- Remove alert from security_best_practices Delta table schema in common.py (schema_list, DDL, select, column comments)
- Remove alert widget and UPDATE SQL field from sat_checks_config.py

* fix(SFE-4549): NS-9 — treat all DRY_RUN modes and unknown enforcement as violations

Only ENFORCED passes. Selective dry-run (non-empty product filter) now returns
DRY_RUN_SELECTIVE violation instead of passing. Unknown/missing enforcement_mode
now returns UNKNOWN_ENFORCEMENT_MODE violation instead of passing.

* fix: escape single quotes in schema comment helpers to prevent SQL parse errors

_set_table_comment and _set_column_comments now escape single quotes in comment
strings before interpolating into SQL. Also removed embedded single-quoted
examples from the additional_details column comment that triggered the error.

* fix: correct SQL single-quote escaping in comment helpers and insertIntoInfoTable

- _set_table_comment/_set_column_comments: use standard SQL '' escaping instead
  of backslash escaping (which is unreliable in Spark SQL)
- insertIntoInfoTable: escape name and category before SQL interpolation
  (was already escaping jsonstr but not the other string fields)

* fix: remove pre-escaped single quotes from schema comment strings

Comment strings passed to _set_table_comment/_set_column_comments were using
SQL-style '' escaping manually, which the helper then doubled again to '''',
causing PARSE_SYNTAX_ERROR. Replaced all ''word'' patterns with plain text.
Affected: account_info.category, account_workspaces table comment,
sat_dasf_mapping table and dasf_control_id column comments.

* removing manual config text from dashboard

* widget name

* chore: bump sat_version to 0.7.0 in initialize.py

---------

Co-authored-by: shdzhang <39942190+shdzhang@users.noreply.github.com>
Co-authored-by: Kiran Anand <16294307+kiran-anand@users.noreply.github.com>
Co-authored-by: Shreel Shah <shreelshah12@gmail.com>