feat(sync): add --select flag to sync specific schemas/tables without deleting the rest - #924
Merged
Merged
Conversation
… deleting the rest `nao sync` previously deleted any previously-synced table/schema that wasn't part of the current run (cleanup_stale_paths). The only way to refresh a single table was to either narrow `include` (which wipes everything else) or sync the whole warehouse. Add a `--select`/`-s` flag that restricts the sync to given schemas/tables and skips stale-path cleanup so nothing outside the selection is removed: nao sync --select analytics.orders nao sync --select analytics nao sync -s analytics.orders -s staging.dim_* A pattern without a dot selects a whole schema; glob wildcards are supported. Selection is applied on top of the existing include/exclude config. Tables and Snowflake semantic views both respect it. Closes #923 Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Contributor
🧹 Preview RemovedThe preview deployment for this PR has been cleaned up. |
Contributor
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using high effort and found 2 potential issues.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 9dbd501. Configure here.
Contributor
There was a problem hiding this comment.
2 issues found across 6 files
Reply with feedback, questions, or to request a fix.
Re-trigger cubic
…tern Surface that `--select` composes with `--provider databases:<conn>` so users discover they can scope a refresh to one connection and avoid connecting to their other databases. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Address review feedback on the --select filter: - Case-sensitivity (P1): fnmatch is case-sensitive on Linux/macOS, so `--select analytics.orders` failed to match Snowflake's uppercased `ANALYTICS.ORDERS`. Lowercase both sides before matching. - Catalog-qualified schemas (P2): the previous "dot means schema.table" heuristic broke when a schema itself contains a dot (e.g. StarRocks `catalog.schema`). Match the pattern against the full name directly and treat a pattern as a whole-schema selector via `<pattern>.*`, so both `catalog.analytics` and `catalog.analytics.orders` work. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
socallmebertille
requested changes
Jun 26, 2026
socallmebertille
left a comment
Contributor
There was a problem hiding this comment.
Tiny changes can be made and it will look awesome
Address review feedback: --select is database-only, so it no longer pollutes the abstract SyncProvider.sync() nor the notion/repositories implementations. - Remove the `select` parameter from the base abstract `sync` and from the Notion and Repository providers. - The CLI passes `select` only to DatabaseSyncProvider (via isinstance) and warns when `--select` is given but no databases provider is active. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.

Problem
There is no way to refresh a single table or schema with
nao syncwithout affecting everything else:includeconfig,nao syncdeletes all other previously-synced tables — the post-sync cleanup (cleanup_stale_paths) removes any schema/table directory that wasn't part of the current run.Closes #923
Solution
Add a
--select/-sflag tonao syncthat restricts the sync to a selection of schemas/tables and skips stale-path cleanup, so nothing outside the selection is removed.Behaviour:
analytics) selects every table in that schema (analytics.*).analytics.orders) selects a specific table; glob wildcards are supported (staging.dim_*).include/excludeconfig — it narrows further, never widens.--selectis passed,cleanup_stale_pathsis skipped, preserving previously-synced tables/schemas outside the selection.Implementation
cli/nao_core/commands/sync/__init__.py— new--select/-sparameter, passed through to the provider.cli/.../providers/databases/provider.py—_matches_selection()helper;sync_database()filters tables/semantic views by the selection; provider skips cleanup and prints aSelect: … (stale cleanup skipped)note when active.cli/.../providers/base.py,notion,repositories— accept theselectkwarg for a consistent interface (only databases honors it).Tests
TestMatchesSelection— schema-only, schema.table, and glob pattern matching.TestSelectSkipsCleanup— cleanup runs without--select, is skipped with it, andselectis threaded intosync_database.make lintpasses; the sync test suite is green (218 passed; theintegration/dir is excluded due to a local unixodbc/pyodbc issue unrelated to this change).🤖 Generated with Claude Code
Note
Low Risk
Scoped CLI and database sync filtering with explicit cleanup skip; other providers unchanged in parent
pre_syncdatabase cleanup still runs.Overview
Adds
nao sync -s/--selectso you can refresh only chosen database schemas or tables (e.g.analytics,analytics.orders,staging.dim_*) without a full warehouse sync.Selection is narrowed on top of existing include/exclude config via
_matches_selection(schema-only patterns map toschema.*; globs usefnmatch). The databases provider applies it to tables and Snowflake semantic views. When--selectis active,cleanup_stale_pathsis skipped so previously synced objects outside the selection stay on disk; the CLI prints a note that stale cleanup was skipped.The
selectargument is threaded through theSyncProvider.syncinterface (repos/Notion accept it but ignore it). Unit tests cover pattern matching and cleanup-on vs cleanup-off behavior.Reviewed by Cursor Bugbot for commit 9dbd501. Bugbot is set up for automated code reviews on this repo. Configure here.