Skip to content

feat(catalog): add catalog schema (XOR-241)#1686

Draft
mesejo wants to merge 23 commits intomainfrom
feat/catalog-schema-cmd
Draft

feat(catalog): add catalog schema (XOR-241)#1686
mesejo wants to merge 23 commits intomainfrom
feat/catalog-schema-cmd

Conversation

@mesejo
Copy link
Collaborator

@mesejo mesejo commented Mar 6, 2026

depends on #1682

mesejo and others added 18 commits March 5, 2026 15:20
Canonical low-level primitive for detecting UnboundTable nodes in an
expression graph. Used by the ExprKind detection logic in subsequent
steps.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Introduces ExprKind (Expr / PartialExpr) and writes it to metadata.json
at build_expr time. Detection is automatic: expressions containing an
UnboundTable node get kind=partial_expr; fully bound expressions get
kind=expr. No user flag needed.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Adds _extract_kind() which reads the kind field written by build_expr
from the tgz archive. _load_catalog_row now uses it so the TUI displays
expr vs partial_expr correctly. Old builds without a kind key fall back
to "expr".

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Makes these symbols available at xorq.catalog.api so the MCP server,
CLI, and downstream consumers can import them without reaching into
internal modules.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…nbound exprs

Four tests covering both layers:
- test_build_expr_kind_bound / _partial: verify metadata.json at the
  compiler level
- test_extract_kind_bound / _partial: verify the full stack through a
  real catalog entry and the TUI helper

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
_extract_kind() in tui.py was a free function that re-opened the tgz to
read metadata.json. Moving it onto CatalogEntry as a .kind property puts
the logic where it belongs — next to .expr and .aliases — and gives every
consumer (TUI, CLI, MCP server) a single, consistent way to query the
kind without knowing about the tgz layout.

tui._load_catalog_row now reads entry.kind directly. The _extract_kind
helper and its json import are removed from tui.py.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…s property

_extract_backends() in tui.py read profiles.yaml from the tgz to return
backend names. Moving it onto CatalogEntry as a .backends property keeps
all tgz-reading logic on the entry object itself, alongside .kind and
.expr.

tui._load_catalog_row now reads entry.backends directly.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…ntry

Deduplicate tarfile+yaml reading logic shared by kind and backends
properties into a single instance method that takes a DumpFiles enum value.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Read kind from expr.yaml before loading the full expression; raise
ValueError with a clear message directing users to compose-add.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Display ExprKind alongside each entry name so users can distinguish
runnable (expr) entries from those needing composition (unbound_expr).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@codecov
Copy link

codecov bot commented Mar 6, 2026

Codecov Report

❌ Patch coverage is 81.61290% with 57 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
python/xorq/catalog/cli.py 24.48% 37 Missing ⚠️
python/xorq/catalog/catalog.py 79.16% 20 Missing ⚠️
Files with missing lines Coverage Δ
python/xorq/catalog/tests/test_catalog.py 100.00% <100.00%> (ø)
python/xorq/catalog/tui.py 71.14% <ø> (+0.82%) ⬆️
python/xorq/cli.py 71.77% <100.00%> (ø)
python/xorq/common/utils/graph_utils.py 99.09% <100.00%> (+0.05%) ⬆️
python/xorq/common/utils/logging_utils.py 89.71% <ø> (ø)
python/xorq/common/utils/tests/test_graph_utils.py 100.00% <100.00%> (ø)
python/xorq/ibis_yaml/compiler.py 95.48% <100.00%> (+0.07%) ⬆️
python/xorq/ibis_yaml/tests/conftest.py 99.35% <100.00%> (+0.02%) ⬆️
python/xorq/ibis_yaml/tests/test_basic.py 100.00% <100.00%> (ø)
python/xorq/ibis_yaml/tests/test_compiler.py 98.64% <100.00%> (+0.04%) ⬆️
... and 7 more

... and 6 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Adds `strict=True` parameter that raises ValueError when more than one
UnboundTable is found, guarding against unsupported multi-source unbound
expressions. Includes four tests covering all cases.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@codspeed-hq
Copy link

codspeed-hq bot commented Mar 6, 2026

Merging this PR will not alter performance

✅ 14 untouched benchmarks


Comparing feat/catalog-schema-cmd (e29cf2a) with main (b9f5e87)

Open in CodSpeed

@mesejo mesejo force-pushed the feat/catalog-schema-cmd branch from ec449bd to 986d60e Compare March 6, 2026 11:02
mesejo and others added 3 commits March 6, 2026 12:58
…ecution

Adds `xorq catalog schema <name-or-alias>` command that prints Schema In
(for unbound/partial expressions) and Schema Out (for all entries) by reading
expr.yaml directly from the tgz — no backend connection required.

- CatalogEntry.schema_out / .schema_in cached properties parse expr.yaml
- CatalogEntry._parse_schema_* staticmethods handle dtype dict → str conversion
- Catalog.get_entry() resolves both entry names and aliases
- --json flag for machine-readable output
- Shell completion covers both names and aliases

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…guarantee

Now that has_unbound_table enforces at most one UnboundTable at build
time, replace the early-return loop with an explicit next()/None pattern
that makes the single-match assumption clear.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@mesejo mesejo force-pushed the feat/catalog-schema-cmd branch from 986d60e to c882a50 Compare March 6, 2026 13:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant