docs: populated and validated the concept pages by Olamideod · Pull Request #1533 · xorq-labs/xorq

Olamideod · 2026-01-23T16:55:12Z

codecov · 2026-01-23T16:57:11Z

Codecov Report

❌ Patch coverage is 38.24000% with 386 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
python/xorq/expr/ml/tests/test_structer.py	21.16%	231 Missing ⚠️
python/xorq/expr/ml/tests/test_pipeline_lib.py	13.43%	58 Missing ⚠️
python/xorq/expr/ml/tests/test_fit_lib.py	16.94%	49 Missing ⚠️
python/xorq/expr/ml/structer.py	70.58%	30 Missing ⚠️
python/xorq/expr/ml/fit_lib.py	75.00%	17 Missing ⚠️
examples/bank_marketing.py	95.00%	1 Missing ⚠️

Files with missing lines	Coverage Δ
python/xorq/expr/ml/pipeline_lib.py	`68.57% <100.00%> (-19.79%)`	⬇️
examples/bank_marketing.py	`96.49% <95.00%> (+1.25%)`	⬆️
python/xorq/expr/ml/fit_lib.py	`75.97% <75.00%> (-12.23%)`	⬇️
python/xorq/expr/ml/structer.py	`67.92% <70.58%> (-19.77%)`	⬇️
python/xorq/expr/ml/tests/test_fit_lib.py	`23.47% <16.94%> (-76.53%)`	⬇️
python/xorq/expr/ml/tests/test_pipeline_lib.py	`28.94% <13.43%> (-71.06%)`	⬇️
python/xorq/expr/ml/tests/test_structer.py	`21.16% <21.16%> (ø)`

... and 211 files with indirect coverage changes

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

codspeed-hq · 2026-01-23T17:04:07Z

Merging this PR will improve performance by ×2.1

⚡ 1 improved benchmark
✅ 3 untouched benchmarks

Performance Changes

	Benchmark	`BASE`	`HEAD`	Efficiency
⚡	`test_into_backend_cache`	508.3 ms	245.7 ms	×2.1

_{Comparing Olamideod:docs/concept-pages (515dfa7) with main (69131a9)}

letsql · 2026-01-23T17:14:49Z

Docs preview: https://pr-1533-ba034dc46600913411f82cf2a2f8d2dbd08e84c2--letsql-docs.netlify.app

…ensure responsive design

letsql · 2026-01-23T23:04:26Z

Docs preview: https://pr-1533-a74157548a24b162381d54ea176c3865d93817df--letsql-docs.netlify.app

letsql · 2026-01-26T07:48:46Z

Docs preview: https://pr-1533-1a1c3f6482aa66e56f470b254a52970509041b8c--letsql-docs.netlify.app

letsql · 2026-01-26T08:08:26Z

Docs preview: https://pr-1533-08c6e56bec5bf7cd42b8a2882f5f8c996a1e0957--letsql-docs.netlify.app

Olamideod · 2026-01-26T08:15:52Z

Docs preview: https://pr-1533-08c6e56bec5bf7cd42b8a2882f5f8c996a1e0957--letsql-docs.netlify.app/

DianaHackmamba · 2026-01-27T17:03:51Z

docs/concepts/advanced_capabilities/feature_serving_patterns.qmd

+
+Hybrid computation balances the benefits of both batch and on-demand patterns while introducing its own complexity.
+
+**You gain**:


Watch out for bold text masquerading as headings.

DianaHackmamba · 2026-01-27T17:04:41Z

docs/concepts/advanced_capabilities/user_defined_functions.qmd

+
+### Benefits:
+
+- Extensibility: Add any logic you need; no waiting for Xorq to implement it.


Missing bold here on the terms before the colon.

DianaHackmamba · 2026-01-27T17:04:55Z

docs/concepts/advanced_capabilities/user_defined_functions.qmd

+
+### Costs:
+
+- Performance: UDFs are slower than built-in operations, typically 2-10x depending on the operation.


Make sure bold is applied consistently throughout.

DianaHackmamba · 2026-01-27T17:05:24Z

docs/concepts/core_concepts/point_in_time_correctness.qmd

Make sure you set up redirects for all of these.

DianaHackmamba · 2026-01-27T17:06:05Z

docs/concepts/execution_and_backends/multi_engine_execution.qmd

+
+Your data lives in PostgreSQL, but you need DuckDB's analytical performance for aggregations. Moving data manually between engines wastes time and introduces errors. Xorq's multi-engine execution lets you move data between backends within a single expression using `into_backend()`. This lets you use each engine for operations it performs best without manual data transfers.
+
+## What you'll understand


As I said elsewhere, if you need a summary like this, then the concept is too long. Make sure concepts are formatted consistently.

DianaHackmamba · 2026-01-27T17:06:33Z

docs/concepts/fundamentals/expression_format.qmd

+[Build system](../reproducibility/build_system.qmd) discusses how `xorq build` generates manifests. [Content-addressed hashing](../reproducibility/content_addressed_hashing.qmd) explains how manifests get unique hashes. [Compute catalog](../reproducibility/compute_catalog.qmd) details how manifests get registered and discovered.
+
+
+


Lots of empty lines here.

DianaHackmamba · 2026-01-27T17:07:44Z

docs/concepts/reproducibility/compute_catalog.qmd

+description: "Understand how the catalog enables discovery, versioning, and reuse of computations"
+---
+
+Three developers independently build customer segmentation features without knowing about each other's work. Each developer builds from scratch because they can't discover what others have already created in the team. Content hashes like `a3f5c9d2` sit in build directories where they remain invisible and unusable to other team members. The compute catalog solves this discovery problem by indexing builds with human-readable names, which enables team-wide discovery and reuse of computational work.


Intros should be simple and direct, not a meandering story.

xorq-labs#1531) **SUMMARY** Adds support for sklearn transformers whose output schema isn't known until fit time (OneHotEncoder, TfidfVectorizer, etc.) by using a KV-encoded format (Array[Struct{key, value}]). Along with some QOL restructuring. Key changes: - Structer: Added needs_target and is_series fields for transformer metadata - KVEncoder: Encode/decode between packed KV format and expanded columns - Registered: OneHotEncoder, TfidfVectorizer, SelectKBest - Removed from_fitted_step - logic now in FittedStep._deferred_fit_other NOTES: Sometimes we need to know if the transformer needs a target or not ; select K best this helped let us deprecate `from_fitted_step` , we also needed to be able to handle a series that is kv encoded to replicate the behavior of TfidfVectorizer. I think this sets up a cleaner pattern of registering structer's and routing the deferred function. --------- Co-authored-by: George Hoersting <ghoersti@Georges-MacBook-Pro.local> Co-authored-by: Claude <noreply@anthropic.com>

fixes xorq-labs#1529

letsql · 2026-01-29T17:29:21Z

Docs preview: https://pr-1533-515dfa7716e34f5fbc022cf920534f5dab76757d--letsql-docs.netlify.app

docs: populated and validated the concept pages

ba034dc

Olamideod requested review from DianaHackmamba and Copilot and removed request for Copilot January 23, 2026 16:57

Copilot started reviewing on behalf of Olamideod January 23, 2026 16:57 View session

Olamideod requested a review from hussainsultan January 23, 2026 16:59

Olamideod added the docs-preview label Jan 23, 2026

letsql bot removed the docs-preview label Jan 23, 2026

docs: Refine concept pages, restructure index, improve diagrams, and …

a741575

…ensure responsive design

Olamideod added the docs-preview label Jan 23, 2026

letsql bot removed the docs-preview label Jan 23, 2026

updated the landing page concept card link to the updated index page

1a1c3f6

Olamideod added the docs-preview label Jan 26, 2026

letsql bot removed the docs-preview label Jan 26, 2026

removed unused core-concept group

08c6e56

Olamideod added the docs-preview label Jan 26, 2026

letsql bot removed the docs-preview label Jan 26, 2026

DianaHackmamba requested changes Jan 27, 2026

View reviewed changes

ghoersti and others added 3 commits January 29, 2026 17:46

docs: adjust hero heading size and text wrapping (xorq-labs#1530)

ebfa0cd

fixes xorq-labs#1529

docs: unify formatting, shorten pages, applied review fixes

515dfa7

Olamideod added the docs-preview label Jan 29, 2026

letsql bot removed the docs-preview label Jan 29, 2026


		Hybrid computation balances the benefits of both batch and on-demand patterns while introducing its own complexity.

		You gain:


		### Benefits:

		- Extensibility: Add any logic you need; no waiting for Xorq to implement it.


		### Costs:

		- Performance: UDFs are slower than built-in operations, typically 2-10x depending on the operation.


		Your data lives in PostgreSQL, but you need DuckDB's analytical performance for aggregations. Moving data manually between engines wastes time and introduces errors. Xorq's multi-engine execution lets you move data between backends within a single expression using `into_backend()`. This lets you use each engine for operations it performs best without manual data transfers.

		## What you'll understand

		[Build system](../reproducibility/build_system.qmd) discusses how `xorq build` generates manifests. [Content-addressed hashing](../reproducibility/content_addressed_hashing.qmd) explains how manifests get unique hashes. [Compute catalog](../reproducibility/compute_catalog.qmd) details how manifests get registered and discovered.

Conversation

Olamideod commented Jan 23, 2026

Uh oh!

codecov bot commented Jan 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

codspeed-hq bot commented Jan 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Merging this PR will improve performance by ×2.1

Performance Changes

Uh oh!

letsql bot commented Jan 23, 2026

Uh oh!

letsql bot commented Jan 23, 2026

Uh oh!

letsql bot commented Jan 26, 2026

Uh oh!

letsql bot commented Jan 26, 2026

Uh oh!

Olamideod commented Jan 26, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

letsql bot commented Jan 29, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

codecov bot commented Jan 23, 2026 •

edited

Loading

codspeed-hq bot commented Jan 23, 2026 •

edited

Loading