Skip to content

Commit 6c79d6d

Browse files
committed
release: v0.3.0
Bump all package versions to 0.3.0 and update documentation. Changes: - CHANGELOG.md: Add v0.3.0 release notes with all new features - Cargo.toml: Bump workspace version and dependencies to 0.3.0 - Package versions: core, react, wasm packages updated to 0.3.0 - docs/api_schema.json: Regenerate schema for new TemplateConfig types - READMEs: Document dbt/Jinja templating, template CLI flags, expanded dialect support
1 parent 2b47984 commit 6c79d6d

13 files changed

Lines changed: 769 additions & 523 deletions

File tree

CHANGELOG.md

Lines changed: 46 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7,23 +7,69 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
77

88
## [Unreleased]
99

10+
## [0.3.0] - 2026-01-22
11+
1012
### Added
1113

1214
#### Core Engine (flowscope-core)
15+
- **Jinja/dbt templating support**: MiniJinja-based preprocessing for dbt projects
16+
- Built-in dbt macros: `ref()`, `source()`, `config()`, `var()`, `is_incremental()`
17+
- RelationEmulator for dbt Relation object attribute access (`.schema`, `.identifier`)
18+
- `this` global variable and `execute` flag for templates
19+
- Custom macro passthrough stubs for graceful handling
20+
- **COPY statement lineage**: Track source/target tables in COPY/COPY INTO (PostgreSQL, Snowflake)
21+
- **ALTER TABLE RENAME lineage**: Track table renames as dataflow edges
22+
- **UNLOAD statement lineage**: Track source tables from Redshift UNLOAD statements
23+
- **Lateral column alias support**: Resolve aliases in same SELECT list (BigQuery, Snowflake, DuckDB, etc.)
24+
- **Backward column inference**: Infer columns for SELECT * without schema from downstream usage
1325
- Type inference for SQL expressions with comprehensive type checking
1426
- New `TYPE_MISMATCH` warning code for detecting incompatible type comparisons and operations
1527
- Schema-aware column type lookup - column references now resolve types from provided schema metadata
1628
- CTE column type propagation to outer queries
1729
- Dialect-aware type compatibility rules (e.g., Boolean/Integer comparison allowed in MySQL but not PostgreSQL)
1830
- NULL comparison anti-pattern detection (`= NULL` warns to use `IS NULL` instead)
1931

32+
#### Completion API
33+
- Smart function completions with signature metadata (params, return types, categories)
34+
- Context-aware scoring: boost aggregates in GROUP BY, window functions in OVER clauses
35+
- Lateral column alias extraction with proper scope isolation
36+
- Type-aware column scoring in comparison contexts
37+
- Dialect-aware keyword parsing using sqlparser tokenizer
38+
- CASE expression type inference from THEN/ELSE branches
39+
40+
#### CLI (flowscope-cli)
41+
- `--template jinja|dbt` flag for templated SQL preprocessing
42+
- `--template-var KEY=VALUE` for template variable injection
43+
- `--metadata-url` for live database schema introspection (PostgreSQL, MySQL, SQLite)
44+
- `--metadata-schema` for schema filtering during introspection
45+
46+
#### React Package (@pondpilot/flowscope-react)
47+
- Web workers for graph/matrix/layout computations (improved UI responsiveness)
48+
- LayoutProgressIndicator component for visual layout feedback
49+
- Debug flags (GRAPH_DEBUG, LAYOUT_DEBUG) for performance diagnostics
50+
51+
#### Web App (app/)
52+
- Template mode selector for dbt/Jinja SQL preprocessing in the toolbar
53+
- Issue-to-editor navigation: click issues to jump to source location
54+
- Issues tab filtering: filter by severity, error code, and file
55+
- Stats popover: complexity dots trigger dropdown with table/column/join counts
56+
- Clear analysis cache option in project menu
57+
- Bundled "dbt Jaffle Shop" demo project showcasing ref/source/config/var
58+
2059
### Changed
2160

2261
#### Core Engine (flowscope-core)
2362
- Unified type system: `CanonicalType` replaces internal `SqlType` enum with broader coverage (Time, Binary, Json, Array)
2463
- `OutputColumn.data_type` now populated with inferred types for SELECT expressions
2564
- Type checking now accepts dialect parameter for dialect-specific rules
2665

66+
### Fixed
67+
68+
#### Core Engine (flowscope-core)
69+
- dbt model cross-statement linking with proper case normalization for Snowflake
70+
- DDL-seeded schema preservation (schemas no longer overwritten by later queries)
71+
- UTF-8 safety in `should_show_for_cursor` when cursor offset is mid-character
72+
2773
### Known Limitations
2874

2975
#### Type Inference

Cargo.toml

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -8,15 +8,15 @@ members = [
88
resolver = "2"
99

1010
[workspace.package]
11-
version = "0.2.0"
11+
version = "0.3.0"
1212
authors = ["PondPilot Team"]
1313
edition = "2021"
1414
license = "Apache-2.0"
1515
repository = "https://github.com/pondpilot/flowscope"
1616

1717
[workspace.dependencies]
18-
flowscope-core = { version = "0.2.0", path = "crates/flowscope-core", default-features = false }
19-
flowscope-export = { version = "0.2.0", path = "crates/flowscope-export", default-features = false }
18+
flowscope-core = { version = "0.3.0", path = "crates/flowscope-core", default-features = false }
19+
flowscope-export = { version = "0.3.0", path = "crates/flowscope-export", default-features = false }
2020
sqlparser = "0.59"
2121
serde = { version = "1.0", features = ["derive", "rc"] }
2222
serde_json = "1.0"

README.md

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -30,6 +30,7 @@ Features:
3030
- Drag and drop SQL files or paste queries directly
3131
- Interactive lineage graph with table and column-level views
3232
- Multi-file project support with schema DDL
33+
- dbt/Jinja template preprocessing for dbt models
3334
- Export to Mermaid, JSON, CSV, Excel, or HTML reports
3435
- All processing happens in your browser — your SQL never leaves your machine
3536

@@ -67,7 +68,8 @@ See [CLI documentation](crates/flowscope-cli/README.md) for all options.
6768
## Key Features
6869

6970
- Client-side analysis with zero data egress
70-
- Multi-dialect coverage (PostgreSQL, Snowflake, BigQuery, DuckdDB and more)
71+
- Multi-dialect coverage (PostgreSQL, Snowflake, BigQuery, DuckDB, Redshift, and more)
72+
- dbt and Jinja templating support with built-in macro stubs (`ref()`, `source()`, `var()`)
7173
- Table and column lineage with schema-aware wildcard expansion
7274
- Structured diagnostics with spans for precise highlighting
7375
- Completion API for SQL authoring workflows

crates/flowscope-cli/README.md

Lines changed: 28 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -62,6 +62,9 @@ Options:
6262
(e.g., postgres://user:pass@host/db, mysql://..., sqlite://...)
6363
--metadata-schema <SCHEMA>
6464
Schema name to filter when using --metadata-url
65+
--template <MODE> Template preprocessing mode [possible values: jinja, dbt]
66+
--template-var <KEY=VALUE>
67+
Template variable (can be repeated)
6568
-o, --output <FILE> Output file (defaults to stdout)
6669
--project-name <PROJECT_NAME>
6770
Project name used for default export filenames [default: lineage]
@@ -70,7 +73,7 @@ Options:
7073
-v, --view <VIEW> Graph detail level for mermaid output [default: table]
7174
[possible values: script, table, column, hybrid]
7275
-q, --quiet Suppress warnings on stderr
73-
--compact Compact JSON output (no pretty-printing)
76+
-c, --compact Compact JSON output (no pretty-printing)
7477
-h, --help Print help
7578
-V, --version Print version
7679
```
@@ -132,3 +135,27 @@ This feature requires the `metadata-provider` feature flag (enabled by default).
132135
```bash
133136
cargo build -p flowscope-cli --no-default-features
134137
```
138+
139+
### dbt and Jinja Templating
140+
141+
FlowScope can preprocess SQL files that use Jinja or dbt-style templating before analysis. This enables lineage extraction from dbt models without running dbt itself.
142+
143+
```bash
144+
# Analyze a dbt model with ref() and source() macros
145+
flowscope --template dbt models/orders.sql
146+
147+
# Pass variables to templates
148+
flowscope --template dbt --template-var target_schema=prod models/*.sql
149+
150+
# Plain Jinja templating (strict mode - undefined variables cause errors)
151+
flowscope --template jinja --template-var env=production query.sql
152+
```
153+
154+
**dbt mode** includes stub implementations of common macros:
155+
- `ref('model')` / `ref('project', 'model')` - model references
156+
- `source('schema', 'table')` - source table references
157+
- `config(...)` - model configuration (returns empty string)
158+
- `var('name')` / `var('name', 'default')` - variable access
159+
- `is_incremental()` - always returns false for static analysis
160+
161+
Variables passed via `--template-var` are accessible in dbt mode through `var()` and in Jinja mode directly as template variables.

crates/flowscope-core/README.md

Lines changed: 7 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -8,12 +8,15 @@ Core SQL lineage analysis engine for FlowScope.
88

99
## Features
1010

11-
- **Multi-Dialect Parsing:** Built on `sqlparser-rs`, supporting PostgreSQL, Snowflake, BigQuery, and Generic ANSI SQL.
11+
- **Multi-Dialect Parsing:** Built on `sqlparser-rs`, supporting PostgreSQL, Snowflake, BigQuery, DuckDB, Redshift, MySQL, SQLite, Databricks, ClickHouse, and Generic ANSI SQL.
1212
- **Deep Lineage Extraction:**
13-
- Table-level dependencies (SELECT, INSERT, UPDATE, MERGE, etc.)
13+
- Table-level dependencies (SELECT, INSERT, UPDATE, MERGE, COPY, UNLOAD, etc.)
1414
- Column-level data flow (including transformations)
15-
- **Complex SQL Support:** Handles CTEs (Common Table Expressions), Subqueries, Joins, Unions, and Window Functions.
16-
- **Schema Awareness:** Can utilize provided schema metadata to validate column references and resolve wildcards (`SELECT *`).
15+
- Cross-statement lineage tracking (CREATE TABLE AS, INSERT INTO ... SELECT)
16+
- **dbt/Jinja Templating:** Preprocess SQL with Jinja or dbt-style templates before analysis, with built-in stubs for `ref()`, `source()`, `config()`, `var()`, and `is_incremental()`.
17+
- **Complex SQL Support:** Handles CTEs (Common Table Expressions), Subqueries, Joins, Unions, Window Functions, and lateral column aliases.
18+
- **Schema Awareness:** Utilize provided schema metadata to validate column references and resolve wildcards (`SELECT *`).
19+
- **Type Inference:** Infer expression types with dialect-aware type compatibility checking.
1720
- **Diagnostics:** Returns structured issues (errors, warnings) with source spans for precise highlighting.
1821

1922
## Structure

0 commit comments

Comments
 (0)