Fusion Diaries — February 2026 #1306
dataders
announced in
Announcements
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
-
Intro
Hi everyone, it's been 52 working days since the last Fusion diaries in November. So where are we at from a high-level? Humor me with this simile:
Just as the Eastern seaboard of the US is emerging from a polar vortex with an eye on inevitable spring, the dbt Fusion engine team is emerging from the depths of complex long-tail bugs with an eye on inevitable General Availability!
But we're getting ahead of our skis here, this is a Fusion diary! In the diarist spirit, we'd love to share the usual tranche of big rocks, work-in-progress, dragons (both at large and vanquished), what to read, and (as always) a meme.
TL;DR
Velocity
preview.73topreview.119)Per usual, for specifics, check out dbt-fusion's CHANGELOG for more information.
Big rocks
The Final Push to General Availability
Eight months ago, we published this blog: The Path to GA: How the dbt Fusion engine rolls out from beta to production.
Looking back at it, I'm impressed with how prescient it was. Especially this Joel Labes classic:
If we were to add the below items to the bajillion undocumented features, the correct total is closer to gazillion:
In the coming weeks, we'll share a clear-eyed picture of the remaining work. In the meantime you can check out the dbt-fusion repo Milestones that we'll be refining over the coming weeks.
Big Rocks: What shipped since November
Package Conformance and Hub updates
Exciting update: all but two of the top 50 most popular packages now work on Fusion!
Not only that, but now the dbt Package hub (hub.getdbt.com) will provide you with Fusion compatibility information based on not only what the package maintainer specifies but also on tests that we now perform in the backend.
If you are a maintainer of a package, or waiting for a package you depend upon to get compatible with Fusion, check out this great guide.
On top of all of that,
dbt-autofix packageshas continued to improve since November. It auto-upgrades your packages to the lowest Fusion compatible version available. Make sure you've got the latest version and give it a whirl!Model Governance (Versioning, Contracts, and Access Groups)!
We've had most of the support for model governance for a while now but haven't yet closed #25 because of one last remaining piece:
deprecation_date. But now that's shipped we can call this done!New, more tolerant CSV parser
In Fusion's earliest days, we chose arrow-csv as the csv parser. It's highly performant and Arrow-native, which was great for integration into the static analysis pipeline. However it is also much more strict than Python's agate library, which was holding a lot of the community back from adopting Fusion!
Just like YAML before it, we have introduced our own new Rust crate: dbt-csv!
This is no easy task, reading CSVs is notoriously difficult (see: So You Want To Write Your Own CSV code?). Instead of giving up, we leaned in to ship a brand new csv parser that's way more user friendly and tolerant of kooky problems that "real-world" CSVs are wont to have.
If you've had trouble with
seed-ing.csv's, most issues should go away after upgrading to2.0.0-preview.118. But reach out if you're still seeing something weird!#1004 tracks the remaining work (it’s not much!)
in Preview: Python models!
We haven't made a big announcement (yet), since we're still testing out this functionality in real-world projects with help from folks in the community.
Some known limitations are tracked in #1294 and we're working to ensure the equivalent changes land in dbt Core.
Beyond that, the big remaining piece of work before GA is figuring out how "static analysis" can/should work with Python models (which, for obvious reasons, can't be analyzed as SQL). See this discussion for more: Source schemas should be first-class, versioned artifacts · Discussion #1042
More than anything the signal we need to call them "GA" is that more folks try them out and tell us if they're working as expected. So please give them a go and report back.
in Preview: Semantic Layer in Fusion now
The Semantic layer team has worked tirelessly to not only get the semantic layer shipped to Fusion, but even more importantly, ship updates to the spec that makes authoring easier.
The new authoring experience and spec is planned to release in dbt Core 1.12, but it's available now in Fusion.
Learn more in the blog post: Modernizing the Semantic Layer Spec (but note the related dragon below with respect to
dbt docs generateand thecatalog.json).Modernizing the Semantic Layer Spec
in alpha: "Lazy" Compilation
In previous Fusion diaries, we've talked about "incremental compilation" in the VS Code extension. The use case is this:
Our first swing at addressing this problem was incremental compilation. You still run a full compile as usual, but, after that first compile completes, Fusion only needs to check the file you've modified and its descendants. This was a big performance increase: Our internal dbt project took four minutes to fully compile, but only 10 seconds to incrementally recompile after changing one file.
The idea of lazy compilation is to drop that first full compile, and only ever have the VS Code extension and language server analyze the models that you are currently editing.
What's shipped now is that when you open your first model in VS Code, that model (and its parents and children) are analyzed first. The full background compile waits until after the initial lazy compilation is complete.
The ultimate goal is to make feedback as performant and specific as possible to keep you in "flow state" (credit: 👑 Sung).
We aspire towards truly lazy compilation in which only edited models are analyzed, but this is a great first step.
To try this out, set
DBT_LSP_LAZY_COMPILATION_ENABLED=1in the VS Code extension (guide for how to do so). Let us know what you think!Screen.Recording.2026-02-13.at.10.54.02.AM.mov
🚧 Work in progress
Heads up on features currently underway
We hope in the coming weeks we'll have the following available. We'll give more of a shout once they're closer to ready!
dbt retry--fail-fast"Baseline mode"
Something we're hoping to put in Fusion users hands next week is something we're calling "baseline mode". We've observed that migrating to Fusion isn't just moving your YAML around to be compatible with the new authoring layer. For some customers it can be more involved especially in these scenarios:
STRUCT,ARRAY,GEOGRAPHY) or built-in functions (AI.PREDICT,JSON_FLATTEN,st_pointfromgeohash) that aren't yet supported by the dbt-fusion engine.The way Fusion, the VS Code extension and the Language Server are built are such that you are very limited with respect to what Fusion features you can get.
So we went back to the drawing board with a goal to create a smoother transition. Expect a discussion that goes deeper on this next week!
🐉 Dragons
Deferral!!!
There's a dragon afoot when it comes to deferral. The big challenge is that in Core, there are only ever two states of a given model:
With Fusion, we've introduced a third place: the schema and logical plan for your models that were generated as a result of static analysis (living in
target/db/schemas/...)This can manifest as errors like
dbt1053(can't find a locally cached schema, because we're deferring to prod and it was never locally cached) ordbt1014(trying to introspect the dev schema since the local cache is missing, but there's nothing to introspect because we're deferring to prod). We're laser focused on this now, but we wanted to call this out to y'all.#807
Docs generate for those who have adopted the new SL spec
We're almost ready to announce the new way to create a
catalog.jsonwhich is a key artifact that powers what we call dbt docs and dbt Platform Catalog. However, there's still some kinks to iron out.Until Fusion can properly generate a
catalog.json, Platform users may experience some roadbumps in keeping their Catalog updated. Catalog uses a Core-powered docs experience, which requires a job runningdbt docs generate. This command only works on Core, so if you are using Fusion on Platform and have a job with documentation, under the hood this is being run with Core.This may work for your project, but there are some cases where Core throws an error on this command. Here are some scenarios you may observe:
'ref' is undefinedor references to jinja in.mdor.ymlfilesThis is a short-term degradation, the Fusion
docs generatecommand will soon be released and Core will soon have the ability to generatecatalog.jsonon Fusion enabled projects.If you're a dbt Platform customer on Fusion and experiencing this issue, please reach out to support and we can deploy a hotfix for you.
👓 Stuff you should read
The Compiler Is Your Best Friend, Stop Lying to It - Daniel Beskin's Blog
I've been reading a lot about software engineering best practices—partly to LARP as an engineer, and partly to build a clearer mental model for how the dbt Fusion engine will shape the developer experience.
So my brain absolutely lit up when I read this article. It goes into how it's an adjustment to get used to compilers and that initial instincts to squelch the compiler's nagging end up biting you in the long run. Instead you have to evolve your mindset to consider the compiler as your friend.
So for those who have been using Fusion or are Fusion-curious, please give a read and let me know what resonates with you!
🏁 Made it to the meme
Not only is Valentine's Day this week but it's also the 10th anniversary of Apache Arrow. In celebration of both, we present to you not a meme but a Valentine's day card replete w/ a dad joke.
Apache Arrow is crucial to dbt's reinvention as dbt Fusion; it's the predominant data format throughout. But more broadly, Arrow dramatically lowers the bar for vendors to building and maintaining query engines. Most importantly, data practitioners' lives improve as a result of the performance improvement that Arrow brings over archaic data protocols.
p.s. one more thing…
🦆 quack check out
2.0.0-preview.119quack 🦆Beta Was this translation helpful? Give feedback.
All reactions