I often have to rebuild the docker container for RunSQL to work after making a change

[CampaginAgent]

This has been driving my crazy, but I can't quite work out steps to reproduce.

I'm often seeing "odd" things happen with the Run SQL button, and the resolution is always to "Rebuild Container" and then it works again.

I'm running DBT PowerUser (0.10.1) in a container: ghcr.io/dbt-labs/dbt-snowflake:1.3.latest I'm on Windows 11 in VSCode

Example 1:
Add a new source

1. Create a new folder.
2. Add a schema.yml file defining my source
3. Build the base .sql file
4. Hit "Run SQL" and I get an error that my source isn't defined.
5. Rebuild Dev container - and hit Run SQL again - everything works.

Similar things happen when adding a new model which is a parent of my current model.

I've tried all sorts of combinations of dbt-build, dbt-compile, etc and I still get the same error until I rebuild my docker contianer.

It's like something is being cached somewhere - OR, am I misunderstanding how RunSQL should work.

What am I missing?

[mdesmet]

Which error do you get exactly?

[CampaginAgent]

There are a lot of different errors, depending on the nature of the change. Here is one change I just made

1. Open schema.yml and change "sources: => - name: " from "cas" to "xxx"
2. Open my base .sql file and change select * from {{ source('cas', 'realistings') }} to select * from {{ source('xxx', 'realistings') }}
3. Hit the Execute DBT SQL file button

The error returned is

Runtime Error
Runtime Error Compilation Error in sql operation t_b34e9b749bda4f0fadf69735f3985b78 (from remote system.sql) Sql Operation 'sql operation.cas_datawarehouse_2.t_b34e9b749bda4f0fadf69735f3985b78' (from remote system.sql) depends on a source named 'xxx.realistings' which was not found

I've tried the following

• The "run dbt model" button
• dbt compile
• dbt run
• dbt build

and they all "work" (they don't return any errors), however the "Execute DBT SQL" button will always return something like the above error until I run "rebuild container" - then it works as expected.

---

This also happens with several other operations, such as adding a new column to a parent (ephemeral) model - in which case I get an error like "Database Error 000904 (42000): SQL compilation error: error line 90 at position 8 invalid identifier 'LISTINGS.LEASED_DATE'" - where as a "dbt run" works perfectly, and the "Execute DBT SQL" works after I rebuild the continer.

dave

[CampaginAgent]

I've been doing a lot more investigation and I'm struggling to figure out what's going on.

Any changes that I make on files in the host (vs-code) are immediately visible within the container (eg, when I cat the file to view it in the docker, its updated)

I've tried every form of dropping & re-building my docker containers/images/caches that I know of. It's now happening almost all of the time for me, and occasionally (once a week or less) for someone else at my office.

My dockerfile is based on ghcr.io/dbt-labs/dbt-snowflake:1.3.latest - and there isn't too much interesting in there.

It feels like execute DBT SQL get's the code from a cache that's not being invalidated - but I'm not a python programmer so that's a bit out of my league.

I've also noticed that restarting VSCode forces execute DBT SQL to get the latest underlying changes in the same way that rebuilding the dev container - ie, it "fixes" the issue (until the next time it occurs)

Any other suggestions? I'm happy to run a debug build or whatever to get you more info on what's going on under the hood.

Cheers,
dave

[anandgupta42]

@CampaginAgent Are you still experiencing this issue? I couldn't replicate the problem since I can't access a Windows machine.

[CampaginAgent]

Yes, it's still an issue. Tested a few minutes ago using:
VS Code 1.8.1.1
DBT Power User v0.16.2
Docker image ghcr.io/dbt-labs/dbt-snowflake:1.5.2
Windows 11

The repository folder is on the host and we are using DevContainers.