Skip to content

"dbt seed" sometimes fails to deactivate the connection #35

@nicoweidner

Description

@nicoweidner

I have the following basic test_seed.csv file in the seeds folder:

field1,field2,field3
value11,value21,1
value12,value22,2

I ran dbt seed twice (testing for issue 33). The second time, I received this error:

15:50:53  Unhandled error while executing seed.example.test_seed
Decodable: ResourceAlreadyExists: {'timestamp': '2024-01-14T15:50:53.806+00:00', 'message': 'Could not change the job state from [STARTING] to [STOPPING], probably because an existing operation is in progress. Please wait for the operation to complete.'}

At the same time, the old records were gone and the stream was populated with two new records. So it looks like:

  • The stream was cleared
  • The connection was restarted to repopulate the stream
  • The stream was repopulated

(Unexpected behavior starts here)

  • The attempt to deactivate the connection failed as it was still in STARTING state

While it is certainly unexpected that the connection already produced records while still being in STARTING state, the dbt client should wait until it observes RUNNING state before deactivation.

Side note: Looks like all 409 errors are classified as ResourceAlreadyExists. That's not precise, as this example shows. Created #36 to cover that.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions