Skip to content

Commit edc17a8

Browse files
authored
Update read_dlt_delta() to read "source_database" instead of "database"
onboard_dataflowspec.py sets key "source_database" when creating bronze dataflow spec dataframe inside source details. However read_dlt_delta tries to read "database" instead.
1 parent 2a93dd9 commit edc17a8

File tree

1 file changed

+2
-2
lines changed

1 file changed

+2
-2
lines changed

src/pipeline_readers.py

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -62,14 +62,14 @@ def read_dlt_delta(spark, bronze_dataflow_spec) -> DataFrame:
6262
if reader_config_options and len(reader_config_options) > 0:
6363
return (
6464
spark.readStream.options(**reader_config_options).table(
65-
f"""{bronze_dataflow_spec.sourceDetails["database"]}
65+
f"""{bronze_dataflow_spec.sourceDetails["source_database"]}
6666
.{bronze_dataflow_spec.sourceDetails["table"]}"""
6767
)
6868
)
6969
else:
7070
return (
7171
spark.readStream.table(
72-
f"""{bronze_dataflow_spec.sourceDetails["database"]}
72+
f"""{bronze_dataflow_spec.sourceDetails["source_database"]}
7373
.{bronze_dataflow_spec.sourceDetails["table"]}"""
7474
)
7575
)

0 commit comments

Comments
 (0)