Adding handling of Nulled lists to beam_row_from_dict#33830
Adding handling of Nulled lists to beam_row_from_dict#33830ahmedabu98 merged 2 commits intoapache:masterfrom
Conversation
|
Checks are failing. Will not request review until checks are succeeding. If you'd like to override that behavior, comment |
|
Run Python_Dataframes PreCommit 3.12 |
|
Assigning reviewers. If you would like to opt out of this review, comment R: @tvalentyn for label python. Available commands:
The PR bot will only process comments in the main thread (not review comments). |
|
will defer to @ahmedabu98 who authored this code. |
ahmedabu98
left a comment
There was a problem hiding this comment.
Thanks for adding this change! Just had one suggestion
Co-authored-by: Ahmed Abualsaud <65791736+ahmedabu98@users.noreply.github.com>
|
@ahmedabu98 Added the change you suggested, thanks for that makes it cleaner. |
ahmedabu98
left a comment
There was a problem hiding this comment.
Thanks for adding to the repo @TobiasBredow :)
LGTM, will merge when tests pass
* Adding handling of Nulled lists to beam_row_from_dict * Update sdks/python/apache_beam/io/gcp/bigquery_tools.py Co-authored-by: Ahmed Abualsaud <65791736+ahmedabu98@users.noreply.github.com> --------- Co-authored-by: Ahmed Abualsaud <65791736+ahmedabu98@users.noreply.github.com>
* Adding handling of Nulled lists to beam_row_from_dict * Update sdks/python/apache_beam/io/gcp/bigquery_tools.py Co-authored-by: Ahmed Abualsaud <65791736+ahmedabu98@users.noreply.github.com> --------- Co-authored-by: Ahmed Abualsaud <65791736+ahmedabu98@users.noreply.github.com>
Addresses #32155 and fixes the handling of missing repeated fields inside a dicts when converting a dict to a beam row. As described in the issue it matches BigQuery standard behaviour of converting Null repeated fields to empty arrays on insert.
https://cloud.google.com/bigquery/docs/reference/standard-sql/data-types#array_nulls
Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:
addresses #123), if applicable. This will automatically add a link to the pull request in the issue. If you would like the issue to automatically close on merging the pull request, commentfixes #<ISSUE NUMBER>instead.CHANGES.mdwith noteworthy changes.See the Contributor Guide for more tips on how to make review process smoother.
To check the build health, please visit https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md
GitHub Actions Tests Status (on master branch)
See CI.md for more information about GitHub Actions CI or the workflows README to see a list of phrases to trigger workflows.