Skip to content

fix: Updates to fail gracefully if there are no assets to resolve key_error #293

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
botanical merged 2 commits into from
Feb 13, 2025
Merged

fix: Updates to fail gracefully if there are no assets to resolve key_error #293

Changes from 1 commit
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
060488d
fix: change condition in remove_thumnail_asset to resolve key error f…
botanical Jan 29, 2025
1109fc3
fix: update to fail safely instead
botanical Feb 12, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion dags/veda_data_pipeline/groups/processing_tasks.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ def remove_thumbnail_asset(ti):
if assets.get("thumbnail"):
assets.pop("thumbnail")
# if thumbnail was only asset, delete assets
if not assets:
if not assets and (assets is not None):
Copy link
Contributor

@anayeaye anayeaye Jan 29, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need to update line 24 too, so that we can test if assets exists

assets = payload.get("assets", {}) should be assets = payload.get("assets", None)

EDIT
I think having an assets key at all is a problem for the existing pipeline because the pipeline and stac schemas have conflicting assets meanings--so this level of logic about other non-thumbnail assets seems unhelpful. @ividito @botanical should we just check if assets is in payload.keys() and then if it is payload.pop("assets")?

Copy link
Contributor

@smohiudd smohiudd Jan 29, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree that the current logic is awkward. I also think we should just pop the assets from the payload.

payload.pop("assets")
return payload

Expand Down