Skip to content

Handle Raised Exception To Continue Sync#289

Merged
cmhedrickREI merged 7 commits intomainfrom
4624-attr-error-fix
May 29, 2025
Merged

Handle Raised Exception To Continue Sync#289
cmhedrickREI merged 7 commits intomainfrom
4624-attr-error-fix

Conversation

@cmhedrickREI
Copy link
Member

Pull Request

Related to GSA/data.gov#4624

About

An analysis of the issue:
The old and current rebuild calls out to our package_create which then calls out to a several other functions such as rollup_save_action then to our translate_spatial. Within the translate spatial we then call the json.dump which we do wrap in an try/except, however this caused the rebuild to fail in the update which then led to the rebuild raising the exception and then holding up the index. Switching out the raise from the original CKAN code we will instead log the package_ids of failing datasets while not holding up the entire rebuild.

I also added a test case in which we take a bad json file that I think helps test bad json that indirectly helps support this change while not directly connected with the rebuild, it does help test for JSON failing json which could trigger the issue with old harvester.

  • Adds make command down to spin down containers.
  • Adds minor lint change
  • Cleans up docker compose file to get rid of version warning

PR TASKS

  • The actual code changes.
  • Tests written and passed.
  • Any changes to docs?
  • Bumped version number in setup.py (also checked on PyPi).

@cmhedrickREI cmhedrickREI changed the title 4624 attr error fix Handle Raised Exception To Continue Sync May 28, 2025
@cmhedrickREI cmhedrickREI requested a review from a team May 28, 2025 19:12
Copy link
Contributor

@neilmb neilmb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, just quibbles.

@cmhedrickREI cmhedrickREI merged commit d41b82e into main May 29, 2025
5 checks passed
@cmhedrickREI cmhedrickREI deleted the 4624-attr-error-fix branch May 29, 2025 13:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants