GTC-3442: Catch rasterio exceptions on missing data for integrated drivers#249
GTC-3442: Catch rasterio exceptions on missing data for integrated drivers#249dmannarino merged 5 commits intostagingfrom
Conversation
|
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## staging #249 +/- ##
===========================================
- Coverage 70.95% 70.89% -0.07%
===========================================
Files 78 78
Lines 2734 2738 +4
===========================================
+ Hits 1940 1941 +1
- Misses 794 797 +3
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
jterry64
left a comment
There was a problem hiding this comment.
I think this looks right, but I'd check it on a map on staging before pushing to prod... I vaguely remember a reason I didn't use self.no_data, and I should've written down. Sorry! But it may have just been unnecessary. Is this the case where it's outside of the extent of the layer?
Yep. |
| """ | ||
|
|
||
| mask = self.alert_drivers > 0 | ||
| mask = ~self.no_data * (self.alert_drivers > 0) |
There was a problem hiding this comment.
good catch of the dark pixels issue
| f"s3://{DATA_LAKE_BUCKET}/wur_integration_alert_drivers_class/{version}/raster/epsg-4326/cog/class.tif" | ||
| ) as reader: | ||
| if reader.tile_exists(tile_x, tile_y, zoom): | ||
| try: |
There was a problem hiding this comment.
That's too bad that the reader.tile_exists() call doesn't catch all particular cases of a TileOutsideBounds requests. I'm just wondering if you could do a ChatGPT/gemini query to see if there is a better function or condition check that would figure out when the access will be out of bounds that wouldn't require catching an exception. But I understand the exception catching is working, so no need to fix unless you find something that seems solid.
There was a problem hiding this comment.
Mmm, worth exploring, thanks! Will do as a follow-up though.
It turns out that I we were emitting 500s for all tiles outside the bounds of the associated datasets, rather than polite 404s.
Pull request type
Please check the type of change your PR introduces:
What is the current behavior?
Trying to generate alerts tiles where they don't exist raises 500s.
Issue Number: GTC-3442
What is the new behavior?
Does this introduce a breaking change?
Other information