Multi-feature search support added to CMR api calls#1109
Multi-feature search support added to CMR api calls#1109tiagodc wants to merge 7 commits intonsidc:mainfrom
Conversation
chuckwondo
left a comment
chuckwondo
left a comment
There was a problem hiding this comment.
@tiagodc, thank you for your contribution, and apologies that it has taken so long to get it reviewed.
I've made comments, but I've also solicited some other maintainers for input as well, so let's see what they have to say about my suggestions and questions.
| points = [] | ||
|
|
||
| for x, y in lon_lat_pairs: | ||
| self.point(x, y) | ||
| points.append(self.params.pop('point')[0]) | ||
|
|
||
| self.params['point'] = points |
There was a problem hiding this comment.
This should be unnecessary because the point method already performs an append each time it is called. That is, the point method is already implemented to support being called multiple times, adding a new point to the list of points on each call.
Therefore, this should be all that's required:
| points = [] | |
| for x, y in lon_lat_pairs: | |
| self.point(x, y) | |
| points.append(self.params.pop('point')[0]) | |
| self.params['point'] = points | |
| for x, y in lon_lat_pairs: | |
| self.point(x, y) | |
| """ | ||
| return super().point(lon, lat) | ||
|
|
||
| def multipoint(self, lon_lat_pairs: Sequence[PointLike]) -> Self: |
There was a problem hiding this comment.
Please add an or_ kwarg so the caller can explicitly set the option, instead of assuming it should be set to True (and please update the docstring to include it):
| def multipoint(self, lon_lat_pairs: Sequence[PointLike]) -> Self: | |
| def points(self, lon_lat_pairs: Sequence[PointLike], *, or_: bool | None = None) -> Self: |
I would prefer or_ to be a kwarg because passing a bool value without a keyword is not intuitive.
However, making it a positional arg allows for it to be set like so (whereas this is not possible when or_ is a kwarg):
earthaccess.search_data(
short_name="ATL06",
polygons=(polygons, True),
)Again, it's not intuitive as to what True means here, so I'd like some input from others on this point (cc: @betolink, @mfisher87, @jhkennedy).
I don't like the "bare" boolean argument, but on the other hand, I don't like that (sub)kwargs cannot be specified via search_data.
In order to pass or_ when it is a kwarg, the caller cannot use search_data, and must instead directly construct a DataGranules instance, like so:
query = (
DataGranules()
.short_name("ATL06")
.polygons(polygons, or_=True)
)(Alternatively, renaming or_ to any_ might make it even more readable -- i.e., any/all is more intuitive than or/and, no?)
| points.append(self.params.pop('point')[0]) | ||
|
|
||
| self.params['point'] = points | ||
| self.options['point'] = {'or': True} |
There was a problem hiding this comment.
I would prefer not to assume the caller wants the or option to be set to True, so let's add an or_ kwarg, defaulted to None:
| self.options['point'] = {'or': True} | |
| self.option("point", "or", or_) |
| """ | ||
| return super().polygon(coordinates) | ||
|
|
||
| def multipolygon(self, multi_coordinates: Sequence[Sequence[PointLike]]) -> Self: |
There was a problem hiding this comment.
Please add an or_ kwarg, as I've described in my comment on the multipoint method:
| def multipolygon(self, multi_coordinates: Sequence[Sequence[PointLike]]) -> Self: | |
| def polygons(self, polygons_: Sequence[Sequence[PointLike]], *, or_: bool | None = None) -> Self: |
| polygons.append(self.params.pop('polygon')) | ||
|
|
||
| self.params['polygon'] = polygons | ||
| self.options['polygon'] = {'or': True} |
There was a problem hiding this comment.
| self.options['polygon'] = {'or': True} | |
| self.option("polygon", "or", or_) |
| """Filter by granules that overlap any bounding box from an input list. | ||
|
|
||
| Parameters: | ||
| boxes: list of tuples of (lower_left_lon, lower_left_lat, upper_right_lon, upper_right_lat) |
There was a problem hiding this comment.
| boxes: list of tuples of (lower_left_lon, lower_left_lat, upper_right_lon, upper_right_lat) | |
| bboxes: list of tuples of (lower_left_lon, lower_left_lat, upper_right_lon, upper_right_lat) |
| lines = [] | ||
|
|
||
| for line in multi_coordinates: | ||
| self.line(line) | ||
| lines.append(self.params.pop('line')) | ||
|
|
||
| self.params['line'] = lines |
There was a problem hiding this comment.
| lines = [] | |
| for line in multi_coordinates: | |
| self.line(line) | |
| lines.append(self.params.pop('line')) | |
| self.params['line'] = lines | |
| self.params["line"] = [ | |
| self.polygon(line).params.pop("line") for line in lines_ | |
| ] |
| """Filter by granules that overlap any series of connected points from an input list. | ||
|
|
||
| Parameters: | ||
| multi_coordinates: a list of lists of (lon, lat) tuples |
There was a problem hiding this comment.
| multi_coordinates: a list of lists of (lon, lat) tuples | |
| lines_: a list of lists of (lon, lat) tuples |
| circles = [] | ||
|
|
||
| for circle in multi_circles: | ||
| self.circle(*circle) | ||
| circles.append(self.params.pop('circle')) | ||
|
|
||
| self.params['circle'] = circles |
There was a problem hiding this comment.
| circles = [] | |
| for circle in multi_circles: | |
| self.circle(*circle) | |
| circles.append(self.params.pop('circle')) | |
| self.params['circle'] = circles | |
| self.params["circle"] = [ | |
| self.circle(*circle).params.pop("circle") for circle in circles_ | |
| ] |
| """Filter by granules that overlap any circle from an input list. | ||
|
|
||
| Parameters: | ||
| multi_circles: list of tuples of (lon, lat, radius) |
There was a problem hiding this comment.
| multi_circles: list of tuples of (lon, lat, radius) | |
| circles_: list of tuples of (lon, lat, radius) |
|
Thank you for reviewing it @chuckwondo. I'll wait to see what the other reviewers say before committing further changes - mainly the I set up the Regardless if it may require further changes to |
|
@mfisher87, @betolink, @jhkennedy, any of you able to look over this? I took a first pass, but I added a question to get input from others about how we want to approach something: https://github.com/nsidc/earthaccess/pull/1109/files#r2491099493. @tiagodc, in the meantime, would you mind merging the latest changes from the |
|
User tiagodc does not have permission to run integration tests. A maintainer must perform a security review of the code changes in this pull request and re-run the failed integration tests jobs, if the code is deemed safe. |
|
User tiagodc does not have permission to run integration tests. A maintainer must perform a security review of the code changes in this pull request and re-run the failed integration tests jobs, if the code is deemed safe. |
|
Hi all, any updates on the review status? @mfisher87, @betolink, @jhkennedy, @chuckwondo |
Apologies for the delay. @mfisher87, @betolink, @jhkennedy, I took an initial pass over this a while back, but included some questions for you in several comments regarding how we might want to deal with some query options in an intuitive manner. I'm not completely happy with what I initially suggested, which is why I'd like more eyes on this with perhaps some alternative suggestions. It's been a while since I looked at this myself, so I'll take another pass, hopefully with fresher eyes and perhaps a slightly different perspective. |
3 participants
This pull request adds multi-feature querying support for all spatial types. The python_cmr library already supports querying with multiple features simultaneously, which are parsed from list inputs in the
_build_urlfunction. This CMR functionality enables easy multipolygon [...] support with minimal code changes.I created five new methods in the
DataGranulesclass to enable explicit search of granules from multi-feature spatial inputs:multi_bounding_boxmultipolygonmultipointmulticirclemultilineThose functions run the single-feature methods internally and update the
paramsattribute inherited from theQueryclass of python_cmr, thus bulding single GET request URLs following the CMR standard.Tests are provided for all the new methods and the documentation was updated to illustrate how to use them.
I recently found out about
earthaccess. It's an amazing tool and I hope this PR will be useful :)📚 Documentation preview 📚: https://earthaccess--1109.org.readthedocs.build/en/1109/